A technology for managing interactions with UI elements.

Abstract

Several techniques described herein are for managing interactions with UI elements within the UI of individual applications managed by a system process. Such techniques can enable UI elements to be (1) displayed by a system process along with content provided by the individual application, and (2) interacted with by a user without the need to display the user interface of the individual application. In some examples, interaction with a UI element is provided to a system process, and through another system process, an action is identified and executed to update the UI element with different information provided by the individual application. Other techniques described herein can enable different endpoints of an application, such as different extensions or the application itself, to handle interactions with information provided by the application, depending on the current state of the electronic device, the application, and / or different extensions.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority to U.S. Non - Provisional Patent Application No. 18 / 613,426, entitled "TECHNIQUES FOR MANAGING INTERACTIONS WITH UI ELEMENTS", filed on March 22, 2024, and to U.S. Provisional Patent Application No. 63 / 464,536, entitled "TECHNIQUES FOR MANAGING INTERACTIONS WITH UI ELEMENTS", filed on May 5, 2023, which are hereby incorporated by reference in their entirety for all purposes.

Background Art

[0002] Electronic devices often include applications that provide information for display on the user interface of an application and / or via widgets of the application. To interact with the information from an application, a user typically needs to launch the application, wait for the application to launch, and navigate to the relevant section of the application. Thus, there is a need to improve techniques for interacting with applications.

Summary of the Invention

[0003] Current techniques for managing interaction with applications are generally ineffective and / or inefficient. For example, some techniques require the user to navigate the application's user interface. This disclosure provides a more effective and / or efficient technique for managing interaction with applications using an example of a widget on the lock screen of an electronic device. It should be recognized that other user interfaces may be used in conjunction with the techniques described herein. For example, the home screen, shared areas of the user interface, and / or notification centers may use the techniques described herein. In addition, the techniques described herein may optionally complement or replace other techniques for managing interaction with applications.

[0004] Several techniques for managing interaction with UI elements within a UI managed by a system process are described herein. While UI elements may correspond to individual applications, such techniques can enable UI elements to be (1) displayed by a system process along with content provided by the individual application, and (2) interacted with by a user without the need to display the user interface of the individual application. In some examples, interaction with a UI element is provided to a system process, and through another system process, an action is identified and executed to update the UI element with different information provided by the individual application. Such updates may occur without activating and / or displaying the user interface of the individual application, and in some examples, without communicating with the individual application. Other techniques described herein can enable different endpoints of an application, e.g., different extensions of the application (e.g., each having its own endpoint), or the application itself (e.g., having its own endpoint), to process interaction with information provided by the application depending on the current state of the electronic device, the application, and / or the different extensions. For example, the same interaction may be handled by either an extension of the application or the application itself, based on whether the application is currently running in the background.

[0005] Some examples describe methods performed by a system process of a computer system. In some examples, the method includes: receiving a first input corresponding to a user interface element of a user interface view while the computer system is displaying a user interface view managed by a system process; triggering an action corresponding to the user interface element, which is performed by an application corresponding to the user interface element, in response to receiving the first input; receiving a communication from the application after triggering the action, which corresponds to a change to the user interface view; and updating the display of the user interface view based on the communication, in response to receiving the communication.

[0006] Some examples describe a non-temporary computer-readable storage medium that stores one or more programs configured to run by one or more processors of a computer system running a system process. Some examples include instructions that, while the computer system is displaying a user interface view managed by a system process, receive a first input corresponding to a user interface element of a user interface view, and in response to receiving the first input, trigger an execution of an action corresponding to the user interface element, which is performed by an application corresponding to the user interface element, and after triggering the execution of the action, receive a communication from the application corresponding to a change to the user interface view, and in response to receiving the communication, update the display of the user interface view based on the communication.

[0007] Some examples describe a temporary computer-readable storage medium that stores one or more programs configured to run by one or more processors of a computer system running a system process. Some examples include instructions that, while the computer system is displaying a user interface view managed by a system process, receive a first input corresponding to a user interface element of the user interface view, and in response to receiving the first input, trigger an execution of an action corresponding to the user interface element, which is performed by an application corresponding to the user interface element, and after triggering the execution of the action, receive a communication from the application corresponding to a change to the user interface view, and in response to receiving the communication, update the display of the user interface view based on the communication.

[0008] In some examples, a computer system that runs a system process is described. In some examples, the computer system comprises one or more processors and memory that stores one or more programs configured to run by one or more processors. In some examples, one or more programs receive a first input corresponding to a user interface element of a user interface view while the computer system is displaying a user interface view managed by a system process, and in response to receiving the first input, the program includes instructions to trigger an execution of an action corresponding to the user interface element, which is performed by an application corresponding to the user interface element, and after triggering the execution of the action, to receive a communication from the application corresponding to a change to the user interface view, and in response to receiving the communication, to update the display of the user interface view based on the communication.

[0009] In some examples, a computer system that executes a system process is described. In some examples, the computer system includes means for performing each of the following steps: receiving a first input corresponding to a user interface element of a user interface view while the computer system is displaying a user interface view managed by a system process; triggering an execution of an action corresponding to the user interface element, which is performed by an application corresponding to the user interface element, in response to receiving the first input; receiving a communication from the application after triggering the execution of the action, which corresponds to a change to the user interface view; and updating the display of the user interface view based on the communication in response to receiving the communication.

[0010] Some examples describe computer program products. Some examples describe computer program products that include one or more programs configured to run on one or more processors of a computer system that runs a system process. Some examples include instructions that, while the computer system is displaying a user interface view managed by a system process, receive a first input corresponding to a user interface element of the user interface view, and in response to receiving the first input, trigger an execution of an action corresponding to the user interface element, which is performed by an application corresponding to the user interface element, and after triggering the execution of the action, receive a communication from the application corresponding to a change to the user interface view, and in response to receiving the communication, update the display of the user interface view based on the communication.

[0011] Some examples describe how a computer system application may perform actions. In some examples, the method includes, while the computer system is displaying a user interface view managed by a system process of the computer system, receiving a request from the system process for a user interface element of the user interface view, the user interface element performing an action corresponding to an interaction with the user interface element that corresponds to an application; performing the action in response to receiving the request for an action; and sending a request to the system process to modify the user interface view based on the request for an action.

[0012] Some examples describe a non-temporary computer-readable storage medium that stores one or more programs configured to run by one or more processors of a computer system running an application. Some examples include instructions that, while the computer system is displaying a user interface view managed by a system process of the computer system, receive requests from the system process to perform actions corresponding to user interface elements of the user interface view, which correspond to an application and interactions with the user interface elements, and in response to receiving a request to perform an action, send a request to the system process to perform the action and, based on the request to perform the action, modify the user interface view.

[0013] Some examples describe a temporary computer-readable storage medium that stores one or more programs configured to run on one or more processors of a computer system running an application. Some examples include instructions that, while the computer system is displaying a user interface view managed by a system process of the computer system, receive requests from the system process to perform actions corresponding to user interface elements of the user interface view, which correspond to an application and interactions with the user interface elements, and in response to receiving a request to perform an action, send a request to the system process to perform the action and, based on the request to perform the action, modify the user interface view.

[0014] In some examples, a computer system running an application is described. In some examples, the computer system comprises one or more processors and memory that stores one or more programs configured to run by one or more processors. In some examples, one or more programs include instructions that, while the computer system is displaying a user interface view managed by the system process of the computer system, receive requests from the system process to perform user interface elements of the user interface view, which correspond to an application, perform actions corresponding to interactions with the user interface elements, and, in response to receiving requests to perform actions, perform the actions and, based on the requests to perform actions, send requests to the system process to modify the user interface view.

[0015] In some examples, a computer system running an application is described. In some examples, the computer system includes means for performing each of the following steps: while the computer system is displaying a user interface view managed by a system process of the computer system, the computer system receives a request from a system process to perform an action corresponding to an interaction with a user interface element of the user interface view, the user interface element corresponding to an application; in response to receiving the request to perform an action, the computer system performs the action; and based on the request to perform an action, the computer system sends a request to the system process to modify the user interface view.

[0016] Some examples describe computer program products. Some examples describe computer program products that include one or more programs configured to run on one or more processors of a computer system running an application. Some examples include instructions that, while the computer system is displaying a user interface view managed by a system process of the computer system, receive requests from the system process to perform actions corresponding to user interface elements of the user interface view, which correspond to an application and interactions with the user interface elements, and in response to receiving requests to perform actions, send requests to the system process to perform actions and modify the user interface view based on the requests to perform actions.

[0017] Some examples describe methods that are performed by system processes in a computer system. Some examples include receiving an indication of a user interface element corresponding to an application; identifying an action corresponding to the user interface element in response to receiving the indication of the user interface element corresponding to the application; sending an instruction to perform the action to a first endpoint of the application in accordance with the determination that the application is in a first state in response to the identification of the action corresponding to the user interface element; and sending an instruction to perform the action to a second endpoint of the application, different from the first endpoint of the application, in accordance with the determination that the application is in a second state different from the first state.

[0018] Some examples describe a non-temporary computer-readable storage medium that stores one or more programs configured to run on one or more processors of a computer system running a system process. Some examples include a program that receives an indication of a user interface element corresponding to an application, identifies an action corresponding to the user interface element in response to receiving the indication of the user interface element corresponding to the application, sends an instruction to a first endpoint of the application to perform the action in response to the identification of the action corresponding to the user interface element, according to a determination that the application is in a first state, and sends an instruction to a second endpoint of the application, different from the first endpoint of the application, according to a determination that the application is in a second state different from the first state, to perform the action.

[0019] Some examples describe a temporary computer-readable storage medium that stores one or more programs configured to run on one or more processors of a computer system running a system process. Some examples include a program that receives an indication of a user interface element corresponding to an application, identifies an action corresponding to the user interface element in response to receiving the indication of the user interface element corresponding to the application, sends an instruction to a first endpoint of the application to perform the action in response to the identification of the action corresponding to the user interface element, according to a determination that the application is in a first state, and sends an instruction to a second endpoint of the application, different from the first endpoint of the application, according to a determination that the application is in a second state different from the first state, to perform the action.

[0020] In some examples, a computer system that runs a system process is described. In some examples, the computer system comprises one or more processors and memory that stores one or more programs configured to run by one or more processors. In some examples, one or more programs include instructions that receive an indication of a user interface element corresponding to an application, identify an action corresponding to the user interface element in response to receiving the indication of the user interface element corresponding to the application, send an instruction to a first endpoint of the application to perform the action in response to the identification of the action corresponding to the user interface element, according to a determination that the application is in a first state, and send an instruction to a second endpoint of the application, different from the first endpoint of the application, according to a determination that the application is in a second state different from the first state, to perform the action.

[0021] In some examples, a computer system that executes a system process is described. In some examples, the computer system includes means for performing each of the following steps: receiving an indication of a user interface element corresponding to an application; identifying an action corresponding to a user interface element in response to receiving an indication of a user interface element corresponding to an application; sending a command to a first endpoint of the application to perform the action in accordance with the determination that the application is in a first state in response to the identification of the action corresponding to the user interface element; and sending a command to perform the action to a second endpoint of the application, different from the first endpoint of the application, in accordance with the determination that the application is in a second state different from the first state.

[0022] Some examples describe computer program products. Some examples describe computer program products that include one or more programs configured to run on one or more processors of a computer system that runs system processes. Some examples include one or more programs that receive an indication of a user interface element corresponding to an application, identify an action corresponding to the user interface element in response to receiving the indication of the user interface element corresponding to the application, send an instruction to a first endpoint of the application to perform the action in response to the identification of the action corresponding to the user interface element, according to a determination that the application is in a first state, and send an instruction to a second endpoint of the application, different from the first endpoint of the application, according to a determination that the application is in a second state different from the first state, to perform the action.

[0023] Some examples describe how a computer system can perform an action. In some examples, the method includes, if the computer system application is in a first state, the first endpoint of the application receiving an instruction from a first system process of the computer system to perform an action, the action corresponding to a user interface element of a user interface view managed by a second system process of the computer system; and the first endpoint of the application performing the action in response to the first endpoint receiving the instruction to perform the action; and if the application is in a second state different from the first state, the second endpoint of the application receiving an instruction from a first system process to perform an action, the second endpoint of the application performing the action in response to the second endpoint receiving the instruction to perform the action.

[0024] Some examples describe non-temporary computer-readable storage media that store one or more programs configured to run by one or more processors of a computer system. In some examples, one or more programs include instructions that, when an application of a computer system is in a first state, receive an instruction from a first system process of the computer system by a first endpoint of the application to perform an action, the action corresponding to a user interface element of a user interface view managed by a second system process of the computer system, and perform the action in response to the first endpoint receiving the instruction to perform the action; and when the application is in a second state different from the first state, receive an instruction from a first system process by a second endpoint of the application, different from the first endpoint, to perform an action, and perform the action in response to the second endpoint receiving the instruction to perform the action.

[0025] In some examples, a temporary computer-readable storage medium is described that stores one or more programs configured to run by one or more processors of a computer system. In some examples, one or more programs include instructions that, when an application of a computer system is in a first state, receive an instruction from a first system process of the computer system by a first endpoint of the application to perform an action, the action corresponding to a user interface element of a user interface view managed by a second system process of the computer system, and perform the action in response to the first endpoint receiving the instruction to perform the action; and when the application is in a second state different from the first state, receive an instruction from a first system process by a second endpoint of the application, different from the first endpoint, to perform an action, and perform the action in response to the second endpoint receiving the instruction to perform the action.

[0026] Some examples describe a computer system comprising one or more processors and memory for storing one or more programs configured to run by one or more processors. In some examples, one or more programs include instructions such that, when the application of the computer system is in a first state, the first endpoint of the application receives an instruction from a first system process of the computer system to perform an action, the action corresponding to a user interface element of a user interface view managed by a second system process of the computer system, and the first endpoint of the application performs the action in response to the first endpoint receiving the instruction to perform the action; and when the application is in a second state different from the first state, the second endpoint of the application, different from the first endpoint, receives an instruction from the first system process to perform an action, and the second endpoint of the application performs the action in response to the second endpoint receiving the instruction to perform the action.

[0027] In some examples, a computer system is described. In some examples, the computer system performs the following steps: when an application of the computer system is in a first state, receiving, by a first endpoint of the application, an instruction to execute an action from a first system process of the computer system, the action corresponding to a user interface element of a user interface view managed by a second system process of the computer system; in response to the first endpoint receiving the instruction to execute the action, executing the action by the first endpoint of the application; when the application is in a second state different from the first state, receiving, by a second endpoint of the application, the second endpoint being different from the first endpoint, an instruction to execute an action from the first system process; and in response to the second endpoint receiving the instruction to execute the action, executing the action by the second endpoint of the application.

[0028] Some examples describe computer program products. Some examples describe computer program products that include one or more programs configured to run on one or more processors of a computer system. Some examples include one or more programs that, when an application of a computer system is in a first state, receive an instruction from a first system process of the computer system by a first endpoint of the application to perform an action, the action corresponding to a user interface element of a user interface view managed by a second system process of the computer system, and perform the action by the first endpoint of the application in response to the first endpoint receiving the instruction to perform the action; and when the application is in a second state different from the first state, receive an instruction from a first system process by a second endpoint of the application, which is different from the first endpoint, to perform an action, and perform the action by the second endpoint of the application in response to the second endpoint receiving the instruction to perform the action.

[0029] In some examples, a non - transient computer - readable storage medium storing a plurality of programs configured to be executed by one or more processors is described. In some examples, the plurality of programs include a first program corresponding to an application, where the first program is configured to receive user interaction data via a system process different from the plurality of programs, and in response to receiving the user interaction data, provide data for display on a user interface managed by the system process via the system process; and a second program corresponding to an extension of the application, where the second program is configured to execute separately from the first program via the system process, receive user interaction data, and in response to receiving the user interaction data, provide data for display on a user interface managed by the system process via the system process.

[0030] In some examples, a computer system storing a plurality of programs configured to be executed by one or more processors of the computer system is described. In some examples, the plurality of programs include a first program corresponding to an application, where the first program is configured to receive user interaction data via a system process different from the plurality of programs and, in response to receiving the user interaction data, provide data for display on a user interface managed by the system process via the system process; and a second program corresponding to an extension of the application, where the second program is configured to execute separately from the first program, and the second program is configured to receive user interaction data via the system process and, in response to receiving the user interaction data, provide data for display on a user interface managed by the system process via the system process.

[0031] Some examples describe non-temporary computer-readable storage media that store applications configured to run on one or more processors of a computer system. In some examples, the application is configured to receive user interaction data via a system process distinct from several programs, and, in response to receiving the user interaction data, provide data for display on a user interface managed by the system process via the system process, and to include instructions to be executed by a second program, the second program being configured to run separately from the application, the second program receiving user interaction data via the system process, and, in response to receiving the user interaction data, providing data for display on a user interface managed by the system process via the system process.

[0032] In some examples, a computer system running an application is described. In some examples, the application is configured such that it receives user interaction data through a system process distinct from multiple programs, and in response to the first program of the application receiving the user interaction data, it provides data for display on a user interface managed by the system process through the system process, and a second program, which is configured to run separately from the first program, contains instructions to be executed by the second program, which receives user interaction data through the system process, and in response to the second program receiving the user interaction data, it provides data for display on a user interface managed by the system process through the system process.

[0033] The executable instructions that perform these functions are optionally contained within a non-temporary computer-readable storage medium or other computer program product configured to be executed by one or more processors. [Brief explanation of the drawing]

[0034] For a better understanding of the various embodiments described, please refer to the following “Modes for Carrying Out the Invention” in conjunction with the drawings. Similar reference numbers refer to corresponding parts throughout those drawings.

[0035] [Figure 1] The Academy of Computational Systems, with several examples.

[0036] [Figure 2] This block diagram shows a device with interconnected subsystems, illustrating several examples.

[0037] [Figure 3] This block diagram shows a device containing an operating system and multiple applications, with several examples.

[0038] [Figure 4] This block, with several examples, shows the different behaviors that occur with different processes on the device to handle interaction with UI views.

[0039] [Figure 5] This block diagram shows the different behaviors that occur with respect to different device processes in order to contextually handle interactions with UI views, using several examples.

[0040] [Figure 6]This flowchart illustrates, with several examples, a method for a system process to handle user input corresponding to UI elements.

[0041] [Figure 7] This flowchart illustrates, with several examples, methods for applications to process user input corresponding to user interface elements.

[0042] [Figure 8] This flowchart illustrates, with several examples, how to send commands to different endpoints of an application.

[0043] [Figure 9] This flowchart illustrates, with several examples, how to process instructions for performing actions.

[0044] [Figure 10] This flowchart illustrates, with several examples, how to process user interaction data from the perspective of multiple programs.

[0045] [Figure 11] This flowchart illustrates, with several examples, how to process instructions for performing actions. [Modes for carrying out the invention]

[0046] The following description includes exemplary methods, parameters, etc. However, it should be noted that such descriptions are not intended to limit the scope of this disclosure, but rather are provided to describe exemplary embodiments.

[0047] The methods and / or processes described herein may include one or more steps that are conditional on one or more conditions being met. It should be understood that a method may be performed over multiple iterations of the same process, and different steps of the method may be met in different iterations. For example, if a method requires performing a first step in response to a determination that one or more criteria are met, and a second step in response to a determination that one or more criteria are not met, a person skilled in the art will understand that the steps of the method are repeated until both conditions are met, not in a specific order. Thus, a method described using conditional steps can be rewritten as a method repeated until each of the conditions described in the method is met. However, this is not required of a claim in a system or computer-readable medium if the claim includes instructions for performing one or more steps that are conditional on one or more conditions being met. Since instructions for a claim in a system or computer-readable medium are stored in one or more processors and / or one or more memory locations, a claim in a system or computer-readable medium includes logic that can determine whether one or more conditions have been met without explicitly repeating the steps of the method until all of the conditions on which the steps in the method are conditional are met. Those skilled in the art will also understand that, as with a method having conditional steps, a system or computer-readable storage medium may repeat the steps of the method as many times as necessary to ensure that all of the conditional steps have been performed.

[0048] In the following description, terms such as “first,” “second,” etc., are used to describe various elements, but these elements should not be limited by those terms. In some examples, these terms are used to distinguish one element from another. For example, without departing from the scope of the various described embodiments, a first subsystem may be called a second subsystem, and similarly, a subsystem device may be called a subsystem device. In some examples, the first subsystem and the second subsystem are two distinct references to the same subsystem. In some embodiments, the first subsystem and the second subsystem are both subsystems, but they are not identical subsystems or subsystems of the same type.

[0049] The terminology used in the descriptions of the various embodiments described herein is intended solely to describe specific embodiments and is not intended to be limiting. In the descriptions of the various embodiments and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless otherwise specified in the context. Furthermore, it should be understood that, as used herein, the terms "and / or" refer to and include any and all possible combinations of one or more of the enumerated items relating to the invention. It will be further understood that, as used herein, the terms "includes," "including," "comprises," and / or "comprising" specify the presence of the described functions, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other functions, integers, steps, actions, elements, components, and / or groups thereof.

[0050] The phrase "if" can be interpreted, at will, depending on the context, as meaning "when," "upon," "in response to determining," "in response to detecting," or "in accordance with a determination that." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" can be interpreted, at will, depending on the context, as meaning "upon determining," "in response to determining," "upon detecting [the stated condition or event]," "in response to detecting [the stated condition or event]," or "in accordance with the determination that [a stated condition or event] is."

[0051] Referring to Figure 1, a block diagram of the computing system 100 is shown. Computing system 100 is a non-limiting example of a computing system that can be used to perform the functions described herein. It should be noted that other computer architectures of computing systems can be used to perform the functions described herein.

[0052] In the illustrated example, the computing system 100 includes a processor subsystem 110 that communicates (e.g., wired or wirelessly) with memory 120 (e.g., system memory) and an I / O interface 130 via an interconnect 150 (e.g., a system bus, one or more memory locations, or other communication channels for connecting multiple components of the computing system 100). In addition, the I / O interface 130 communicates (e.g., wired or wirelessly) with an I / O device 140. In some examples, the I / O interface 130 is included together with the I / O device 140 so that the two constitute a single component. Note that there may be one or more I / O interfaces, and each I / O interface communicates with one or more I / O devices. In some examples, multiple instances of the processor subsystem 110 can communicate via the interconnect 150.

[0053] The computing system 100 may be any of various types of devices, including but not limited to a system-on-a-chip, a server system, a personal computer system (e.g., a smartphone, smartwatch, wearable device, tablet, laptop computer, and / or desktop computer), or a sensor. In some examples, the computing system 100 is included in or communicates with a physical component for the purpose of modifying the physical component in response to an instruction. In some examples, the computing system 100 receives an instruction to modify the physical component and causes the physical component to modify in response to the instruction. In some examples, the physical component is modified via an actuator, an electrical signal, and / or an algorithm. Examples of such physical components include acceleration control, brakes, gearboxes, hinges, motors, pumps, refrigeration systems, springs, suspension systems, steering control, vacuum systems, and / or valves. In some examples, a sensor includes one or more hardware components that detect information about the physical environment adjacent to (e.g., surrounding) the sensor. In some examples, the hardware components of a sensor include sensing components (e.g., image sensors or temperature sensors), transmitting components (e.g., laser or radio transmitters), receiving components (e.g., laser or radio receivers), or any combination thereof. Examples of sensors include angle sensors, chemical sensors, brake pressure sensors, contact sensors, non-contact sensors, electrical sensors, flow sensors, force sensors, gas sensors, humidity sensors, image sensors (e.g., camera sensors, radar sensors, and / or LiDAR sensors), inertial measurement units, leak sensors, level sensors, light detection and ranging systems, metal sensors, motion sensors, particle sensors, photoelectric sensors, position sensors (e.g., global positioning systems), precipitation sensors, pressure sensors, proximity sensors, radio detection and ranging systems, radiation sensors, speed sensors (e.g., measuring the velocity of an object), temperature sensors, time-of-flight sensors, torque sensors, and ultrasonic sensors. In some examples, a sensor includes a combination of multiple sensors. In some examples, sensor data is captured by fusing data from one sensor with data from one or more other sensors.Although Figure 1 shows a single computing system, computing system 100 can also be implemented as two or more computing systems working together.

[0054] In some examples, the processor subsystem 110 includes one or more processors or processing units configured to execute program instructions for performing the functions described herein. For example, the processor subsystem 110 can run an operating system, a middleware system, one or more applications, or any combination thereof.

[0055] In some examples, the operating system manages the resources of the computing system 100. Examples of operating system types covered herein include batch operating systems (e.g., Multiple Virtual Storage (MVS)), time-sharing operating systems (e.g., Unix), distributed operating systems (e.g., Advanced Interactive eXecutive (AIX)), network operating systems (e.g., Microsoft Windows Server), and real-time operating systems (e.g., QNX). In some examples, the operating system may include procedures, sets of instructions, software components, and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components. In some examples, the operating system uses a priority-based scheduler that assigns priorities to different tasks that the processor subsystem 110 can perform. In such examples, the priority assigned to a task is used to identify the next task to be performed. In some examples, the priority-based scheduler identifies the next task to be performed when the previous task has finished execution. In some examples, the highest-priority task is performed until completion unless another higher-priority task is ready.

[0056] In some examples, a middleware system provides one or more services or capabilities, or both, to an external application (e.g., one or more applications running on the processor subsystem 110) that is not provided by the operating system (e.g., data management, application services, messaging, authentication, API management, etc.). In some examples, a middleware system is designed so that a heterogeneous computer cluster provides hardware abstraction, low-level device control, implementation of commonly used functions, inter-process message passing, package management, or any combination thereof. Examples of middleware systems include Lightweight Communications and Marshalling (LCM), PX4, Robot Operating System (ROS), and ZeroMQ. In some examples, a middleware system uses a graph architecture to represent processes and / or operations, with processing taking place on nodes that can receive, post, and multiplex sensor data messages, control messages, status messages, planning messages, actuator messages, and other messages. In such examples, the graph architecture can define an application (e.g., an application running on the processor subsystem 110 as described above) such that different operations of the application are contained within different nodes in the graph architecture.

[0057] In some examples, messages sent from a first node in a graph architecture to a second node in the graph architecture are executed using a publish-subscribe model, where the first node publishes data on a channel that the second node can subscribe to. In such examples, the first node can store the data in memory (e.g., memory 120 or some local memory in the processor subsystem 110) and notify the second node that the data has been stored in memory. In some examples, the first node notifies the second node that the data has been stored in memory by sending a pointer (e.g., a memory pointer such as identifying a memory location) to the second node so that the second node can access the data from the location where the first node stored the data. In some examples, the first node sends the data directly to the second node so that the second node does not need to access memory based on the data received from the first node.

[0058] The memory 120 may include computer-readable media (e.g., non-temporary or temporary computer-readable media) available for storing program instructions executable by the processor subsystem 110 in order to cause the computing system 100 to perform the various operations described herein. For example, the memory 120 may store program instructions for performing functions related to methods 600, 700, 800, and 900 described below (e.g., Figures 6, 7, 8, and 9).

[0059] Memory 120 can be implemented using a variety of physical, non-temporary storage media, such as hard disk storage devices, floppy disk storage devices, removable disk storage devices, flash memory, random access memory (RAM-SRAM, EDO RAM, SDRAM, DDR SDRAM, RAMBUS RAM, etc.), and read-only memory (PROM, EEPROM, etc.). The memory of the computing system 100 is not limited to primary storage devices such as memory 120. The computing system 100 may also include other forms of storage devices, such as cache memory in the processor subsystem 110 and secondary storage devices on I / O devices 140 (e.g., hard drives, storage arrays, etc.). In some examples, these other forms of storage devices may also store program instructions executable by the processor subsystem 110 and perform the operations described herein. In some examples, the processor subsystem 110 (or each processor in the processor subsystem 110) includes a cache or other form of onboard memory.

[0060] The I / O interface 130 can be any type of interface configured to communicate with other devices. In some examples, the I / O interface 130 includes a bridge chip (e.g., a southbridge) from a front bus to one or more rear buses. The I / O interface 130 can communicate with one or more I / O devices (e.g., I / O device 140) via one or more corresponding buses or other interfaces. Examples of I / O devices include storage devices (hard drives, optical drives, removable flash drives, storage arrays, SANs, or their associated controllers), network interface devices (e.g., to a local or wide area network), sensor devices (e.g., cameras, radar, LiDAR, ultrasonic sensors, GPS, inertial measurement devices, etc.), and auditory or visual output devices (e.g., speakers, lights, screens, projectors, etc.). In some examples, the computing system 100 communicates with a network via a network interface device (e.g., configured to communicate via Wi-Fi®, Bluetooth®, Ethernet, etc.). In some examples, the computing system 100 is connected to a network either directly or via a wired connection.

[0061] Figure 2 shows a block diagram of device 200 having interconnected subsystems, in several examples. In the illustrated examples, device 200 includes three different subsystems (i.e., a first subsystem 210, a second subsystem 220, and a third subsystem 230) that communicate with each other (e.g., wired or wirelessly) to form a network (e.g., a personal area network, a local area network, a wireless local area network, a metropolitan area network, a wide area network, a storage area network, a virtual private network, an enterprise internal private network, a campus area network, a system area network, and / or a controller area network). An example of a possible computer architecture of subsystems like those included in Figure 2 is shown in Figure 1 (i.e., computing system 100). Although three subsystems are shown in Figure 2, device 200 may include more or fewer subsystems.

[0062] In some examples, some subsystems are not connected to other subsystems (for example, the first subsystem 210 can be connected to the second subsystem 220 and the third subsystem 230, but the second subsystem 220 cannot be connected to the third subsystem 230). In some examples, some subsystems are connected via one or more wires, while others are connected wirelessly. In some examples, messages are set up between the first subsystem 210, the second subsystem 220, and the third subsystem 230 so that when an individual subsystem sends a message, other subsystems receive the message (for example, via wire and / or bus). In some examples, one or more subsystems are wirelessly connected to one or more computing systems outside of device 200, such as a server system. In such examples, subsystems may be configured to communicate wirelessly with one or more computing systems outside of device 200.

[0063] In some examples, device 200 includes a housing that completely or partially encloses subsystems 210-230. Examples of device 200 include home appliances (e.g., refrigerators or air conditioning systems), robots (e.g., robotic arms or robotic vacuum cleaners), and vehicles. In some examples, device 200 is configured to navigate within a physical environment (with or without user input).

[0064] In some examples, one or more subsystems of device 200 are used to control, manage, and / or receive data from one or more other subsystems of device 200 and / or from one or more remote computing systems from device 200. For example, a first subsystem 210 and a second subsystem 220 could be cameras that capture images, and a third subsystem 230 could use the captured images for decision-making. In some examples, at least a portion of device 200 functions as a distributed computing system. For example, a task could be divided into different parts, with a first part performed by the first subsystem 210 and a second part performed by the second subsystem 220.

[0065] In this specification, “installed application” refers to a software application that has been downloaded to a computer system (e.g., computing system 100 and / or device 200) and is ready to be launched (e.g., opened) on the device. In some examples, a downloaded application becomes an installed application by an installation program that extracts the program portion from the downloaded package and integrates that extracted portion with the operating system of the computer system.

[0066] In this specification, the terms “open application” or “running application” refer to a software application that has retained state information (e.g., as part of the system / global internal state and / or application internal state). An open or running application is optionally one of the following types of applications: ● The active application (or foreground / active process) currently displayed on the display screen of the device in which the application is being used. ● Background applications (or background processes) that are not currently displayed but whose processes are handled by one or more processors, as well as ● An application that is not running but has state information stored in memory (volatile and non-volatile, respectively) that can be used to resume the execution of the application, either suspended or suspended.

[0067] In this specification, the term “closed application” refers to a software application that does not retain state information (for example, state information for a closed application is not stored in the device’s memory). Therefore, closing an application involves stopping and / or removing the application process for the application and removing the state information for the application from the device’s memory. Generally, opening a second application while a first application is running does not close the first application. When the second application is displayed and the first application is stopped from being displayed, the first application becomes a background application.

[0068] Here, we focus on techniques for managing interaction with UI elements corresponding to applications (e.g., user applications and / or non-system applications). Such techniques are described in the context of widgets on the lock screen of an electronic device. It should be noted that other user interface views may be used in conjunction with the techniques described herein. For example, a media selection interface on a television and / or a vehicle dashboard may use the techniques described herein. In addition, the techniques described herein may optionally complement or replace other techniques for managing interaction with UI elements corresponding to applications.

[0069] Figure 3 is a block diagram showing a device (e.g., device 300) containing an operating system (e.g., operating system 310) and multiple applications (e.g., a first application 360 and a second application 380), in several examples. In some examples, device 300 is a telephone, tablet, laptop, desktop computer, and / or wearable device. It should be noted that device 300 may contain more or fewer components than those shown in Figure 3. In addition, some components may (1) be divided into multiple components (for example, a UI renderer 330 may be divided into multiple system components that run within the operating system 310), (2) be combined with other components (for example, the UI view manager 340 may be combined with the intent manager 350), (3) and / or (4) be integrated into or outside the operating system 310, including within one of the multiple applications or the user space of device 300 (for example, the intent manager 350 may run outside the operating system 310 in one or each of the multiple applications, or as a user process outside the operating system 310 and outside the multiple applications).

[0070] As described above, device 300 includes an operating system 310. In some examples, the operating system 310 includes one or more embodiments of the operating system described above with respect to Figure 1. For example, the operating system 310 can manage the resources of device 300. In some examples, the operating system 310 manages interactions with the input and / or output devices of device 300, and / or interactions communicating with device 300. For example, the operating system 310 can detect inputs performed with respect to the input devices of device 300 (e.g., user input such as taps on touch-sensitive surfaces, clicks on mouse buttons, key presses on keyboards, and / or finger movements on touchpads) (e.g., events corresponding to inputs) and determine how to respond, including launching, activating, and / or notifying one or more applications and / or processes of device 300.

[0071] As shown in the diagram, the operating system 310 includes a UI host 320, a UI renderer 330, a UI view manager 340, and an intent manager 350. In some examples, the UI host 320 is a system process of the operating system 310 that manages the UI views provided by the operating system 310. Examples of UI views provided by the operating system 310 include a lock screen (e.g., a UI displayed while (1) device 300 is locked and triggering an output, and / or (2) before unlocking the lock screen) (e.g., a UI with limited functionality compared to other UIs of device 300 when device 300 is unlocked), a home screen (e.g., a UI displayed while device 300 is unlocked, having (1) representations of different applications that can be selected to display the UI of different applications, and / or (2) content provided by applications in the form of widgets), and / or a notification center (e.g., a UI that includes representations of events that have occurred with respect to different applications of device 300, where each representation includes content received by device 300 and / or provided by applications of device 300), which includes content received by device 300 and / or provided by one or more applications installed on device 300.

[0072] In some examples, the UI host 320 communicates with the UI renderer 330 to render UI elements for display. In some examples, in response to communication with the UI renderer 330, the UI host 320 receives and displays UI elements rendered by one or more output devices of device 300. In other examples, in response to communication with the UI host 320, the UI renderer 330 triggers the display of rendered UI elements by one or more output devices of device 300. In any set of the examples described above, the UI renderer 330 can combine different UI elements into a single UI prepared for display by device 330. In some examples, the UI host 320 and / or the UI renderer 330 communicate with the UI view manager 340 to retrieve at least some content provided by other applications on device 300. In such examples, the UI host 320 and / or the UI renderer 330 can create an area of ​​UI to contain content from another application and communicate with the UI view manager 340 to retrieve the content of the area of ​​UI.

[0073] In some examples, the UI view manager 340 communicates with applications (e.g., a first application 360, a first extension 370, a second application 380, and / or a second extension 390) to store archives corresponding to each application. In some examples, archives corresponding to individual applications are stored by the UI view manager 340 and can be used by the UI view manager 340 without starting, running, or displaying the user interfaces of the individual applications, and / or without communicating with the individual applications. For example, an archive may define a UI element in the form of a virtual button that, when selected, makes the UI element appear to be pressed down and / or changes the color of a different UI element. In such an example, the execution of the archive (including making the UI element appear to be pressed down and / or changing the color of a different UI element) can be performed without communicating with the individual applications, avoiding the need to run the individual applications. In some examples, an archive contains (1) instructions corresponding to widgets of an application and / or (2) instructions for displaying and / or rendering the UI of the widgets. In some examples, an archive contains a data structure used to render at least a portion of the UI view corresponding to the widgets. In some examples, the archive includes instructions (e.g., code and / or definitions of one or more actions for rendering UI elements via the UI host 320 and / or UI renderer 330), and / or rendered UI views for the UI elements corresponding to the archive (e.g., optionally rendered by the application and / or extensions of the application via the UI renderer 330). Such UI elements may include information about what should be displayed in response to the occurrence of different events (e.g., user interaction with the UI element, content received by device 300, and / or the passage of time). In some examples, the archive includes a timeline of events that define what of the different UI elements should be displayed and / or when they should be displayed.The event timeline may be part of a UI element, or / or may be separate from a UI element.

[0074] In some examples, the UI view manager 340 manages UI views corresponding to widgets (e.g., and / or other types of UI elements) of different applications. Such UI views can be displayed and / or managed by the UI view manager 340 (instead of requiring, for example, that the application for each widget is running when the content by the widget is displayed). For example, the UI view manager 340 can display the lock screen of device 300 having UI views of widgets corresponding to 10 different applications, without all 10 applications having to be running. In some examples, the lock screen of device 300 is an example of a live process, including a foreground process (e.g., a background process actively running on device 300 via a UI host 320 (e.g., the UI host 320 may be specific to the lock screen, just as different UI hosts are specific to different system UIs (e.g., the home screen and / or notification center))). In such examples, the applications (e.g., the first application 360, the first extension 370, the second application 380, and / or the second extension 390) do not need to be live processes (sometimes called suspended, paused, or closed processes) when the UI views managed by the UI view manager 340 are displayed, unless the UI view manager 340 requires information and / or updates from the individual applications, as will be further explained below. (e.g., they are not live processes.)

[0075] As described above, device 300 includes multiple applications. Such applications may correspond to different functions, organizations, and / or stakeholders. A non-exclusive list of application examples includes accessory applications (e.g., applications for controlling and / or viewing information of one or more accessories communicating with device 300), list applications (e.g., applications for viewing and modifying lists of data), media applications (e.g., applications for controlling and / or viewing media output (e.g., audio (e.g., recordings, music, and / or podcasts), video, and / or images)) (e.g., stored by device 300 and / or stored by remote devices communicating with device 300), and vehicle applications (e.g., applications for controlling and / or viewing information of a vehicle communicating with device 300).

[0076] In some examples, an application includes extensions (e.g., parts of the application and / or processes running separately from the application) to provide functionality not provided by the application or also provided by the application. In some examples, an extension includes instructions for (1) corresponding to widgets of the application and / or (2) displaying and / or rendering the UI of the widgets. In some examples, an extension generates data presented to the widgets and / or UI views. Other types of extensions that do not correspond to widgets are within the scope of this disclosure, and it should be noted that an extension refers to one or more sets of functionality. As shown in the figures, the first application 360 includes the first extension 370, and the second application 380 includes the second extension 390. In some examples, the first application 360 and / or the second application 380 are installed on device 300 and operate and / or run at least partially outside the operating system 310.

[0077] In some examples, the first application 360 and / or the second application 380 are installed via an online repository of installable applications (sometimes called an application store). In such examples, installing the first application 360 via the online repository may include installing the first extension 370 of the first application 360. In other examples, installing the first application 360 via the online repository may not include installing the first extension 370; instead, after the first application 360 is installed on device 300, the first application 360 downloads and / or installs the first extension 370.

[0078] Using the configuration described above with respect to Figure 3, techniques for handling interactions with UI views managed by the UI View Manager 340 are described herein. Such processing may be performed via the Intent Manager 350 and / or the UI View Manager 340.

[0079] In some examples, the intent manager 350 is used to handle interactions with UI views (e.g., UI elements of a UI view) using intents provided by the application. For example, an application (e.g., the application itself or an extension of the application) can define one or more actions that occur as a result of (e.g., when an event occurs, such as input being detected, a threshold being reached, and / or a certain amount of time elapsed). One or more actions may be arbitrary code provided by the application to be executed by a system process (e.g., the UI view manager 340 and / or the intent manager 350). Such code may (1) cause a system process to execute one or more instructions, and / or (2) notify the application and / or endpoints of different applications (e.g., a particular function). In some examples, one or more actions may (1) be executed without needing to communicate with the application (e.g., the application that provided the one or more actions, and / or another application identified by the one or more actions), and / or (2) without running the application before, during, and / or after executing the one or more actions. In some examples, one or more actions defined by a first application may be used by a second application distinct from the first application. For example, the first application may define an action to turn on a light associated with the first application, and this action may be callable and / or usable by the second application. In some examples, intents provided by an application may be defined semantically so that the UI view manager 340 and / or intent manager 350 interpret the intent and identify one or more actions to be performed on device 300 to satisfy the intent. In such examples, the intent is defined in a syntax that can be interpreted by the UI view manager 340 and / or intent manager 350.In some examples, the intents described above function in configurations where the display of a UI view is separated from the processing of actions associated with the UI view. For example, a first device may be provided with an archive corresponding to a UI view (e.g., as described above), and when an interaction and / or event occurs with respect to the UI view, the first device communicates with a second device managing the UI view to identify one or more actions to be performed. In such examples, notifications of inputs and / or events are communicated between the devices, and other actions described herein occur as described herein, except that the results are sent from the second device to the first device. In some examples, when the second device manages a UI view displayed on the first device, the first device can handle some interactions with the UI view without communicating with the second device. For example, some interactions can correspond to actions that can be resolved without the first device needing to communicate with the second device (for example, the first device and / or the second device can disable certain actions (e.g., outputting media on the first device) and / or enable certain actions (e.g., opening a web page on the browser of the first device)).

[0080] In some examples, the intent manager 350 stores a mapping of UI elements, UI input types (e.g., tap, select, swipe, tap and hold, rotate, double tap, etc.), and / or applications (e.g., first application 360, first extension 370, second application 380, and / or second extension 390) to identify the application that communicates (1) an action defined by the application, and / or (2) the occurrence of a detected input and / or an action defined by the application. In some examples, the UI view manager 340 sends a request for the result of the mapping to the detected input in order to receive the result of the mapping. In other examples, the UI view manager 340 sends a request for the intent manager 350 to resolve the detected input without the intent manager 350 having to resolve the detected input through the UI view manager 340. In some examples of the above description, the first virtual button may be mapped to a first operation of the first extension 370, so that when an input is detected with respect to the first virtual button, the first operation provided by the first extension 370 is performed and / or the first extension 370 is notified. In some examples of the above description, if the detected input is of a first type (e.g., a tap input rather than a non-tap input), the second virtual button may be mapped to a second operation of the first extension 370, so that when an input of the first type is detected with respect to the second virtual button, the second operation provided by the first extension 370 is performed and / or the first extension 370 is notified. The results of such interactions will be further described below with reference to Figures 4 and 5.

[0081] Figure 4 is a block diagram illustrating different operations that occur with respect to different processes (e.g., one or more processes described above with respect to Figure 3) of a device (e.g., computing system 100, device 200, and / or device 300) for handling interaction with a UI view, according to several examples. In the block diagrams of Figures 4 and 5, the vertical lines of the different processes represent the operations performed for each process. For example, 404 is performed by the UI renderer 330, 408 is sent by the UI renderer 330 and received by the UI view manager 340, and 422 includes one or more communications between the extension 402 and the UI view manager 340.

[0082] As shown in the figure, the process of the device in Figure 4 includes a UI renderer 330, a UI view manager 340, an intent manager 350, and extensions 402 (e.g., a first application 360, a first extension 370, a second application 380, and / or a second extension 390). As described above, it should be noted that one or more processes can be combined and / or divided into multiple processes, and / or one or more processes can run on a device different from its own.

[0083] In some examples, the behavior shown in Figure 4 occurs after the extension 402 has defined one or more actions to be performed by the UI view manager 340 and / or intent manager 350 in response to detecting a particular type of input. Such definitions may be (1) provided to the UI view manager 340 and / or intent manager 350 and (2) used to determine how to respond to the input.

[0084] In 404, the UI renderer 330 (and / or the UI host 320, as described above) renders (e.g., generates and / or displays) a UI view. In some examples, the UI view includes parts corresponding to the extension 402 (e.g., one or more UI elements) (sometimes referred to as the UI view and / or UI elements corresponding to the extension 402). In some examples, the UI view is managed by a system process of the device including the UI renderer 330 (e.g., the UI host 320 and / or UI view manager 340, as described above). For example, the UI view could be the device's lock screen, home screen, notification center, and / or parts thereof, and the UI elements could correspond to one or more different applications and / or operating systems (e.g., operating system 310).

[0085] In 406, the UI renderer 330 (and / or the UI host 320 as described above) detects input to the UI view (e.g., user input such as tap or non-tap input). In 408, in some examples, upon detecting input, the UI renderer 330 (and / or the UI host 320 as described above) notifies the UI view manager 340 of the input. In other examples, the UI renderer 330 (and / or the UI host 320 as described above) notifies the UI view manager 340 of the input upon determining that the input is valid input with respect to the UI view (e.g., intended, directed, and / or exceeds a threshold (e.g., time, length, and / or distance)). Figure 4 shows the UI renderer 330 notifying the UI view manager 340, but it should be noted that in some examples the UI view manager 340 is not notified by the UI renderer 330, and instead the intent manager 350 is notified by the UI renderer 330.

[0086] In some examples, at 410, the UI view manager 340 responds to the notification at 408. In such examples, the response may include a request to update the display of the UI view based on the input. In some examples, the response is sent without requiring (1) notification to the intent manager 350 and / or (2) communication with the extension 402. In such examples, the response is processed by the UI view manager 340 using one or more actions defined by the extension 402 and provided to the UI view manager 340 as described above. In examples where the response is processed by the UI view manager 340, 414-426 do not occur. In other examples, the response is sent by the UI view manager 340 before, during (e.g., at least partially in parallel with), and / or after notifying the intent manager 350 about the input and / or communicating with the extension 402. In some examples, in 412, upon receiving a request to update the display of the UI view, the UI renderer (and / or UI host 320 as described above) renders (e.g., generates and / or displays) the updated UI view. In examples where the intent manager 350 and / or extension 402 are communicated by the UI view manager 340, the updated UI view may be updated again depending on the results of actions performed by the intent manager 350 and / or extension 402 (e.g., as shown in 426). In such examples, the updated UI view is executed optimistically, assuming that one or more actions corresponding to the input will resolve successfully (e.g., before one or more actions are performed).

[0087] In 414, upon receiving a notification in 408 (e.g., identification of the application corresponding to the UI element to which the input is directed), the UI view manager 340 notifies the intent manager 350 of the input. In some examples, the notification to the intent manager 350 includes identification of the application corresponding to the input (e.g., extension 402), identification of the UI element corresponding to the input, and / or identification of the type of input of the input.

[0088] In 416, upon notification of input, the intent manager 350 determines an action corresponding to the input (for example, including and / or representing one or more actions defined by extension 402). In some examples, the action is determined using mapping as described above.

[0089] In 418, depending on the decision to take an action (for example, depending on whether the action requires communication with the extension 402), the intent manager 350 sends identification information for the action to the extension 402. In some examples, the identification includes information in a notification provided to the intent manager 350 (for example, as described above) and / or information defined within the action. In some examples, the identification of the action is performed based on how the intent manager 350 communicates with the extension 402 (for example, different methods are used for different actions and / or different endpoints are used for different actions so that the communication does not need to include identification of the action).

[0090] In 420, upon receiving an action identification, the extension 402 performs one or more actions corresponding to the action. In some examples, to perform one or more actions, the extension 402 starts execution and / or transitions to an active state (e.g., as a background process, not a foreground process in some examples) (e.g., the extension 402 was not executing before receiving the action identification, while the UI view was being rendered, and / or while input was being detected). In some examples, such actions require the extension 402 to communicate with other extensions and / or other applications on the device containing the extension 402 and / or on different devices.

[0091] In 422, in response to having performed at least one of one or more actions (and, in some examples, all of one or more actions), the extension 402 communicates about the action. In some examples, such communication includes the extension 402 indicating the result of at least one of the one or more actions (e.g., whether the action was successful or not). In some examples, such communication includes a request to update the archive and / or UI view stored by the UI view manager 340. In such examples, the request to update the archive may be a first communication to the UI view manager 340 in response to having performed at least one of one or more actions. In other examples, the request to update the archive is a response to a request from the UI view manager 340 for an update to the archive. For example, the extension 402 may send a communication to the UI view manager 340 indicating that the extension 402 needs to update the archive, and in response to receiving the communication, the UI view manager 340 may send a request to the extension 402 for an update to the archive. In some examples, an update to the archive and / or UI view may be a first communication to the UI view manager 340 in response to performing at least one of one or more actions. In some examples, the update may include updating the appearance of UI elements (e.g., changes to the set of instructions and / or updated rendering of UI elements) and / or UI views, adding additional UI elements and / or UI views that were not previously provided, deleting previously provided UI elements and / or UI views, and / or updating the timeline corresponding to the archive to identify when and / or what should be displayed.

[0092] In 424, the UI view manager sends a request to the UI renderer 330 (and / or UI host 320) to render a new and / or different UI view. The new and / or different UI view may be based on the update received in 422 and / or events occurring with respect to the currently displayed UI view that are not based on the update received in 422. In some examples, the new and / or different UI view is rendered in response to the update received in 422. In other examples, the new and / or different UI is rendered after the update received in 422, in response to the UI view manager 340 determining that an event has occurred to change the UI view. In 426, the UI renderer 330 renders (e.g., generates and / or displays) the updated UI view in accordance with the request in 424.

[0093] Figure 5 is a block diagram illustrating different behaviors that occur with respect to different processes (e.g., one or more processes described above with respect to Figure 3) of a device (e.g., computing system 100, device 200, and / or device 300) for contextually handling interaction with a UI view, according to several examples.

[0094] As shown in the figure, the process of the device in Figure 5 includes a UI renderer 330, a UI view manager 340, an intent manager 350, an application 502 (e.g., a first application 360 and / or a second application 380), and an extension 504 (e.g., a first extension 370 and / or a second extension 390). In some examples, the extension 504 corresponds to application 502, such that the extension 504 is an extension of application 502. As described above, it should be noted that one or more processes can be combined and / or divided into multiple processes, and / or one or more processes can run on a device different from its own.

[0095] Similar to Figure 4, in some examples, the behavior shown in Figure 5 occurs after the extension 402 has defined one or more actions to be performed by the UI view manager 340 and / or intent manager 350 in response to detecting a particular type of input. Also, similar to Figure 4, 506 (corresponding to 404), 508 (corresponding to 406), 510 (corresponding to 408), and 512 (corresponding to 414) in Figure 5 may occur in the same or similar manner as described above with respect to Figure 4. In addition, Figure 5 does not include the actions corresponding to 410 and 412, but such actions may occur in the same or similar manner in Figure 5 as described above with respect to Figure 4.

[0096] In 514, the intent manager 350 (or, in some examples, the UI view manager 340) determines the action corresponding to the input (for example, in the same or similar manner as described above with respect to 416 in Figure 4), and also determines the state of application 502 and / or the device containing application 502. In some examples, the state of application 502 and / or the device containing application 502 influences, controls, and / or is used to determine whether an action indication is sent to application 502 (which may be done in the same or similar manner as described above with respect to 418, as shown by 516) or extension 504 (which may be done in the same or similar manner as described above with respect to 418, as shown by 522). In such examples, the action indication is not sent to other components (for example, the action indication is not sent to extension 504 when it is sent to application 502, and the action indication is not sent to application 502 when it is sent to extension 504).

[0097] In some examples, determining the state includes determining whether application 502 is currently a foreground, background, suspended, paused, or closed process. In such examples, depending on the configuration of intent manager 350 (for example, with respect to application 502, intent manager 350 is configured to communicate with application 502 when application 502 is a foreground or background process but is not a suspended, paused, or closed process), intent manager 502 sends an identification of the action to either application 502 or extension 504. In some examples, determining the state includes determining whether application 502 and / or extension 504 are specifically configured to process the action. In such examples, depending on the configuration of intent manager 350 (for example, with respect to application 502 and / or extension 504, intent manager 350 is configured to communicate with application 502 when performing the action), intent manager 502 sends an identification of the action to either application 502 or extension 504.

[0098] In some examples, determining the state includes determining the amount of unused computing resources on the device containing application 502. In such examples, depending on the configuration of intent manager 350 (for example, with respect to application 502, intent manager 350 is configured to communicate with application 502 when the device is idle and / or has at least a default amount of unused computing resources), intent manager 350 sends an identification of an action to either application 502 or extension 504. In some examples, determining the state includes determining the battery level of the device containing application 502 and / or whether the device is currently charging. In such examples, depending on the configuration of intent manager 350 (for example, with respect to application 502, intent manager 350 is configured to communicate with application 502 when the device has at least a default amount of battery remaining and / or is currently charging), intent manager 502 sends an identification of an action to either application 502 or extension 504. It should be noted that other types of states can be used to determine whether to send an action indication to application 502 or to extension 504. In some examples, an action indication may be sent to both application 502 and extension 504, depending on the configuration of intent manager 350 (e.g., with respect to application 502 and / or extension 504).

[0099] In some examples, the configuration of the intent manager 350 (as described above regarding whether to send it to application 502 and / or extension 504) is included in the information provided by extension 504 and / or application 502 when establishing an input to an action or a mapping of input types.

[0100] After sending the action indication, 518 (corresponding to 420), 524 (corresponding to 420), 520 (corresponding to 422), 526 (corresponding to 422), 528 (corresponding to 424 and based on communications received by the UI View Manager 340 (for example, in 526 or 520, or the most recent if sent by both application 502 and extension 504)), and 530 (corresponding to 426) in Figure 5 may occur in the same or similar manner as described above with respect to Figure 4.

[0101] Figure 6 is a flowchart illustrating a method (e.g., Method 600) for a system process to process user input corresponding to user interface elements, using several examples. Some operations in Method 600 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0102] In some examples, method 600 is executed in a system process of a computer system (e.g., computing system 100, device 200, and / or device 300) (e.g., an operating system process such as a daemon that manages the interaction between the computer system's renderer and one or more applications) (e.g., UI view manager 340 and / or intent manager 350). In some examples, the computer system is a phone, tablet, laptop, desktop computer, and / or wearable device.

[0103] In 602, while the computer system is displaying a user interface view (e.g., a widget and / or one or more graphical user interface elements) managed by a system process (e.g., directed by a system process, associated with a system process, and / or corresponding to a system process), the system process receives a first input (e.g., tap input or non-tap input such as swipe input, gaze input, mouse click, air gesture, and / or hold-and-press input) (e.g., in 406 or 508) corresponding to (e.g., directed to, located in, and / or relative to) the user interface elements (e.g., icons, virtual buttons, virtual sliders, and / or toggles) of the user interface view (e.g., managed by a system process as described above) (e.g., while the computer system is displaying the user interface view). In some examples, the first input is received from the computer system's renderer (e.g., UI renderer 330). In some examples, the first input is received from a host (e.g., UI host 320) configured to control what is displayed in the user interface managed by the computer system's operating system (e.g., operating system 310). In some examples, the user interface view is displayed by the computer system's renderer (e.g., UI renderer 330), and a system process instructs the renderer what to display. In some examples, the system process causes the user interface view to be displayed. In some examples, before the computer system displays the user interface view, the user interface view is stored by the system process to be displayed according to a determination that the display criteria have been met. In some examples, a process other than the system process (e.g., a different system process, and / or a process that is not a system process but a user process (e.g., an application process)) causes the user interface view to be displayed.In some examples, the user interface view is displayed (1) on the computer system's lock screen (for example, while the computer system is locked and has fewer accessible functions than when it is unlocked), (2) on the computer system's home screen (for example, while the computer system is displaying one or more user interface elements that, when selected, display the user interface of the corresponding application), (3) within the individual user interface of a separate application running on the computer system that is different from the application to which the user interface view is displayed, and / or (4) within the computer system's notification user interface (for example, while the computer system is displaying one or more notifications received by the computer system from one or more processes (for example, a process running on the computer system and / or a process running on a different computer system)).

[0104] In 604, while the computer system is displaying a user interface view managed by a system process, upon receiving a first input, the system process triggers the execution of an action (e.g., an action and / or a set of instructions corresponding to the action) corresponding to a user interface element (e.g., in 420, 518, or 524), the execution of which is performed by an application corresponding to the user interface element (e.g., in the foreground or background while the user interface view is being displayed simultaneously) (e.g., a separate application having one or more graphical user interface elements that can be displayed by the computer system (e.g., installed via an application stored in the computer system), or an extension of the separate application (e.g., part of the separate application configured to run separately from the separate application, but in some examples installed together with the separate application (e.g., via a single installation request)) (e.g., first application 360, first extension 370, second application 380, second extension 390, extension 402, application 502, or extension 504). In some examples, triggering the execution of an action involves identifying an application and sending a command to a second system process distinct from the system process (e.g., UI View Manager 340 and / or Intent Manager 350) to identify each action corresponding to a user interface element (and, in some examples, corresponding to the type of input of the first input). In some examples, triggering the execution of an action involves identifying an application and, in accordance with the identification of the action, identifying the action by looking up the action corresponding to the user interface element (and, in some examples, corresponding to the type of input of the first input). In some examples, the action is provided by the application. In some examples, the identification information of the action, which has the identification information of the user interface element, is provided to the system process by the application before the computer system displays the user interface view.

[0105] In 606, while the computer system is displaying a user interface view managed by a system process, and after triggering the execution of an action, the system process receives communications (e.g., messages, requests, and / or instructions) from the application corresponding to changes to the user interface view (e.g., in 422, 520, or 526) (e.g., visual changes and / or changes to the appearance of the user interface view) (e.g., while the computer system is displaying the user interface view). In some examples, the communications include updates to the user interface view. In some examples, the communications include a new user interface view to replace the existing one. In some examples, after sending the communications, the application stops execution.

[0106] In 608, while the computer system is displaying a user interface view managed by a system process, the system process updates the display of the user interface view based on the communication received (for example, in 424 or 528). In some examples, updating the user interface view involves a request to display the updated user interface view to the renderer and / or host mentioned above.

[0107] In some examples, the operation involves modifying the state of a second computer system distinct from the computer system (e.g., accessories, output devices, display generation components, and / or vehicles) or modifying local variables of an application. In some examples, the second computer system has a corresponding application that enables the state of the second computer system to be modified by the computer system. In some examples, the computer system communicates with the second computer system (e.g., via wired and / or wireless connections). In some examples, the second computer system is remote from and / or separate from the computer system.

[0108] In some examples, user interface elements include toggles (for example, (1) controls that graphically represent different possible states of a toggle (e.g., two or more) and (2) the current state of the toggle, and which visually change to graphically represent a new state of the toggle's current state in response to input from the computer system). In some examples, a toggle is a binary control configured to be used to change between two different states.

[0109] In some examples, while the computer system is displaying a user interface view managed by a system process, the system process receives a second input corresponding to a second user interface element of the user interface view. In some examples, the second user interface element is different from the user interface element. In some examples, while the computer system is displaying a user interface view managed by a system process, in response to receiving the second input, the system process triggers the execution of a second action corresponding to the second user interface element. In some examples, the execution of the second action is performed by an application. In some examples, the second action is different from the action. In some examples, while the computer system is displaying a user interface view managed by a system process, in response to receiving the second input, the system process updates the display of the user interface view (e.g., in 410 and 412) (e.g., the display of the second user interface element and / or the user interface view) (e.g., based on receiving the second input) (e.g., according to the expected outcome of the execution of the second action) (e.g., at least partially in parallel with triggering the execution of the second action) (e.g., in the system process). In some examples, the second user interface element is updated before the second operation is completed, before communication is received from the application based on the second input, and / or before the state indicated by the updated display of the user interface view is established. In some examples, the user interface view is updated before the second operation is performed. In some examples, the command to update the user interface view is sent before the command to perform the second operation is sent. In some examples, after updating the display of the user interface view, the computer system updates the display of the user interface view (e.g., in a system process) to revert to the state before the display of the user interface view was updated (e.g., immediately and / or immediately before), in accordance with the determination that the execution of the second operation failed.

[0110] In some examples, the application is inactive (e.g., not running in the background and / or foreground, and / or suspended) before triggering the execution of an action (e.g., immediately before and / or within a certain amount of time before (e.g., 10 seconds to 10 minutes)). In some examples, triggering the execution of an action includes transitioning the application to an active state (e.g., starting execution, starting execution in the background and / or foreground, and / or transitioning to an unpaused state).

[0111] In some examples, triggering an action involves starting an application in the background of the computer system (e.g., running and / or operating it) (e.g., changing it from an inactive state to an active state). In some examples, starting in the background does not involve displaying the application's user interface.

[0112] In some examples, changes to the user interface view include changes to the timeline of how the user interface view is updated in response to the occurrence of one or more events.

[0113] In some examples, changes to the user interface view include the rendered portion of the user interface view. In some examples, the rendered portion is used to update the display of the user interface view. In some examples, updating the display of the user interface view includes displaying the rendered portion.

[0114] In some examples, while the computer system displays a second user interface view simultaneously with the user interface view, a system process receives a third input corresponding to the second user interface view. In some examples, the second user interface view is managed by a system process. In some examples, the second user interface view is distinct from the user interface view. In some examples, while the computer system displays the second user interface view simultaneously with the user interface view and in response to receiving the third input, a system process triggers the execution of a third action corresponding to the second user interface view. In some examples, the execution of the third action is performed by a second application distinct from the application (e.g., the first application 360, the first extension 370, the second application 380, the second extension 390, extension 402, application 502, or extension 504). In some examples, the second application corresponds to the second user interface view. In some examples, while the computer system is displaying a second user interface view simultaneously with the user interface view, and after triggering the execution of a third action, the system process receives a second communication from the second application corresponding to changes in the second user interface view. In some examples, while the computer system is displaying the second user interface view simultaneously with the user interface view, and in response to receiving the second communication, the system process updates the display of the second user interface view based on the second communication.

[0115] In some examples, while the computer system is displaying a user interface view managed by a system process, the system process receives a fourth input corresponding to a third user interface element of the user interface view. In some examples, the third user interface element is different from the user interface element. In some examples, while the computer system is displaying a user interface view managed by a system process, in response to receiving the fourth input, the system process triggers the execution of a fourth action corresponding to the third user interface element. In some examples, the execution of the fourth action is performed by the application. In some examples, while the computer system is displaying a user interface view managed by a system process, and after triggering the execution of the fourth action, the system process receives a third communication from the application corresponding to a second change to the user interface view. In some examples, the second change to the user interface view is different from the change to the user interface view. In some examples, while the computer system is displaying a user interface view managed by a system process, in response to receiving the third communication, the system process updates the display of the user interface view based on the third communication. In some examples, updating the display of the user interface view based on the third communication includes updating a third user interface element.

[0116] It should be noted that the process details described above in relation to Method 600 (for example, Figure 6) are also applicable to similar methods to those described herein. For example, Method 700 may optionally include one or more characteristics of the various methods described above with reference to Method 600. For example, the system process of Method 600 may be the system process of Method 700. For brevity, these details will not be repeated below.

[0117] Figure 7 is a flowchart illustrating, with several examples, a method for an application to process user input corresponding to user interface elements (e.g., Method 700). Some operations of Method 700 are arbitrarily combined, the order of some operations is arbitrarily changed, and some operations are arbitrarily omitted.

[0118] In some examples, Method 700 is executed on an application of a computer system (e.g., computing system 100, device 200, and / or device 300) (e.g., a separate application having one or more graphical user interface elements that can be displayed by the computer system (e.g., installed via an application stored on the computer system), or an extension of a separate application (e.g., a part of a separate application configured to run separately from the separate application, but in some examples installed together with the separate application (e.g., via a single installation request))) (e.g., a first application 360, a first extension 370, a second application 380, a second extension 390, an extension 402, an application 502, or an extension 504). In some examples, the computer system is a telephone, a tablet, a laptop, a desktop computer, or a wearable device.

[0119] In 702, while the computer system is displaying a user interface view (e.g., a widget and / or one or more graphical user interface elements) managed by the computer system's system processes (e.g., operating system processes such as daemons that manage the interaction between the computer system's renderer and one or more applications) (e.g., UI View Manager 340 and / or Intent Manager 350), the application receives a request from the system processes to perform an action (e.g., an action and / or a set of instructions corresponding to the action) corresponding to an interaction (e.g., tap input or non-tap input, e.g., swipe input, gaze input, mouse click, air gesture, and / or hold-and-press input) with the user interface elements of the user interface view (e.g., the displayed elements and / or the area of ​​the user interface view) (e.g., managed by the system processes as described above), and the user interface elements correspond to the application (of the application, about the application, part of the application, and / or composed of the application). In some examples, the identification of the user interface elements and / or the rendered version of the user interface elements are sent from the application to the system processes before the user interface view is displayed. In some examples, interaction with user interface elements is managed by the application. In some examples, the user interface view is displayed by the computer system's renderer (e.g., UI renderer 330), and the system process instructs the renderer what to display. In some examples, the system process causes the user interface view to be displayed. In some examples, before the computer system displays the user interface view, the user interface view is stored by the system process so that it can be displayed according to the determination that the display criteria have been met.In some cases, a process different from the system process (e.g., a different system process, and / or a process that is not a system process but a user process (e.g., an application)) may display a user interface view. In some cases, the user interface view may be displayed (1) on the computer system's lock screen (e.g., while the computer system is locked, with fewer functions accessible than when it is unlocked), (2) on the computer system's home screen (e.g., while the computer system is displaying one or more user interface elements that, when selected, display the user interface of the corresponding application), (3) within the individual user interface of a separate application running on the computer system, and / or (4) within the computer system's notification user interface (e.g., while the computer system is displaying one or more notifications received by the computer system from one or more processes (e.g., a process running on the computer system and / or a process running on a different computer system)).

[0120] In 704, while the computer system is displaying a user interface view managed by the computer system's system processes, the application receives a request to perform an action, and in response, performs the action (for example, in 420 or 518).

[0121] In 706, while the computer system is displaying a user interface view managed by the computer system's system processes, the application sends a request to the system processes (for example, in 422, 520, or 526) to modify the user interface view (e.g., visual changes and / or changes to the appearance of the user interface view) based on the request to perform an action (e.g., receiving the request to perform an action and / or performing the action results in changes to the user interface view). In some examples, the modification request includes updating the user interface view. In some examples, the modification request includes a new user interface view to replace the existing one. In some examples, after sending the modification request, the application stops execution (e.g., in the background).

[0122] In some examples, the operation involves modifying the state of a second computer system distinct from the computer system (e.g., accessories, output devices, display generation components, and / or vehicles) or modifying local variables of an application. In some examples, the second computer system has a corresponding application that enables the state of the second computer system to be modified by the computer system. In some examples, the computer system communicates with the second computer system (e.g., via wired and / or wireless connections). In some examples, the second computer system is remote from and / or separate from the computer system.

[0123] In some examples, upon receiving a request to perform an action, the application transitions from an inactive state (as described above with respect to method 600) to an active state (as described above with respect to method 600) before performing the action.

[0124] In some examples, transitioning from an inactive state to an active state involves starting the application in the background of the computer system. In some examples, starting the application in the background does not involve displaying the application's user interface. In some examples, the application performs its actions in the background of the computer system and / or transitions back to an inactive state without starting in the foreground of the computer system.

[0125] In some examples, a request to change the user interface view involves changing the request to a timeline of how the user interface view is updated in response to the occurrence of one or more events.

[0126] In some examples, a request to modify the user interface view includes the rendered portion of the user interface view. In some examples, the rendered portion is used to update the display of the user interface view. In some examples, updating the display of the user interface view includes displaying the rendered portion.

[0127] In some examples, the display of a user interface element changes based on interaction with the user interface element during and independently of the execution of an action. In some examples, a user interface element is updated before an action is completed, before a request is sent to change the user interface view based on a request to perform an action, and / or before the state indicated by the change is established by the application. In some examples, a user interface element is changed before an action is performed. In some examples, a command to change the display of a user interface element is sent before a request to perform an action is sent. In some examples, after changing the display of a user interface element, the computer system updates the display of the user interface view (e.g., in a system process) to revert to the state before the update (e.g., immediately and / or immediately prior) in accordance with a determination that the execution of the action failed.

[0128] It should be noted that the process details described above in relation to Method 700 (for example, Figure 7) are also applicable in ways similar to other methods described herein. For example, Method 600 optionally includes one or more characteristics of the various methods described above with reference to Method 700. For example, an application of Method 700 may be an application of Method 700. For brevity, these details will not be repeated below.

[0129] Figure 8 is a flowchart illustrating several examples of methods (e.g., Method 800) for sending commands to different endpoints of an application. Some operations in Method 800 are combined at will, the order of some operations can be changed at will, and some operations can be omitted at will.

[0130] In some examples, method 800 is performed in a system process of a computer system (e.g., computing system 100, device 200, and / or device 300) (e.g., an operating system process such as a process that maintains a list of user interface elements (e.g., corresponding to different applications) and their corresponding actions resulting from the inputs corresponding to each user interface element) (e.g., UI view manager 340 and / or intent manager 350). In some examples, the computer system is a phone, tablet, laptop, desktop computer, or wearable device.

[0131] In 802, the system process receives an indication of a user interface element corresponding to an application (e.g., as described above with respect to 600) (e.g., in 408, 414, 510, or 512) (e.g., data indicating and / or directed inputs). In some examples, the indication is received by the system process while a user interface view corresponding to the computer system is displayed (e.g., as described above with respect to 600). In some examples, the indication is received while the application is not running. In some examples, the indication is received while the application is running in the background (and, in some examples, not displaying the application's user interface). In some examples, the indication corresponds to received input (e.g., tap input, or non-tap input such as swipe input, gaze input, mouse click, air gesture, and / or hold-and-press input) corresponding to a user interface element. In some examples, the indication of a user interface element includes application indications. In some examples, the indication of a user interface element includes indications of the type of input received with respect to the user interface element. In some examples, the indication is received by a second system process distinct from the system process (e.g., UI view manager 340 and / or intent manager 350), such as a daemon that manages the interaction between the computer system's renderer and one or more applications. In some examples, the indication is not received from the computer system's renderer (e.g., UI renderer 330) and / or host (e.g., configured to control what is displayed in the user interface managed by the computer system's operating system) (e.g., UI host 320).

[0132] Upon receiving an indication of a user interface element corresponding to the application in 804, the system process identifies the action corresponding to the user interface element (e.g., an action, method, function, and / or a set of one or more instructions) (e.g., in 416 or 514). In some examples, the mapping of user interface elements and actions is received from the application before receiving an indication of the user interface element corresponding to the application.

[0133] In 806, upon identifying an action corresponding to a user interface element and determining that the application is in a first state (e.g., the application is currently running in the background and / or the application is currently performing and / or not performing a particular operation), the system process sends an instruction (e.g., a request) (e.g., in 516) to the first endpoint of the application (e.g., directly and / or indirectly, via an intermediate computer system, device, and / or system such as a server and / or a set of one or more computer systems) to perform an action.

[0134] In 808, in response to identifying an action corresponding to a user interface element, and in accordance with the determination that the application is in a second state different from a first state (e.g., the application is not currently running in the background and / or the application is currently performing a particular action and / or not performing it), the system process sends a command to perform the action (e.g., in 522) to a second endpoint of the application (e.g., an extension) different from the first endpoint of the application. In some examples, the determination that the application is in a first or second state is made without communication with the application. In some examples, in accordance with the determination that the application is in a second state, the computer system refrains from sending a command to perform the action to the second endpoint (e.g., directly and / or indirectly). In some examples, in accordance with the determination that the application is in a first state, the computer system refrains from sending a command to perform the action to the first endpoint (e.g., directly and / or indirectly). In some examples, the first endpoint is an extension of the application and the second endpoint is not an extension of the application, or vice versa. In some examples, the first endpoint is the full version of the application, while the second endpoint is not, or vice versa. In some examples, the first endpoint is the first version of the application, while the second endpoint is a second version of the application that differs from the first version. In some examples, the first endpoint is a reduced version of the application compared to the version corresponding to the second endpoint.

[0135] In some examples, user interface elements are contained within the application's user interface view (e.g., a widget). In some examples, a user interface view contains multiple user interface elements. In some examples, one or more of these user interface elements perform different actions using the same or different input types as the user interface elements.

[0136] In some examples, the first endpoint corresponds to an application (e.g., the first endpoint corresponds to a first process of the application that starts executing the application in response to receiving communication via the first endpoint (e.g., the application starts running in the background of the computer system)) (e.g., first application 360, second application 380, or application 502). In some examples, the second endpoint corresponds to an extension of the application (e.g., first extension 370, second extension 390, extension 402, or extension 504) (e.g., the second endpoint corresponds to a first process of the extension that starts executing the extension in response to receiving communication via the second endpoint (e.g., the extension starts running in the background of the computer system)). In some examples, the extension runs separately from the application (e.g., is configured to run separately) (e.g., the extension runs in the foreground and / or background of the computer system without the application running in the foreground and / or background of the computer system).

[0137] In some examples, the first state corresponds to the active state (for example, when an application is currently running in the background and / or foreground of a computer system, e.g., when it sends an action to perform an action).

[0138] In some examples, the second state corresponds to an inactive state (for example, when an application is not currently running in the background and / or foreground of the computer system, e.g., when sending an action to perform an action).

[0139] In some examples, the system process receives a second indication for a second user interface element corresponding to the application. In some examples, the second indication is different from the first indication. In some examples, the second user interface element is different from the first user interface element. In some examples, upon receiving the second indication for a second user interface element corresponding to the application, the system process identifies a second action corresponding to the second user interface element. In some examples, the second action is different from the first action. In some examples, upon identifying the second action corresponding to the second user interface element, the system process sends a command to the first endpoint of the application to perform the second action. In some examples, the command to perform the second action is sent to the first endpoint according to the determination that the second action and / or the second user interface element is of a first type. In some examples, the command to perform the action is selectively sent to either the first or second endpoint according to the determination that the action and / or the user interface element is of a second type different from the first type. In some examples, the second action is specific to the first endpoint and not specific to the second endpoint. In some examples, the second action is application-specific, not extension-specific. In some examples, the second type is always sent to the first endpoint and never to the second endpoint.

[0140] In some examples, the system process receives a third indication for a third user interface element corresponding to the application. In some examples, the third indication is different from the indication. In some examples, the third user interface element is different from the user interface element. In some examples, upon receiving the third indication for the third user interface element corresponding to the application, the system process identifies a third action corresponding to the third user interface element. In some examples, the third action is different from the action. In some examples, upon identifying the third action corresponding to the third user interface element, the system process sends a command to the application's second endpoint to perform the third action. In some examples, the command to perform the third action is sent to the second endpoint according to the determination that the third action and / or the third user interface element is of a third type different from the first and second types. In some examples, the third action is specific to the second endpoint and not to the first endpoint. In some examples, the third action is specific to the extension, not the application. In some examples, the third type is always sent to the second endpoint and never to the first endpoint.

[0141] In some examples, a system process receives a fourth indication for a fourth user interface element corresponding to a second application different from the first application. In some examples, the fourth indication is different from the first indication. In some examples, the fourth user interface element is different from the first user interface element. In some examples, upon receiving the fourth indication for the fourth user interface element corresponding to the second application, the system process identifies a fourth action corresponding to the fourth user interface element. In some examples, the fourth action is different from the first action. In some examples, upon identifying the fourth action corresponding to the fourth user interface element, the system process sends an instruction to a second computer system different from the first computer system to perform the fourth action. In some examples, the instruction to perform the fourth action is sent to the second computer system according to the determination that the fourth action and / or the fourth user interface element is a fourth type different from the first type. In some examples, the fourth action is specific to the second computer system. In some examples, the fourth type is always sent to the second computer system and not to individual endpoints of the computer system. In some examples, the system process receives a fifth indication of a fifth user interface element corresponding to a third application distinct from the application. In some examples, the fifth indication is different from the indication and the fourth indication. In some examples, the fifth user interface element is different from the user interface element and the fourth user interface element. In some examples, upon receiving the fifth indication of the fifth user interface element corresponding to the third application, the system process identifies a fifth action corresponding to the fifth user interface element. In some examples, the fifth action is different from the action and the fourth action.In some examples, upon identifying a fifth action corresponding to a fifth user interface element, the system process sends an instruction to perform the fifth action to a specific endpoint of the computer system. In some examples, the instruction to perform the fifth action is sent to the specific endpoint according to the determination that the fifth action and / or the fifth user interface element is of a fifth type different from the first and fourth types. In some examples, the fifth action is specific to the specific endpoint and not specific to the second computer system. In some examples, the fourth type is always sent to the specific endpoint and never to the second computer system.

[0142] It should be noted that the process details described above in relation to Method 800 (e.g., Figure 8) are also applicable to similar methods to those described herein. For example, Method 900 may optionally include one or more characteristics of the various methods described above with reference to Method 800. For example, the system process of Method 800 may be the system process of Method 900. For brevity, these details will not be repeated below.

[0143] Figure 9 is a flowchart illustrating a method (e.g., method 900) for processing instructions to perform an action, using several examples. In method 900, some actions are combined at will, the order of some actions is changed at will, and some actions are omitted at will.

[0144] In some examples, method 900 is performed on a computer system (e.g., computing system 100, device 200, and / or device 300). In some examples, the computer system is a telephone, tablet, laptop, desktop computer, or wearable device.

[0145] In 902, if an application of a computer system (e.g., as described above in Method 600) (e.g., a first application 360, a first extension 370, a second application 380, or a second extension 390, extension 402, application 502, or extension 504) is in a first state (e.g., as described above in Method 600), the computer system receives an instruction to perform an action (e.g., in 418 or 516) from a first system process of the computer system (e.g., as described above in Method 600) by the first endpoint of the application (e.g., as described above in Method 800), and the action corresponds to a user interface element of a user interface view (e.g., as described above in Method 600) managed by a second system process of the computer system (e.g., a system process different from the first system process). In some examples, the user interface element corresponds to an application.

[0146] In 904, upon receiving an instruction for the first endpoint to perform an action, the computer system has the first endpoint of the application perform the action (for example, as described above with respect to method 800) (for example, in 518).

[0147] In 906, if the application is in a second state different from the first state (for example, as described above with respect to method 800), the computer system receives an instruction from the first system process to perform an action by the second endpoint of the application (for example, in 522), the second endpoint being different from the first endpoint.

[0148] In 908, upon receiving a command for the second endpoint to perform an action, the computer system has the second endpoint of the application perform the action (for example, in 524). In some examples, when the second endpoint performs the action, the first endpoint does not (for example, the second endpoint performs the action without the first endpoint performing the action). In some examples, when the first endpoint performs the action, the second endpoint does not perform the action (for example, the first endpoint performs the action without the second endpoint performing the action).

[0149] In some examples, user interface elements are contained within the application's user interface view (e.g., a widget). In some examples, a user interface view contains multiple user interface elements. In some examples, one or more of these user interface elements perform different actions using the same or different input types as the user interface elements.

[0150] In some examples, the first endpoint corresponds to an application (e.g., the first endpoint corresponds to a first process of the application that starts execution of the application in response to receiving communication via the first endpoint (e.g., the application starts running in the background of the computer system)) (e.g., first application 360, or second application 380). In some examples, the second endpoint corresponds to an extension of the application (e.g., first extension 370, second extension 390, extension 402, or extension 504) (e.g., the second endpoint corresponds to a first process of the extension that starts execution of the extension in response to receiving communication via the second endpoint (e.g., the extension starts running in the background of the computer system)). In some examples, the extension runs separately from the application (e.g., is configured to run separately) (e.g., the extension runs in the foreground and / or background of the computer system without the application running in the foreground and / or background of the computer system).

[0151] In some examples, the first state corresponds to the active state (for example, when an application is currently running in the background and / or foreground of a computer system, e.g., when it sends an action to perform an action).

[0152] In some examples, the second state corresponds to an inactive state (for example, when an application is not currently running in the background and / or foreground of the computer system, e.g., when sending an action to perform an action).

[0153] In some examples, if the second action is a second type of action different from the individual type of action, the computer system receives an instruction from the first system process of the computer system, via the first endpoint of the application, to perform the second action, which corresponds to a second user interface element of a second user interface view managed by the second system process of the computer system. In some examples, the computer system performs the second action via the first endpoint of the application, in response to the first endpoint receiving the instruction to perform the second action. In some examples, the instruction to perform the action is selectively sent to the first or second endpoint, depending on the determination that the action and / or user interface element are of a distinct type. In some examples, the second action is specific to the first endpoint but not to the second endpoint. In some examples, the second action is specific to the application but not to the extension. In some examples, the second type is always sent to the first endpoint and never to the second endpoint.

[0154] In some examples, if the third action is a third type of action different from the action of the first type, the computer system receives an instruction from the first system process of the computer system, via the second endpoint of the application, to perform the third action, which corresponds to a third user interface element of a third user interface view managed by the second system process of the computer system. In some examples, the computer system performs the third action via the second endpoint of the application, in response to the second endpoint receiving the instruction to perform the third action. In some examples, the third action is specific to the second endpoint and not specific to the first endpoint. In some examples, the third action is specific to the extension, not the application. In some examples, the third type is always sent to the second endpoint and never to the first endpoint.

[0155] It should be noted that the process details described above in relation to Method 900 (for example, Figure 9) are also applicable to methods similar to those described herein. For example, Method 800 optionally includes one or more characteristics of the various methods described above with reference to Method 900. For example, the second system process of Method 900 may be the system process of Method 800. For brevity, these details will not be repeated below.

[0156] Figure 10 is a flowchart illustrating a method (e.g., Method 1000) for processing user interaction data from the perspective of multiple programs, using several examples. In Method 1000, some operations are combined at will, the order of some operations is changed at will, and some operations are omitted at will.

[0157] In some examples, method 1000 is executed on a computer system (e.g., computing system 100, device 200, and / or device 300) that stores and / or executes multiple programs (e.g., a first application 360, a first extension 370, a second application 380, a second extension 390, or an extension 504). In some examples, the computer system is a telephone, tablet, laptop, desktop computer, or wearable device.

[0158] In 1002, a first program (e.g., first application 360 or second application 380) corresponding to an application of a computer system (e.g., as described above with respect to method 900) receives user interaction data (e.g., in 516) (e.g., as described above with respect to instructions to perform actions of method 900) through a system process (e.g., UI view manager 340 and / or intent manager 350) that is different from the first program (e.g., and the second program described below). In some examples, multiple programs include the first program but do not include a system process. In some examples, the user interaction data corresponds to interactions with user interface elements of a user interface view managed by a system process (e.g., as described above with respect to method 900).

[0159] In 1004, upon receiving user interaction data, the first program provides the data to the system process (for example, to the system process) (for example, as described above with respect to a request to modify the user interface of method 700) for display on the user interface (for example, as described above with respect to method 900) which is managed by the system process (for example, in 520).

[0160] In 1006, a second program corresponding to an extension of the application (e.g., a first extension 370, a second extension 390, an extension 402, or an extension 504) (e.g., a first extension 370, a second extension 390, an extension 402, or an extension 504) receives user interaction data via a system process. In some examples, the second program is (1) generated by the first program (e.g., part of the first program, downloaded with the first program, included in the first program, and / or executed by the first program), and (2) configured to run separately from the first program. In some examples, either the first program or the second program (e.g., not both) receives the user interaction data. In such examples, the determination of which program receives the user interaction data is as described above.

[0161] In 1008, upon receiving user interaction data, the second program provides, via the system process, data for display within a user interface managed by the system process (e.g., the same or different data as provided by the first program upon receiving user interaction data) (e.g., in 526).

[0162] It should be noted that the process details described above in relation to Method 1000 (e.g., Figure 10) are also applicable to methods similar to those described herein. For example, Method 1000 optionally includes one or more characteristics of the various methods described above with reference to Method 600. For example, the system process of Method 900 may be the system process of Method 600. For brevity, these details will not be repeated below.

[0163] Figure 11 is a flowchart illustrating a method (e.g., method 1100) for processing instructions to perform an action, using several examples. In method 1100, some actions are combined at will, the order of some actions is changed at will, and some actions are omitted at will.

[0164] In some examples, method 1100 is performed on a computer system (e.g., computing system 100, device 200, and / or device 300). In some examples, the computer system is a telephone, tablet, laptop, desktop computer, or wearable device.

[0165] In 1102, an application of the computer system (e.g., a first application 360, a first extension 370, a second application 380, a second extension 390, an application 502, an extension 402, or an extension 504) receives user interaction data (e.g., as described above with respect to instructions for performing actions of method 900) (e.g., in 516) via a system process different from the application (e.g., as described above with respect to method 900) (e.g., a UI view manager 340 and / or intent manager 350). In some examples, the user interaction data corresponds to interactions with user interface elements of a user interface view managed by a system process (e.g., as described above with respect to method 900).

[0166] In 1104, in response to the first program of the application (e.g., application 502) receiving user interaction data, the application provides data to be displayed on a user interface managed by the system process (e.g., as described above with respect to method 700) (e.g., as described above with respect to a request to change the user interface of method 700) (e.g., in 520).

[0167] In 1106, the application causes a second program (e.g., an extension of the application) (e.g., extension 402 or extension 504) to run (as described above). In some examples, the second program is configured to run separately from the first program. In some examples, the second program is configured to receive user interaction data via a system process (e.g., in 522). In some examples, the second program is configured to provide data for display on a user interface managed by the system process (e.g., the same or different data as provided by the first program in response to the receipt of user interaction data) via a system process, in response to the second program receiving user interaction data (e.g., 526).

[0168] It should be noted that the process details described above in relation to Method 1100 (e.g., Figure 11) are also applicable to methods similar to those described herein. For example, Method 700 optionally includes one or more characteristics of the various methods described above with reference to Method 1100. For example, the system process of Method 1100 may be the system process of Method 700. For brevity, these details will not be repeated below.

[0169] In some examples of the methods described above, the following examples apply: In some examples, the user interface view corresponds to a widget (e.g., the user interface view is of a widget). In some examples, the widget and / or user interface view are not live (e.g., the user interface view is not continuously updated, but rather updated in response to the occurrence of events and / or the passage of a certain amount of time, which exceeds the amount of time that the user interprets as continuous). In some examples, the user interface view is updated according to a timeline that is updated through the operation between (1) a system process and (2) an application and / or the application's program / process. In some examples, different user interface views are (1) rendered and / or generated before the user interface view is displayed, and (2) provided to a system process that manages the different user interface views for display according to the timeline. In some examples, the system process is as described above with respect to Figures 3-5, It manages updates to the face view and / or corresponding data. In some examples, an extension of the application (e.g., a second endpoint of method 800 or 900 or a second program in Figure 10 or Figure 11) is defined in the same programming language as the application and / or shares at least some of the same resources (e.g., memory and / or data in the processor) as the application (e.g., a first endpoint of method 800 or 900 or a first program in Figure 10 or Figure 11). In some examples, the application performs the same action (e.g., the action is encapsulated) when it receives a specific instruction from a system process, regardless of the type of device and / or channel used to communicate with the application. In some examples, the action performed by the system process is any amount of code provided by the application.In some examples, actions performed by a system process are semantically defined such that the system process interprets the actions differently on different types of computer systems (e.g., the same in some examples). In some examples, actions are identified and / or determined by a third system process (e.g., the second system process described above or a different system process). In such examples, the third system process can analyze the code of the application and / or actions provided to the third system process in order to identify actions for encapsulation by the third system process for the system processes (e.g., the first system process and / or the second system process). In some examples, actions to update a user interface view require multiple communications between the system process and the application (e.g., communications including a request to the system process to change the user interface view, a request for missing content to the application, and a response to the system process with the missing content). In some examples, the application is downloaded from an application store (e.g., an online store containing multiple applications from different developers and / or entities). In such examples, the application may include extensions (e.g., a second endpoint of method 800 or 900, or a second program in Figure 10 or Figure 11). In some cases, the user interface view is displayed on the computer system's lock screen and / or home screen.

[0170] The above description is based on the purpose of explanation and is written with reference to specific examples. However, the above illustrative description is not intended to be exhaustive or to limit this disclosure to the exact form disclosed. Many modifications and variations are possible in light of the above teachings. These examples have been selected and written to best illustrate the principles of the art and their practical applications. Thereafter, those skilled in the art can best utilize the art and various examples with various modifications suitable for their specific intended use.

[0171] While the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art. Such changes and modifications should be understood to fall within the scope of the present disclosure and examples as defined by the claims.

[0172] As described above, one aspect of the technology involves collecting and using data available from various sources to improve how a device interacts with a user. This disclosure intends that in some cases, this collected data may include personal information data that can be used to uniquely identify or contact or locate a particular individual. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, home addresses, or any other identifying information.

[0173] This disclosure acknowledges that such use of personal data in this technology may be beneficial to the user. For example, personal data can be used to change how a device interacts with the user. Thus, such use of personal data enables better user interaction. Furthermore, other uses of personal data that may benefit the user are also conceivable in this disclosure.

[0174] This disclosure further assumes that any entity responsible for collecting, analyzing, disclosing, transferring, storing, or otherwise using such personal data will comply with appropriately defined privacy policies and / or privacy practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for keeping personal data confidential and secure. For example, personal information from users should be collected for the entity's lawful and legitimate use and should not be shared or sold for any other purpose. Furthermore, such collection should only be carried out after informing and obtaining the user's consent. In addition, such entities will take all necessary steps to protect and secure access to such personal data and to ensure that others who have access to that personal data comply with those privacy policies and procedures. Furthermore, such entities may undergo third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices.

[0175] Notwithstanding the foregoing, the Disclosure also envisions embodiments that allow a user to selectively prevent the use of or access to personal data. That is, the Disclosure envisions that hardware and / or software elements may be provided to prevent or prevent access to such personal data. For example, in the case of imaging, the technology may be configured to allow a user to choose to “opt in” or “opt out” of participating in the collection of personal data while registering for the service.

[0176] Therefore, while this disclosure broadly covers the use of personal data to implement one or more different disclosed embodiments, it is also conceivable that these different embodiments could be implemented without requiring access to such personal data. In other words, the different embodiments of the technology would not be rendered inoperable by the absence of all or part of such personal data. For example, content could be displayed to a user by inferring the location based on minimal personal information, such as non-personal data or content requested by a device associated with the user or other non-personal information.

Claims

1. It is a method, In the system processes of a computer system, While the computer system is displaying a user interface view managed by the system process, Receiving a first input corresponding to a user interface element of the user interface view, In response to receiving the first input, the execution of an action corresponding to the user interface element, the execution of the action being performed by the application corresponding to the user interface element, After triggering the execution of the aforementioned action, the application receives communication corresponding to the change to the user interface view, Upon receiving the aforementioned communication, the display of the user interface view is updated based on the aforementioned communication. Methods that include...

2. The method according to claim 1, wherein the operation includes modifying the state of a second computer system different from the computer system, or modifying local variables of the application.

3. The method according to claim 1 or 2, wherein the user interface element includes a toggle.

4. While the computer system is displaying the user interface view managed by the system process, Receiving a second input corresponding to a second user interface element of the user interface view, Upon receiving the second input, The execution of a second action corresponding to the second user interface element, wherein the execution of the second action is performed by the application, and the execution of the second action is performed by the application, Updating the display of the aforementioned user interface view, The method according to any one of claims 1 to 3, further comprising:

5. The method according to any one of claims 1 to 4, wherein the application is in an inactive state before triggering the execution of the operation.

6. The method according to any one of claims 1 to 5, wherein causing the execution of the aforementioned operation includes causing the application to start in the background of the computer system.

7. The method according to any one of claims 1 to 6, wherein the changes to the user interface view include changes to a timeline of how the display of the user interface view is updated in response to the occurrence of one or more events.

8. The method according to any one of claims 1 to 7, wherein the modification to the user interface view includes a rendered portion of the user interface view.

9. While the computer system displays the first user interface view and the second user interface view simultaneously, the second user interface view is managed by the system process, and the second user interface view differs from the first user interface view. Receiving a third input corresponding to the second user interface view, In response to receiving the third input, the execution of a third action corresponding to the second user interface view, wherein the execution of the third action is performed by a second application different from the application, and the second application triggers the execution of the third action, which is performed by the second application and corresponds to the second user interface view. After triggering the execution of the third action described above, the system receives a second communication from the second application corresponding to a change in the second user interface view, Upon receiving the second communication, the display of the second user interface view is updated based on the second communication. The method according to any one of claims 1 to 8, further comprising:

10. While the computer system is displaying the user interface view managed by the system process, A third user interface element of the user interface view, wherein the third user interface element receives a fourth input that is different from the user interface element and corresponds to the third user interface element, In response to receiving the fourth input, the execution of a fourth action corresponding to the third user interface element, wherein the execution of the fourth action triggers the execution of a fourth action performed by the application, After triggering the execution of the fourth action described above, the application receives a third communication corresponding to the second change to the user interface view, Upon receiving the third communication, the display of the user interface view is updated based on the third communication. The method according to any one of claims 1 to 9, further comprising:

11. A non-temporary computer-readable medium for storing one or more programs configured to be executed by one or more processors of a computer system that executes a system process, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 10.

12. A computer system that executes system processes, One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, A computer system comprising, wherein one or more programs include instructions for performing the method according to any one of claims 1 to 10.

13. A computer system that executes system processes, A computer system comprising means for performing the method described in any one of claims 1 to 10.

14. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that executes a system process, wherein the one or more programs include instructions that perform the method according to any one of claims 1 to 10.

15. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system that executes system processes, wherein the one or more programs are While the computer system is displaying a user interface view managed by the system process, The system receives a first input corresponding to a user interface element of the user interface view, In response to receiving the first input, the execution of an action corresponding to the user interface element, the execution of the action being performed by the application corresponding to the user interface element, triggers the execution of the action. After triggering the execution of the aforementioned action, the application receives a communication corresponding to the change in the user interface view. A non-temporary computer-readable storage medium containing instructions for updating the display of the user interface view based on the communication received.

16. A computer system that executes system processes, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are While the computer system is displaying a user interface view managed by the system process, The system receives a first input corresponding to a user interface element of the user interface view, In response to receiving the first input, the execution of an action corresponding to the user interface element, the execution of the action being performed by the application corresponding to the user interface element, triggers the execution of the action. After triggering the execution of the aforementioned action, the application receives a communication corresponding to the change in the user interface view. A computer system including an instruction to update the display of the user interface view based on the communication received.

17. A computer system that executes system processes, While the computer system is displaying a user interface view managed by the system process, means for receiving a first input corresponding to a user interface element of the user interface view, In response to receiving the first input, means for triggering the execution of an action corresponding to the user interface element, wherein the execution of the action is performed by the application corresponding to the user interface element, After triggering the execution of the aforementioned action, means for receiving communication from the application corresponding to changes in the user interface view, In response to receiving the aforementioned communication, means for updating the display of the user interface view based on the aforementioned communication, A computer system equipped with the following features.

18. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that executes system processes, wherein the one or more programs are While the computer system is displaying a user interface view managed by the system process, The system receives a first input corresponding to a user interface element of the user interface view, In response to receiving the first input, the execution of an action corresponding to the user interface element, the execution of the action being performed by the application corresponding to the user interface element, triggers the execution of the action. After triggering the execution of the aforementioned action, the application receives a communication corresponding to the change in the user interface view. A computer program product that includes instructions for updating the display of the user interface view based on the communication received.

19. It is a method, In computer system applications, While the computer system is displaying a user interface view managed by the system processes of the computer system, The system process receives a request for a user interface element of the user interface view, the user interface element of which corresponds to the application, to perform an action corresponding to interaction with the user interface element, In response to receiving the request to perform the aforementioned operation, the operation is performed, Based on the request to perform the aforementioned operation, a request to modify the user interface view is sent to the system process. Methods that include...

20. The method according to claim 19, wherein the operation includes modifying the state of a second computer system different from the computer system, or modifying local variables of the application.

21. Upon receiving the request to perform the aforementioned operation, the system transitions from an inactive state to an active state before performing the aforementioned operation. The method according to claim 19 or 20, further comprising:

22. The method according to claim 22, wherein the transition from the inactive state to the active state includes starting the application in the background of the computer system.

23. The method according to any one of claims 19 to 22, wherein the request to change the user interface view includes a request for a change to a timeline of how to update the display of the user interface view in response to the occurrence of one or more events.

24. The method according to any one of claims 19 to 23, wherein the request to change the user interface view includes a rendered portion of the user interface view.

25. The method according to any one of claims 19 to 24, wherein the display of the user interface element changes during and independently of the execution of the operation based on the interaction with the user interface element.

26. A non-temporary computer-readable medium for storing one or more programs configured to be executed by one or more processors of a computer system that runs an application, wherein the one or more programs include instructions for performing the method according to any one of claims 19 to 25.

27. A computer system that runs an application, One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 19 to 25.

28. A computer system that runs an application, A computer system comprising means for performing the method described in any one of claims 19 to 25.

29. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that runs an application, wherein the one or more programs comprises instructions that perform the method according to any one of claims 19 to 25.

30. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system that runs an application, wherein the one or more programs are While the computer system is displaying a user interface view managed by the system processes of the computer system, The system process receives a request to perform an action corresponding to the interaction with the user interface element of the user interface view, wherein the user interface element corresponds to the application. In response to receiving the request to perform the aforementioned operation, the operation is performed, A non-temporary computer-readable storage medium containing instructions for sending a request to the system process to modify the user interface view based on the request to perform the aforementioned operation.

31. A computer system that runs an application, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are While the computer system is displaying a user interface view managed by the system processes of the computer system, The system process receives a request to perform an action corresponding to the interaction with the user interface element of the user interface view, wherein the user interface element corresponds to the application. In response to receiving the request to perform the aforementioned operation, the operation is performed, A computer system including instructions for sending a request to the system process to change the user interface view based on the request to perform the aforementioned operation.

32. A computer system that runs an application, While the computer system is displaying a user interface view managed by the system processes of the computer system, The system process includes means for receiving a request to perform an action corresponding to an interaction with the user interface element of the user interface view, wherein the user interface element corresponds to the application, and the user interface element is a user interface element of the user interface view, and the user interface element is a user interface element of the application, In response to receiving the request to perform the aforementioned operation, means for performing the aforementioned operation, Means for sending a request to the system process to change the user interface view based on the request to perform the aforementioned operation, A computer system equipped with the following features.

33. A computer program product comprising one or more programs configured to run on one or more processors of a computer system that runs an application, wherein the one or more programs are While the computer system is displaying a user interface view managed by the system processes of the computer system, The system process receives a request to perform an action corresponding to the interaction with the user interface element of the user interface view, wherein the user interface element corresponds to the application. In response to receiving the request to perform the aforementioned operation, the operation is performed, A computer program product including instructions for sending a request to the system process to change the user interface view based on the request to perform the aforementioned operation.

34. It is a method, In the system processes of a computer system, Receiving indications of user interface elements corresponding to the application, Upon receiving the indication of the user interface element corresponding to the application, the system identifies the action corresponding to the user interface element, In accordance with the identification of the action corresponding to the user interface element, In accordance with the determination that the application is in a first state, a command to perform the action is sent to the first endpoint of the application. In accordance with the determination that the application is in a second state different from the first state, the command to perform the action is sent to a second endpoint of the application, which is different from the first endpoint of the application. Methods that include...

35. The method according to claim 34, wherein the user interface element is included in the user interface view of the application.

36. The method according to claim 34 or 35, wherein the first endpoint corresponds to the application, the second endpoint corresponds to an extension of the application, and the extension is executed separately from the application.

37. The method according to any one of claims 34 to 36, wherein the first state corresponds to an active state.

38. The method according to any one of claims 34 to 37, wherein the second state corresponds to an inactive state.

39. Receiving a second indication of a second user interface element corresponding to the aforementioned application, In response to receiving the second indication of the second user interface element corresponding to the application, the second action corresponding to the second user interface element is identified. In response to identifying the second action corresponding to the second user interface element, the application sends a command to the first endpoint to perform the second action. The method according to any one of claims 34 to 38, further comprising:

40. Receiving a third indication of a third user interface element corresponding to the aforementioned application, In response to receiving the third indication of the third user interface element corresponding to the application, the third action corresponding to the third user interface element is identified. In response to identifying the third action corresponding to the third user interface element, the application sends a command to the second endpoint to execute the third action. The method according to any one of claims 34 to 39, further comprising:

41. Receiving a fourth indication of a fourth user interface element corresponding to a second application different from the aforementioned application, In response to receiving the fourth indication of the fourth user interface element corresponding to the second application, the system identifies the fourth action corresponding to the fourth user interface element, In response to identifying the fourth action corresponding to the fourth user interface element, a command to perform the fourth action is transmitted to a second computer system different from the computer system. Receiving a fifth indication of a fifth user interface element corresponding to a third application different from the aforementioned application, In response to receiving the fifth indication of the fifth user interface element corresponding to the third application, the system identifies a fifth action corresponding to the fifth user interface element, In response to identifying the fifth action corresponding to the fifth user interface element, the system transmits an instruction to perform the fifth action to an individual endpoint of the computer system. The method according to any one of claims 34 to 40, further comprising:

42. A non-temporary computer-readable medium for storing one or more programs configured to be executed by one or more processors of a computer system that executes a system process, wherein the one or more programs include instructions for performing the method according to any one of claims 34 to 41.

43. A computer system that executes system processes, One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 34 to 41.

44. A computer system that executes system processes, A computer system comprising means for performing the method described in any one of claims 34 to 41.

45. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that executes a system process, wherein the one or more programs include instructions to perform the method according to any one of claims 34 to 41.

46. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system that executes system processes, wherein the one or more programs are Receive indications for user interface elements corresponding to the application, Upon receiving the indication of the user interface element corresponding to the application, the system identifies the action corresponding to the user interface element. In accordance with the identification of the action corresponding to the user interface element, In accordance with the determination that the application is in a first state, a command to perform the action is sent to the first endpoint of the application. A non-temporary computer-readable storage medium including instructions for transmitting the instructions to perform the action to a second endpoint of the application, which is different from the first endpoint of the application, in accordance with the determination that the application is in a second state different from the first state.

47. A computer system that executes system processes, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are Receive indications for user interface elements corresponding to the application, Upon receiving the indication of the user interface element corresponding to the application, the system identifies the action corresponding to the user interface element. In accordance with the identification of the action corresponding to the user interface element, In accordance with the determination that the application is in a first state, a command to perform the action is sent to the first endpoint of the application. A computer system including instructions for sending the instructions to perform the action to a second endpoint of the application, which is different from the first endpoint of the application, in accordance with the determination that the application is in a second state different from the first state.

48. A computer system that executes system processes, A means for receiving indications of user interface elements corresponding to an application, A means for identifying an action corresponding to a user interface element in response to receiving the indication of the user interface element corresponding to the application, In accordance with the identification of the action corresponding to the user interface element, A means for sending a command to perform the action to the first endpoint of the application in accordance with the determination that the application is in a first state, A means for transmitting the command to perform the action to a second endpoint of the application, which is different from the first endpoint of the application, in accordance with the determination that the application is in a second state different from the first state, A computer system equipped with the following features.

49. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system that executes system processes, wherein the one or more programs are Receive indications for user interface elements corresponding to the application, Upon receiving the indication of the user interface element corresponding to the application, the system identifies the action corresponding to the user interface element. In accordance with the identification of the action corresponding to the user interface element, In accordance with the determination that the application is in a first state, a command to perform the action is sent to the first endpoint of the application. A computer program product including instructions, which, upon determination that the application is in a second state different from the first state, transmits the instructions to perform the action to a second endpoint of the application different from the first endpoint of the application.

50. It is a method, When an application of a computer system is in a first state, the first endpoint of the application receives an instruction from a first system process of the computer system to perform an action, the action being an instruction that corresponds to a user interface element of a user interface view managed by a second system process of the computer system. In response to the first endpoint receiving the command to perform the action, the first endpoint of the application performs the action, If the application is in a second state different from the first state, the second endpoint of the application, which is different from the first endpoint, receives the instruction to perform the action from the first system process. In response to the second endpoint receiving the command to perform the action, the second endpoint of the application performs the action, Methods that include...

51. The method according to claim 50, wherein the user interface element is included in the user interface view of the application.

52. The method according to any one of claims 50 to 51, wherein the first endpoint corresponds to the application, the second endpoint corresponds to an extension of the application, and the extension is executed separately from the application.

53. The method according to any one of claims 50 to 52, wherein the first state corresponds to an active state.

54. The method according to any one of claims 50 to 53, wherein the second state corresponds to an inactive state.

55. If the second action is a second type of action different from the individual types of the first action, the first endpoint of the application receives an instruction from the first system process of the computer system to perform the second action, wherein the second action corresponds to a second user interface element of a second user interface view managed by the second system process of the computer system. In response to the first endpoint receiving the instruction to perform the second action, the first endpoint of the application performs the second action, The method according to any one of claims 50 to 54, further comprising:

56. If the third action is a third type of action different from the type of action described above, the second endpoint of the application receives an instruction from the first system process of the computer system to perform the third action, wherein the third action corresponds to a third user interface element of a third user interface view managed by the second system process of the computer system. In response to the second endpoint receiving the instruction to perform the third action, the second endpoint of the application performs the third action, The method according to any one of claims 50 to 55, further comprising:

57. A non-temporary computer-readable medium for storing one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs include instructions for performing the method according to any one of claims 50 to 56.

58. A computer system, One or more processors, A computer system comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 50 to 56.

59. A computer system, A computer system comprising means for performing the method described in any one of claims 50 to 56.

60. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs include instructions for performing the method according to any one of claims 50 to 56.

61. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs are When an application of a computer system is in a first state, the first endpoint of the application receives an instruction from a first system process of the computer system to perform an action, the action being an instruction that corresponds to a user interface element of a user interface view managed by a second system process of the computer system. In response to the first endpoint receiving the command to perform the action, the first endpoint of the application performs the action. If the application is in a second state different from the first state, the second endpoint of the application, which is different from the first endpoint, receives the instruction to perform the action from the first system process. A non-temporary computer-readable storage medium containing instructions, which, in response to the second endpoint receiving the instructions for performing the action, is used by the second endpoint of the application to perform the action.

62. A computer system, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are When an application of a computer system is in a first state, the first endpoint of the application receives an instruction from a first system process of the computer system to perform an action, the action being an instruction that corresponds to a user interface element of a user interface view managed by a second system process of the computer system. In response to the first endpoint receiving the command to perform the action, the first endpoint of the application performs the action. If the application is in a second state different from the first state, the second endpoint of the application, which is different from the first endpoint, receives the instruction to perform the action from the first system process. A computer system including instructions, in which the second endpoint of the application performs the action in response to the second endpoint receiving the instruction to perform the action.

63. A computer system, When an application of a computer system is in a first state, the first endpoint of the application has means for receiving an instruction from a first system process of the computer system to perform an action, wherein the action corresponds to a user interface element of a user interface view managed by a second system process of the computer system. In response to the first endpoint receiving the command to perform the action, the first endpoint of the application provides means for performing the action, If the application is in a second state different from the first state, the application has a second endpoint, which is different from the first endpoint, that has means for receiving the command to perform the action from the first system process. In response to the second endpoint receiving the command to perform the action, the second endpoint of the application provides means for performing the action, A computer system equipped with the following features.

64. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs include instructions, When an application of a computer system is in a first state, the first endpoint of the application receives an instruction from a first system process of the computer system to perform an action, the action being an instruction that corresponds to a user interface element of a user interface view managed by a second system process of the computer system. In response to the first endpoint receiving the command to perform the action, the first endpoint of the application performs the action. If the application is in a second state different from the first state, the second endpoint of the application, which is different from the first endpoint, receives the instruction to perform the action from the first system process. A computer program product comprising instructions, which, in response to the second endpoint receiving the instructions for performing the action, is performed by the second endpoint of the application.

65. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs are A first program corresponding to an application, wherein the first program is User interaction data is received via a system process different from the aforementioned multiple programs. A first program is configured to, upon receiving the aforementioned user interaction data, provide data for display on a user interface managed by the system process via the system process, A second program corresponding to the extension of the application, the second program being configured to run separately from the first program, includes, The user interaction data is received via the aforementioned system process. A non-temporary computer-readable storage medium configured to, upon receiving the user interaction data, provide data for display on the user interface managed by the system process via the system process.

66. A computer system, One or more processors, The system comprises a memory that stores one or more programs configured to be executed by one or more processors, and the one or more programs are A first program corresponding to an application, wherein the first program is User interaction data is received via a system process different from the aforementioned multiple programs. A first program is configured to, upon receiving the aforementioned user interaction data, provide data for display on a user interface managed by the system process via the system process, A second program corresponding to the extension of the application, the second program being configured to run separately from the first program, includes, The user interaction data is received via the aforementioned system process. A computer system configured to, upon receiving the aforementioned user interaction data, provide data for display on the user interface managed by the system process via the system process.

67. A non-temporary computer-readable storage medium for storing an application configured to run by one or more processors of a computer system, wherein the application is User interaction data is received via a system process different from the aforementioned multiple programs. Upon receiving the user interaction data, the first program of the application provides data for display on a user interface managed by the system process via the system process. A second program, the second program is configured to run separately from the first program, and includes instructions to be executed by the second program, the second program is The user interaction data is received via the aforementioned system process. A non-temporary computer-readable storage medium configured to, upon receiving the user interaction data of the second program, provide data for display on the user interface managed by the system process via the system process.

68. A computer system, One or more processors, The system includes a memory that stores an application configured to run on one or more processors, and the application is: User interaction data is received via a system process different from the aforementioned application. Upon receiving the user interaction data, the first program of the application provides data for display on a user interface managed by the system process via the system process. A second program, the second program is configured to run separately from the first program, and includes instructions to be executed by the second program, the second program is The user interaction data is received via the aforementioned system process. A computer system configured such that, upon receiving the user interaction data, the second program provides, via the system process, data for display on the user interface managed by the system process.

Images