Barrier-free service management method and device, chip, electronic equipment and medium

Abstract

The application provides an accessibility service management and control method and device, a chip, electronic equipment and a medium. The management and control method is applied to electronic equipment, the electronic equipment is installed with a first application and a second application, the second application supports an accessibility service and has a use permission of the accessibility service; the management and control method comprises the following steps: obtaining an accessibility event for describing an interface state change when the first application exists an interface state change; and if the second application is in a first white list, sending the accessibility event to the second application through the accessibility service. The application can avoid the risk of user information leakage.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to a method, device, chip, electronic equipment and medium for managing accessibility services. Background Technology

[0002] The operating system of an electronic device can provide accessibility services to users, offering accessible user interaction features to users with disabilities or those temporarily unable to fully interact with the device. For example, a text-to-speech application that supports accessibility services can provide users with the ability to read screen content aloud.

[0003] When a user interacts with the user interface displayed on an electronic device, the accessibility service can obtain accessibility events generated by the system. These accessibility events can be used to indicate changes in the state of the user interface. If the user has granted accessibility access to an accessibility application (or an application that supports the accessibility service), the accessibility application can receive accessibility events sent by the accessibility service and respond accordingly to provide accessible user interaction functions.

[0004] Because once a user grants accessibility permissions to an accessibility application, the application can obtain information about changes in the user interface state through accessibility services, there is a risk of user information leakage. For example, even if an accessibility application is actually malicious, it can still obtain user interface state change information indiscriminately with user authorization. Summary of the Invention

[0005] This application provides a method, device, chip, electronic device, and medium for managing accessibility services, which can avoid the risk of user information leakage.

[0006] Operating systems for electronic devices (such as Android) can provide accessibility services to users. These services can run in the background and receive accessibility events from the system. Accessibility events can represent a series of state changes in the user interface, such as changes in focus, input content, or button clicks. Any change in the user interface will generate a corresponding accessibility event, which the system can then notify the accessibility service of. If accessibility service management is not implemented (i.e., accessibility applications are not managed to use the accessibility service), then once a user grants accessibility applications permission to use the accessibility service, the applications can receive accessibility events distributed by the service; in other words, the applications have access to all system accessibility events.

[0007] Since accessible applications can obtain information about changes in the user interface state through accessibility events, there is a risk of user information leakage. Therefore, to address this risk, accessibility applications can implement controls over their use of accessibility services.

[0008] In a first aspect, embodiments of this application provide a method for managing accessibility services, applied to an electronic device. The electronic device has a first application and a second application installed. The second application supports accessibility services and has accessibility service usage permissions. The management method includes: when the interface state of the first application changes, obtaining an accessibility event to describe the interface state change; if the second application is in a first whitelist, sending the accessibility event to the second application through the accessibility service.

[0009] In this embodiment of the application, the second application can be any accessible application, and the user has enabled their access to the accessible service, while the first application can be any ordinary application (i.e., a non-accessible application).

[0010] When an electronic device is running the first application in the foreground, it can display the user interface of that application, allowing the user to operate the device as needed to use the application. For example, the user can use the touchscreen of the electronic device to click on controls on the user interface of the first application, enter user information (such as login account password, payment password, and other sensitive user information), and perform swipe operations on the user interface.

[0011] User actions on the user interface can trigger the operating system to generate accessibility events. Accessibility services can intercept these events and distribute them to whitelisted accessibility applications.

[0012] In one embodiment, the accessibility service can set the state values ​​of user operations (such as clicking, double-clicking, swiping, etc.) through setServiceinfo (service information setting function), thereby enabling the accessibility service to intercept accessibility events generated by the operating system (i.e., obtain event information of accessibility events) when the user operates the user interface. The event information of accessibility events can include event type (EventType) and event object (Source). The event type can represent the type of user operation corresponding to the accessibility event; the event object can represent the object of the user operation corresponding to the accessibility event, which can be a button, pop-up, or floating window on the user interface, etc.

[0013] Besides user actions that can cause changes in the user interface state, other examples include non-user actions that can also lead to changes in the user interface state. For instance, the user interface may display information received by the application in real time, thereby triggering the system to generate corresponding accessibility events. In one embodiment, accessibility events may include application interface content such as control IDs (unique codes) and text content to describe changes in the application's interface state.

[0014] This application embodiment can perform accessibility service control based on a whitelist. The whitelist may record the identifiers of one or more accessibility applications, or it may not record any accessibility application identifiers, so that accessibility applications in the whitelist can obtain accessibility events reflecting changes in the user interface state without sending accessibility events to accessibility applications outside the whitelist. In one embodiment, the whitelist may include accessibility applications commonly used by people with disabilities, so as to support the normal use of these accessibility applications while performing accessibility service control.

[0015] Feasibly, an electronic device can have one or more standard applications and one or more accessibility applications installed. If multiple accessibility applications are installed and all are authorized by the user, after receiving the aforementioned accessibility event, the electronic device can sequentially or simultaneously determine whether each accessibility application is in a first whitelist, and distribute the accessibility event to each accessibility application in the whitelist based on the determination result. Upon receiving an accessibility event, the accessibility application can either remain inactive or respond to the accessibility event to provide accessible user interaction functions.

[0016] In one example, an electronic device has multiple accessible applications installed, such as a text-to-speech app, and the user allows these applications to use accessibility services. When the electronic device uses a news app (using a news app as an example, but other installed apps could also be used), the user interface can display news information and the text-to-speech app's identifier. The text-to-speech app provides the user with a text-to-speech service in its set reading mode. If the user performs an upward swipe gesture on the touchscreen to request an update to the news information, the system can generate a corresponding accessibility event and send it to the accessibility service. Assuming the news app's whitelist only includes the text-to-speech app, the accessibility service will send the accessibility event to the text-to-speech app, not to any other accessible applications.

[0017] In one embodiment, if the second application is not in the first whitelist, the step of sending the accessibility event to the second application via the accessibility service can be skipped. In other embodiments, while not sending the accessibility event to accessibility applications outside the whitelist, a corresponding prompt message can be displayed so that the user is aware of the accessibility service management result. In yet another embodiment, the user can also choose, based on the prompt message, whether to request that the accessibility event be sent to accessibility applications outside the whitelist.

[0018] As can be seen, the embodiments of this application are based on a whitelist, which can control the use of accessibility services by accessibility applications. It selectively allows accessibility applications in the whitelist to use accessibility services, while prohibiting (either completely or within a certain range) accessibility applications outside the whitelist from using accessibility services. This differentiates the sending of accessibility events to trusted accessibility applications, thus preventing malicious application developers from obtaining user interface state change information and thus avoiding the risk of user information leakage.

[0019] Considering that accessibility incidents do not always involve user information pages, and that users install accessibility applications to use them for accessible user interaction functions, in addition to addressing the risk of user information leakage, to support accessibility applications in providing accessible user interaction functions, a general system control can be implemented. Based on a whitelist, accessibility services can be restricted for accessibility applications not on the system whitelist only for user information pages, while for other types of pages, accessibility services for accessibility applications not on the system whitelist can be unrestricted.

[0020] It is feasible to perform general system control based on a system whitelist. Among other feasible implementations, if the first application provides an application whitelist, general system control can also be performed based on the application whitelist provided by the first application.

[0021] In one embodiment of the first aspect described above, the control method further includes: determining whether the user interface of the first application includes user information-related controls when the interface state changes; if the user interface includes user information-related controls (i.e., the user interface is a user information page), then performing the step described above of sending accessibility events to the second application through the accessibility service if the second application is in the first whitelist.

[0022] During general system management, the system whitelist can be used as the whitelist corresponding to the first application. Accessibility events reflecting changes in the user information page status can be sent to accessibility applications on the whitelist, but not to accessibility applications outside the whitelist. In one embodiment, if the user interface includes user information-related controls, the system whitelist is used as the first whitelist, and the steps described above, where the second application is on the first whitelist, are executed to send accessibility events to the second application through the accessibility service.

[0023] In one embodiment of general system management, if the user interface does not include user information-related controls, so that accessibility events do not involve changes in the state of the user information page, then accessibility events can be sent to a second application through the accessibility service, regardless of whether the second application is in the system whitelist.

[0024] For example, user information-related controls can be password controls, account controls, or other controls that involve sensitive user information. If the user interface includes a password control, then the user interface is a password control page, where users typically enter their password information (such as payment password, login password, etc.), causing accessibility events to affect the state changes of the password control page.

[0025] Feasibly, the user information page can be identified by an accessibility service or a functional module developed by the developer. In one embodiment of the first aspect described above, the control method further includes: using an accessibility service or a module in an electronic device for controlling accessibility services, and according to a set window mechanism, performing the step of determining whether the user interface of the first application includes user information-related controls when the interface state changes; the window mechanism is used to describe the switching of the interface window of the first application and the controls within the interface window of the first application.

[0026] Feasibly, the operating system of an electronic device can provide a windowing mechanism, which can be configured to include information such as application window switching, which window appears after switching, and which controls are included in the new window. Thus, based on this windowing mechanism, it is possible to identify whether the currently displayed user interface is a user information page.

[0027] In one embodiment, the system whitelist can be a whitelist provided by the manufacturer of the electronic device, and the manufacturer can update the provided whitelist in real time and distribute it to the electronic device. In another feasible implementation, the updated whitelist can be distributed to the electronic device in real time via a system cloud.

[0028] In one embodiment, during general system control (i.e., accessibility service control based on a system whitelist), the electronic device can output corresponding prompts without sending accessibility events to a second application. If a user requests modification of the system whitelist based on the prompts, the system whitelist stored on the electronic device can be modified according to the user's needs. That is, the modified system whitelist is a combination of the whitelist provided by the device manufacturer and the user's personalized whitelist. Thus, system whitelists can differ across different electronic devices to not only meet general system control requirements but also satisfy the personalized control needs of device users.

[0029] In one embodiment, the first application can be any application already installed on the electronic device; that is, there is no distinction between different applications, and general system control is always performed on all applications. Thus, when the interface state of any application changes, if the currently displayed user interface is a user information page, accessibility service control is performed based on the system whitelist to restrict accessibility services for applications not on the system whitelist. Otherwise, accessibility events can be sent to each accessibility application through the accessibility service, so that accessibility services are not restricted for applications not on the system whitelist.

[0030] In another embodiment, different types of applications can also be distinguished. For example, applications can be divided into third-party open applications and general applications based on whether they can request customized accessibility protection (i.e., request customized control).

[0031] When differentiating between different application types, for third-party open applications, customized accessibility protection requests can be met by implementing customized controls based on an application whitelist to suit the individual needs of the application developers. For general applications, general system controls are implemented based on a system whitelist.

[0032] During the customized control period, the application whitelist is used as the whitelist corresponding to the first application.

[0033] In one embodiment of the first aspect above, the control method further includes: if the first application is not an application that can request accessibility protection (i.e., a general application, rather than a third-party open application), performing the above-mentioned step of determining whether the user interface of the first application includes user information-related controls when the interface state changes.

[0034] By implementing general system controls when general applications run on electronic devices, it is possible to at least ensure that there is no risk of user information leakage when users use general applications, even if the developers of general applications do not have specific control requirements, and to minimize the impact on the accessibility of applications in providing accessible user interaction functions.

[0035] As described above, the general system control can be as follows: If the user interface includes user information-related controls, accessibility service control is performed based on a system whitelist. Accessibility events are sent to accessibility applications on the system whitelist via the accessibility service, but not to accessibility applications outside the system whitelist, thus restricting the use of accessibility services by accessibility applications outside the system whitelist. Conversely, if the user interface does not include user information-related controls, there are no restrictions on accessibility applications outside the system whitelist using accessibility services; accessibility events can be sent to accessibility applications outside the system whitelist via the accessibility service.

[0036] In contrast, customized control can be implemented as follows: if a third-party open application requests accessibility protection, accessibility service control is performed based on the application whitelist. Accessibility events are sent to accessibility applications in the application whitelist through the accessibility service, but not to accessibility applications outside the application whitelist, so as to restrict accessibility applications outside the application whitelist from using accessibility services, regardless of whether the user interface includes user information-related controls at this time.

[0037] In one implementation method that distinguishes different types of applications (referred to as implementation method 1), customized control can be applied only to third-party open applications.

[0038] In Method 1, general system control is executed when running general applications, while customized control is executed based on accessibility protection requests when running third-party open applications.

[0039] In another implementation method that differentiates between different types of applications (referred to as implementation method 2), not only customized control can be applied to third-party open applications, but also general system control can be applied during non-customized control periods. That is, by combining customized control and general system control, barrier-free service control can be achieved when electronic devices run third-party open applications.

[0040] In implementation method 2, general system control is executed when running general applications, while customized control is executed based on accessibility protection requests when running third-party open applications, and general system control is executed during non-customized control periods.

[0041] By implementing general system controls on third-party open applications during the non-customized control period, it is possible to at least ensure that users will not face the risk of information leakage when using third-party open applications during the non-customized control period, even if the developers of the third-party open applications do not have specific control requirements during the non-customized control period. At the same time, it is possible to minimize the impact on the accessibility of applications in providing accessible user interaction functions.

[0042] It is evident that by combining customized control with general system control, in addition to achieving the personalized control objectives required by application developers during the customized control period of third-party open applications to avoid the risk of information leakage (i.e., not limited to user information) when users use third-party open applications during this period, it can also at least ensure that users will not face the risk of user information leakage when using third-party open applications during the non-customized control period, and minimize the impact on the accessibility of applications in providing accessible user interaction functions.

[0043] Feasibly, developers of third-party open applications can choose to request accessibility protection during the entire runtime of the third-party open application, or they can choose to request accessibility protection when the third-party open application displays one or more user interfaces, based on their own needs. Thus, in one embodiment of the first aspect above, the time period during which the first application requests accessibility protection (i.e., the customized control time period) is the time period during which the first application runs, or the time period during which the first application displays the setting user interface.

[0044] In one embodiment of the first aspect above, if the electronic device provides an accessibility protection interface to the first application, then the first application is an application capable of requesting accessibility protection (i.e., the aforementioned third-party open application); if the electronic device does not provide an accessibility protection interface to the first application, then the first application is not an application capable of requesting accessibility protection (i.e., the aforementioned general application); wherein, the accessibility protection interface is used for application calls to request accessibility protection.

[0045] Conveniently, electronic devices can provide accessibility protection interfaces to third-party open applications with customized control requirements, but not to general applications. In one implementation, third-party open applications can start / stop requesting accessibility protection by calling the accessibility protection interface.

[0046] In one embodiment, third-party open applications may integrate accessibility protection interfaces, while general applications may not integrate accessibility protection interfaces. In this case, applications with integrated interfaces are third-party open applications, while applications without integrated interfaces are general applications.

[0047] Corresponding to implementation method 1 or implementation method 2 above, in one embodiment of the first aspect (denoted as embodiment a), the control method further includes: if the first application is an application that can request accessibility protection (i.e., a third-party open application, rather than a general application), determining whether the first application has requested accessibility protection when there is a change in the interface state of the first application (i.e., determining whether the current moment is during the customized control period); if the first application has requested accessibility protection, performing the above-mentioned step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

[0048] When an electronic device runs a third-party open application, the third-party open application can request accessibility protection to enable the electronic device to perform customized control. During the customized control period, for accessibility events generated when the interface state of the third-party open application changes, the accessibility events can be sent to accessibility applications in the application whitelist, instead of sending them to accessibility applications outside the application whitelist, in order to avoid the risk of information leakage when users use third-party open applications during the customized control period.

[0049] Feasibly, developers of third-party open applications can provide custom application whitelists. Different third-party open applications can have different application whitelists to meet the personalized control needs of different application developers. In this case, since the developer specifies not only the timing of the customized control but also the objects to be controlled during the customized control period, the whitelist used during the customized control period can be the application whitelist. Thus, in one implementation of embodiment a above, if the first application provides a whitelist, then the first whitelist is the whitelist provided by the first application.

[0050] Alternatively, developers of third-party open applications may not provide a custom application whitelist. In this case, since the developer specifies the timing of the custom control but not the objects to be controlled during the custom control period, the whitelist used during the custom control period can be the system whitelist, i.e., the system whitelist is used as the application whitelist. Thus, in another implementation of embodiment a above, if the first application does not provide a whitelist, the first whitelist is the system whitelist.

[0051] Corresponding to the above implementation method 1, in one implementation of the above embodiment a, the control method further includes: if the first application does not request accessibility protection, the accessibility event can be sent to the second application through the accessibility service.

[0052] In implementation method 1, only customized control can be implemented for third-party open applications. During the non-customized control period of third-party open applications, accessibility services can be used without restriction for accessibility applications not on the system whitelist, and accessibility events can be sent directly to accessibility applications not on the system whitelist.

[0053] Corresponding to the above implementation method 2, in one implementation of the above embodiment a, the control method further includes: if the first application does not request accessibility protection, determining whether the user interface of the first application includes user information-related controls when the interface state changes; if the user interface includes user information-related controls, then if the system whitelist includes the second application, sending the accessibility event to the second application through the accessibility service.

[0054] In implementation method 2, customized control can be performed during the customized control period of the third-party open application, while general system control can be performed during the non-customized control period of the third-party open application. By combining customized control and general system control, the risk of user information leakage during the non-customized control period can be addressed while meeting the personalized control needs of the application.

[0055] In one embodiment, if the first application does not request accessibility protection and the user interface does not include user information-related controls, so that accessibility events generated during non-customized management do not involve user information pages and there is no risk of user information leakage, then accessibility events can be sent to the second application through the accessibility service.

[0056] As mentioned above, by setting up a whitelist, accessibility service control based on the whitelist can be implemented on demand, allowing accessible applications on the whitelist to use accessibility services while prohibiting accessible applications outside the whitelist from using them. The contents of the whitelist can be set by application developers and device manufacturers as needed, and typically trusted accessible applications can be added to the whitelist, while untrusted accessible applications are not included.

[0057] During customized accessibility management, only accessibility applications on the whitelist can use accessibility services; those not on the whitelist cannot. During system-wide accessibility management, when a device displays a user information page, only whitelisted accessibility applications can use accessibility services; those not on the whitelist cannot. However, when a device displays a non-user information page, no accessibility management is applied, and accessibility applications not on the whitelist can use accessibility services, even if they are known to be malicious.

[0058] To prevent malicious applications from using accessibility services under any circumstances, a blacklist can be set up. The blacklist can contain the identifiers of one or more accessibility applications, or it can contain no accessibility application identifiers at all. This restricts accessibility applications on the blacklist from obtaining accessibility events reflecting changes in the user interface state under any circumstances, thereby preventing malicious applications from obtaining information about interface state changes.

[0059] In one embodiment of the first aspect above, the control method further includes: if the second application is not in the system blacklist, performing the step described above of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

[0060] After generating an accessibility event, it can be first determined whether the second application is on the system blacklist. If so, the accessibility event will not be sent to the second application; otherwise, based on the whitelist, general system controls or customized controls can be implemented. In this way, malicious applications can be completely prohibited from using accessibility services, while untrusted applications can be prohibited from using accessibility services in certain situations as needed.

[0061] The system blacklist can be a blacklist provided by the manufacturers of electronic devices, and these manufacturers can update the blacklist in real time and distribute it to the electronic devices. One feasible implementation is to use a system cloud to distribute the updated blacklist to the electronic devices in real time.

[0062] In one embodiment, when an electronic device performs accessibility service control based on a system blacklist, it can output corresponding prompts without sending accessibility events to a second application. If a user requests modification of the system blacklist based on the prompts, the system blacklist stored on the electronic device can be modified according to the user's needs. That is, the modified system blacklist is a combination of the blacklist provided by the device manufacturer and the user's personalized blacklist. In this way, the system blacklists on different electronic devices can differ, so as to not only comply with the general system control but also meet the personalized control needs of device users.

[0063] Secondly, embodiments of this application provide a management device for accessibility services, applied to an electronic device. The electronic device has a first application and a second application installed. The second application supports accessibility services and has accessibility service usage permissions. The management device includes: an acquisition module, used to acquire accessibility events describing the interface state changes when the interface state of the first application changes; and a processing module, used to send the accessibility events to the second application through the accessibility service if the second application is in a first whitelist.

[0064] Thirdly, embodiments of this application provide a chip, including: a processor, which is configured to execute computer program instructions stored in a memory, wherein when the computer program instructions are executed by the processor, the chip is triggered to execute the method as described in any of the first aspects.

[0065] Fourthly, embodiments of this application provide an electronic device, which includes one or more memories for storing computer program instructions and one or more processors, wherein when the computer program instructions are executed by one or more processors, the electronic device is triggered to perform the method as described in any of the first aspects.

[0066] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method as described in any of the first aspects.

[0067] In a sixth aspect, embodiments of this application provide a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method as described in any of the first aspects.

[0068] The technical effects of the aforementioned aspects can be referenced from each other, and will not be elaborated further here. Attached Figure Description

[0069] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below.

[0070] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0071] Figure 2a and Figure 2b A schematic diagram illustrating two methods for enabling accessibility applications to access accessibility services, as provided in the embodiments of this application.

[0072] Figure 3 A schematic diagram illustrating the use of accessibility services in an accessibility application provided in this application embodiment;

[0073] Figure 4 A schematic diagram of a password control page provided in an embodiment of this application;

[0074] Figure 5 A flowchart illustrating a method for managing accessibility services provided in an embodiment of this application;

[0075] Figure 6 A flowchart illustrating another accessibility service management method provided in this application embodiment;

[0076] Figure 7 This is a software structure block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0077] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0078] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0079] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0080] It should be understood that the term "at least one" as used in this document refers to one or more, and "more than one" refers to two or more. The term "and / or" as used in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0081] It should be understood that although the terms "first," "second," etc., may be used to describe the set thresholds in the embodiments of this application, these set thresholds should not be limited to these terms. These terms are only used to distinguish the set thresholds from each other. For example, without departing from the scope of the embodiments of this application, the first set threshold may also be referred to as the second set threshold, and similarly, the second set threshold may also be referred to as the first set threshold.

[0082] The method provided in any embodiment of this application can be applied to electronic devices such as mobile phones, tablets, personal digital assistants (PDAs), desktops, laptops, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices. This application does not impose any special limitations on the specific form of the aforementioned electronic devices.

[0083] The accessibility service management method provided in any embodiment of this application can be applied to... Figure 1 In the electronic device 100 shown. Figure 1 A schematic diagram of the structure of the electronic device 100 is shown.

[0084] Electronic device 100 may include processor 110, internal memory 121, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, display screen 194, etc. Sensor module 180 may include pressure sensor, touch sensor, etc.

[0085] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0086] In some embodiments, the processor 110 may be a system-on-chip (SoC), which may include a central processing unit (CPU), and may further include other types of processors. In some embodiments, the processor 110 may be a PWM control chip.

[0087] The processor 110 may also include necessary hardware accelerators or logic processing hardware circuitry, such as an ASIC, or one or more integrated circuits for controlling the execution of the program. Furthermore, the processor 110 may have the capability to operate one or more software programs, which may be stored in a storage medium.

[0088] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0089] In some embodiments, the memory of the electronic device 100 may be a read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), or other types of dynamic storage devices capable of storing information and instructions. It may also be an electrically erasable programmable read-only memory (EEPROM), or any computer-readable medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer.

[0090] In some embodiments, the processor 110 and the memory can be combined into a single processing device, or they can be separate components. The processor 110 can be used to execute program code stored in the memory. In specific implementations, the memory can be integrated into the processor 110, or it can be independent of the processor 110.

[0091] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0092] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0093] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0094] In one embodiment, the electronic device 100 can receive a system whitelist / blacklist from the system cloud via a mobile communication module 150 and / or a wireless communication module 160 to obtain the system whitelist / blacklist. If it is the first time obtaining the system whitelist / blacklist, the obtained system whitelist / blacklist can be stored in the internal memory 121. If it is not the first time obtaining the system whitelist / blacklist, the system whitelist / blacklist stored in the internal memory 121 can be updated in real time based on the obtained system whitelist / blacklist. Similarly, the electronic device 100 can also obtain an application whitelist provided by the application developer of the installed application to achieve the first acquisition or real-time update of the application whitelist.

[0095] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0096] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0097] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to speaker 170A, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0098] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0099] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0100] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0101] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0102] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0103] The electronic device 100 can implement audio functions through an audio module 170, a speaker 170A, a headphone jack 170D, and an application processor. Examples include screen reading, music playback, and recording. In one embodiment, the screen reading function can be implemented based on a text-to-speech application that supports accessibility services installed on the electronic device 100.

[0104] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 194. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor. The electronic device 100 may also calculate the touch location based on the detection signal from the pressure sensor. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities may correspond to different operation commands.

[0105] A touch sensor, also known as a "touch device," can be located on the display screen 194. The touch sensor and the display screen 194 together form a touchscreen, also known as a "touchscreen." The touch sensor detects touch operations applied to or near it. The touch sensor can then transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 194.

[0106] In one embodiment, when the electronic device 100 displays the user interface of an application via a touchscreen, the user can use the application's functions by performing touch actions such as clicking and swiping on the touchscreen. In response to the user's actions, the application's interface state may change. The operating system of the electronic device 100 can then generate corresponding accessibility events to describe these changes. The electronic device 100 can distribute these accessibility events to accessible applications on the whitelist, using the accessibility services provided by the operating system, while refraining from distributing them to accessibility applications outside the whitelist, thereby achieving accessibility service management.

[0107] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 192 can be an indicator light, used to indicate charging status, battery level changes, or to indicate messages, missed calls, notifications, etc.

[0108] In one embodiment, by implementing accessibility service control, there may be situations where accessibility events are not distributed to some accessibility applications, that is, these accessibility applications are prohibited from using accessibility services. In this case, corresponding prompts can be output through motor 191, indicator 192, and display screen 194 to prompt the user via vibration, indicator light, screen pop-up, etc.

[0109] In addition, the electronic device runs an operating system on top of the aforementioned components. For example... operating system, operating system, Operating systems, etc. Applications can be installed and run on an operating system.

[0110] Operating systems can provide accessibility services to users. These services can utilize text-to-speech, haptic feedback, gesture navigation, trackballs, and directional navigation to provide accessibility functionality to applications, making them more user-friendly. Accessibility services can run in the background, receiving accessibility events from the system. These events represent changes in the user interface state, such as changes in focus, input content, or button clicks. Accessibility services can also retrieve the active window and its contents.

[0111] When the user interface of an application undergoes a state change, the system can generate an accessibility event describing the interface state change and notify the accessibility service. Accessible applications can obtain accessibility events through the accessibility service and can also provide operation instructions through the accessibility service to provide accessible user interaction functions.

[0112] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0113] As mentioned above, the operating system of an electronic device can provide accessibility services to users. These services can be used to provide accessible user interaction functions to users with disabilities or those who are temporarily unable to fully interact with the device. For example, a text-to-speech application that supports accessibility services can provide users with the ability to read screen content aloud.

[0114] Application developers can implement their applications as accessible applications. After users install an accessible application on their electronic devices, they can grant the application permission to use accessibility services. Once authorized by the user, the accessible application can typically run in the background and provide accessible user interaction functions by using accessibility services.

[0115] In one implementation, the user can enable accessibility applications to access accessibility services in the settings app of the electronic device. In another implementation, the user can enable accessibility applications to access accessibility services in the control center of the electronic device. The following sections will discuss these implementations in conjunction with... Figure 2a and Figure 2bTaking enabling accessibility services in a text-to-speech application as an example, we will explain these two implementation methods.

[0116] refer to Figure 2a After installing the text-to-speech application 204 on their mobile phone, the user can click the icon of the settings application 201 to launch it. The phone will then display the settings application 201 page in response to the user's action, which may include a button for accessibility 202. If the user clicks the accessibility 202 button, the phone will display the accessibility 202 page in response to the user's action, which may include a button for accessibility (i.e., accessibility services) 203. If the user clicks the accessibility 203 button, the phone will display the accessibility 203 page in response to the user's action, which may include a button for the text-to-speech application 204.

[0117] Since the accessibility service is not used by default after the user installs the Read Aloud app 204, the user needs to manually enable the accessibility service. At this time, the button of the Read Aloud app 204 can be displayed as closed.

[0118] If the user clicks the button on the reading application 204, the phone will respond to the user's action by displaying page 205 of the reading application 204. Page 205 may display a switch button 206 in a closed state, which allows the user to enable / disable the reading application 204's accessibility services. The user can click the switch button 206 to toggle its on / off state. Page 205 may also display a description 207 of the reading application 204 and a schematic diagram 208 of the phone's user interface after the user has authorized the reading application 204.

[0119] If the user clicks the switch button 206, the phone responds to the user's operation by updating the page of the reading application 204 from page 205 to page 209. Page 209 can display the switch button 206 in the on state, as well as an interface diagram 208 and a floating window 210. The floating window 210 can include multiple controls for controlling the usage state of the reading application 204, such as controls for pausing / starting reading.

[0120] refer to Figure 2bThe phone's control center page may display a control switch 211 for the reading application 204. Control switch 211 allows the user to enable / disable the reading application 204's accessibility services. If the user clicks control switch 211, the phone responds to the user's action and displays page 212 of the reading application 204. This page 212 may display a switch button 213 in an off state, which allows the user to enable / disable the reading application 204's accessibility services. The user can click switch button 213 to toggle its on / off state.

[0121] Since both implementation methods are used to enable / disable the reading application 204's accessibility services, page 212 can be the same as page 205, and switch button 213 can be switch button 206. Thus, if the user clicks switch button 213, the phone responds to the user's action by updating the display of the reading application 204's page on page 212. The updated page content can be as follows: Figure 2a As shown on page 209 of the document.

[0122] After a user grants accessibility permissions to an accessibility application, if accessibility service management is not implemented (i.e., the use of accessibility services by the accessibility application is not controlled), the accessibility application can receive all accessibility events from the system. The following example illustrates the process by which an accessibility application obtains accessibility events.

[0123] For example, referring to Figure 2, an electronic device may have application A, application B and accessibility application C installed, and the user has enabled accessibility application C to use accessibility services.

[0124] If an electronic device is running application A and a user interacts with the user interface of application A, causing a change in the interface state, the system can generate a corresponding accessibility event 1 to reflect this change and send accessibility event 1 to the accessibility service. If accessibility service control is not implemented, the accessibility service can send accessibility event 1 to accessible application C, which can then obtain the interface state change information carried in accessibility event 1. After receiving accessibility event 1, accessible application C can either remain still or provide accessible user interaction functions based on it.

[0125] For example, suppose accessible application C has the function of automatically receiving electronic red envelopes. If application A experiences a change in its interface state due to receiving an electronic red envelope, the system can generate a corresponding accessibility event and notify the accessibility service. The accessibility service then sends the accessibility event to accessible application C. Based on the received accessibility event, accessible application C can issue a command through the accessibility service to automatically receive the electronic red envelope. In this way, the electronic device can automatically receive the electronic red envelope for the user without any user interaction.

[0126] Based on the same implementation principle, if the electronic device subsequently switches to running application B, and the user performs an operation on the user interface of application B, causing a change in the interface state of application B, the system can generate a corresponding accessibility event 2 to reflect this interface state change and send accessibility event 2 to the accessibility service. If accessibility service control is not implemented, the accessibility service can send accessibility event 2 to accessible application C, and accessibility application C can obtain the interface state change information carried in accessibility event 2. After obtaining accessibility event 2, accessibility application C can either remain inactive or provide accessible user interaction functions based on it.

[0127] In some embodiments, the user interface of application A and the user interface of application B may include a user information page (such as a password control page), where users can enter sensitive user information such as account and password.

[0128] refer to Figure 4 , Figure 4 This illustrates a password control page, specifically a recharge page that includes a payment password control. For example... Figure 4 As shown, an application installed on a user's mobile phone can provide recharge services to the user by displaying a recharge page. While the recharge page is displayed on the user's mobile phone, the user can enter a payment password in the payment pop-up window 401 on the recharge page to request payment. The user's mobile phone can display the entered payment password in an encrypted manner in the payment pop-up window 401.

[0129] When a user enters their payment password, the app's operation causes a change in the state of the recharge page. The system can then generate a corresponding accessibility event to describe this interface state change and send the event to the accessibility service. If accessibility service control is not implemented, the accessibility service can distribute the accessibility event to every accessibility app on the user's phone, allowing those apps to obtain information about the recharge page's state change.

[0130] As shown above, without accessibility service control, once a user grants access to an accessibility application, the application can obtain user interface state change information through the accessibility service, posing a risk of user information leakage. For example, even if an accessibility application is actually malicious, it can obtain user interface state change information indiscriminately with user authorization.

[0131] To address the risk of user information leakage, see [link / reference] Figure 5 One embodiment of this application provides a method for managing accessibility services, applied to an electronic device. The electronic device has a first application and a second application installed, the second application supporting accessibility services and having accessibility service usage permissions. Based on this, the management method may include the following steps 501 and 502:

[0132] Step 501: When the interface state of the first application changes, obtain an accessibility event to describe the interface state change.

[0133] Step 502: If the second application is in the first whitelist, send the accessibility event to the second application through the accessibility service.

[0134] Feasibly, the identifiers of trusted accessibility applications can be set in a whitelist. The whitelist can record the identifiers of one or more accessibility applications, or it can be empty, thus limiting the accessibility events reflecting changes in the user interface state that can be received by accessibility applications in the whitelist. In this way, the accessibility service can send accessibility events to accessibility applications in the whitelist as needed, and not to accessibility applications outside the whitelist, thereby achieving accessibility service control.

[0135] Figure 5 The control method shown can perform accessibility service control based on a whitelist, selectively allowing accessibility applications on the whitelist to use accessibility services while prohibiting accessibility applications outside the whitelist from using accessibility services. This differentiates the sending of accessibility events to trusted accessibility applications, thus preventing malicious application developers from obtaining user interface state change information and thus avoiding the risk of user information leakage.

[0136] Considering that accessibility incidents do not always involve user information pages, and that users install accessibility applications to use them for accessible user interaction functions, in order to support accessibility applications in providing accessible user interaction functions, while addressing the risk of user information leakage, accessibility service management can be achieved through general system control.

[0137] Feasibly, general system control can be achieved by using a system whitelist provided by the electronic device manufacturer, implementing accessibility service control only for user information pages, while not restricting accessibility services for other types of pages that are not on the system whitelist.

[0138] Figure 5 In one embodiment of the control method shown, different applications may not be distinguished, thus allowing for consistent system-wide control across all applications. In this way, whenever an application experiences a change in its interface state, it can be determined whether the currently displayed user interface is a user information page. If so, accessibility service control is performed based on the system whitelist; otherwise, accessibility events can be sent to accessibility applications outside the system whitelist via accessibility services.

[0139] Figure 5 In another embodiment of the control method shown, different types of applications can be distinguished. Specifically, applications can be categorized into third-party open applications and general applications based on whether they can request customized accessibility protection (i.e., request customized control). In this way, customized control can be applied to third-party open applications as needed, while general applications are subject to general system control instead of customized control. By distinguishing between third-party open applications and general applications, the personalized control needs of application developers can be met.

[0140] Feasible, customized control can be implemented as follows: if a third-party open application requests accessibility protection, accessibility service control is performed based on the application whitelist, regardless of whether the user interface includes user information-related controls. The application whitelist can be a whitelist provided by the application developer, or, if the application developer does not provide a whitelist, the system whitelist can be used as the application whitelist.

[0141] In one implementation method 1, which distinguishes different types of applications, customized control can be applied only to third-party open applications.

[0142] In another implementation method (2) that differentiates between different types of applications, not only customized controls can be applied to third-party open applications, but also general system controls can be applied during periods when no customized controls are applied. This combines customized controls with general system controls to achieve seamless service control when electronic devices run third-party open applications.

[0143] By implementing general system controls on third-party open applications during the non-customized control period, it is possible to at least ensure that users will not face the risk of information leakage when using third-party open applications during the non-customized control period, even if the developers of the third-party open applications do not have specific control requirements during the non-customized control period. At the same time, it is possible to minimize the impact on the accessibility of applications in providing accessible user interaction functions.

[0144] Figure 5The specific technical implementation of the control method shown can be found in the descriptions of other embodiments of this application, and will not be repeated here.

[0145] Based on the above, taking the distinction between different types of applications as an example, see [link / reference]. Figure 6 One embodiment of this application provides another method for managing accessibility services, which is applied to electronic devices and may include the following steps 601 to 608.

[0146] Step 601: When the interface state of the first application installed on the electronic device changes, an accessibility event describing the interface state change is obtained.

[0147] Step 602: Determine whether the second application installed on the electronic device is in the system blacklist. If the second application supports accessibility services and has accessibility service permissions, do not send accessibility events to the second application; otherwise, proceed to step 603.

[0148] In this step, accessibility events are not sent to accessibility applications on the system blacklist. For accessibility applications not on the blacklist, further control is implemented based on the application whitelist / system whitelist, combined with the specific operation of the application.

[0149] Step 603: Determine whether the first application is an application that can request accessibility protection. If yes, proceed to step 604; otherwise, proceed to step 608.

[0150] This step is used to determine whether the currently running application is a third-party open application or a general application. If it is a third-party open application, step 604 is executed to further determine whether it is under the customized control period of the third-party open application. If it is a general application, step 608 is executed to perform general control based on the system whitelist for general applications.

[0151] Step 604: Determine whether the first application has requested accessibility protection when the interface state of the first application changes. If yes, proceed to step 605; otherwise, proceed to step 608.

[0152] This step is used to determine whether the application is currently under customized control. If it is under customized control, step 605 is executed to perform customized control based on the application whitelist. If it is not under customized control, step 608 is executed to perform general control based on the system whitelist during the non-customized control period of the application.

[0153] Step 605: Determine whether the first application provides a whitelist. If yes, proceed to step 606; otherwise, proceed to step 607.

[0154] This step is used to determine whether the currently running application is a third-party open application that provides a whitelist or a third-party open application that does not provide a whitelist. If it is the former, step 606 is executed to perform customized control based on the application whitelist provided by the third-party open application. If it is the latter, step 607 is executed to perform customized control using the system whitelist as the application whitelist provided by the third-party open application.

[0155] Step 606: Determine whether the second application is in the whitelist provided by the first application. If so, send the accessibility event to the second application through the accessibility service; otherwise, do not send the accessibility event to the second application.

[0156] This step is used to customize control based on the whitelist provided by the third-party open applications, so as to restrict accessibility applications in the whitelist to use accessibility services, while accessibility applications outside the whitelist cannot use accessibility services.

[0157] Step 607: Determine whether the second application is in the system whitelist. If so, send the accessibility event to the second application through the accessibility service; otherwise, do not send the accessibility event to the second application.

[0158] This step is used to customize control based on the application whitelist (replaced by the system whitelist) for third-party open applications that do not provide a whitelist, so as to restrict accessibility applications in the whitelist to use accessibility services, while accessibility applications outside the whitelist cannot use accessibility services.

[0159] Step 608: Determine whether the user interface of the first application includes user information-related controls when the interface state changes. If yes, proceed to step 607; otherwise, send the accessibility event to the second application through the accessibility service.

[0160] This step is used to perform general control based on the system whitelist for general applications, or to perform general control based on the system whitelist during the non-customized control period of third-party open applications, so as to restrict accessibility applications in the whitelist to obtain the status change information of the user information page, while accessibility applications outside the whitelist cannot obtain the status change information of the user information page, but can obtain the status change information of non-user information pages.

[0161] Figure 6 The embodiments shown can flexibly adjust the control strategy for accessibility services to meet the personalized control needs of application developers and the general control needs of the system to deal with the risk of user information leakage, and to minimize the impact on the accessibility application's ability to provide users with accessible user interaction functions.

[0162] Figure 6The illustrated embodiment is merely an example of the technical implementation of this application, and it may also include more or fewer steps. For example, if blacklist control is not involved, step 602 can be omitted, and step 603 can be executed directly after step 601.

[0163] The software systems of electronic devices can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. Taking the layered architecture of the Android system as an example, combined with... Figure 7 The software structure diagram of the electronic device shown is used as an example to illustrate the accessibility service management process of this application embodiment, taking the differentiation of different types of applications as an example. In one embodiment, the electronic device can be the aforementioned electronic device 100.

[0164] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom: application layer, application framework layer, system library and runtime layer, hardware abstraction layer (HAL), and kernel layer.

[0165] The application layer can include a series of application packages, which can be divided into three categories: accessible applications such as text-to-speech applications, third-party open applications such as instant messaging applications and banking applications, and general applications such as camera, gallery, calendar, call, and map.

[0166] After installing accessibility applications, users can use methods such as Figure 2a or Figure 2b The shown activation method enables accessibility applications to access accessibility services. Once authorized by the user, accessibility applications can run in the background and, under controlled and permitted conditions, use accessibility services to obtain accessibility events that reflect changes in the user interface state. They can then provide accessible user interaction functions either without user interaction or based on these events.

[0167] Electronic devices can provide accessibility protection interfaces to each third-party open application, allowing these applications to request accessibility protection by calling these interfaces, thereby triggering the electronic device to execute the aforementioned customized controls. If no accessibility protection interface is provided to general applications, the electronic device can execute the aforementioned general system controls when the general applications are running.

[0168] Feasibly, third-party open applications can call the accessibility protection interface as needed at the start of runtime to request accessibility protection, or call the accessibility protection interface when displaying a specific page to trigger customized control during the execution of the interface call by the electronic device.

[0169] Unlike general system accessibility management, which relies on a system whitelist, customized accessibility management relies on an application whitelist. Based on whether an application provides its own whitelist, third-party open applications can be divided into two categories: those that provide a whitelist, allowing for customized management during their operation; and those that do not provide a whitelist, in which case the system whitelist can be used as their whitelist for customized management. In one example... Figure 7 Instant messaging applications in China offer application whitelists, while banking applications do not.

[0170] The application framework layer provides an Application Programming Interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 7 As shown, the application framework layer may include accessibility services, accessibility service management modules, view systems, notification managers, resource managers, window managers, content providers, etc.

[0171] The accessibility service management module may include an application whitelist setting module, an accessibility protection execution module, a system notification and prompt module, a general control module, and a system black / whitelist management module.

[0172] For any third-party open application that provides a whitelist, the application whitelist setting module can set the provided whitelist as the application whitelist of that third-party open application, so as to support the implementation of accessibility service control based on its application whitelist during the customized management of that third-party open application.

[0173] Developers of electronic devices can distribute system whitelists via the system cloud to support general system management. In addition to system whitelists, developers can also distribute system blacklists via the system cloud. Correspondingly, the system black / whitelist management module can manage black / whitelists distributed by the system cloud (initial distribution or subsequent updates).

[0174] The general management module can identify whether the current page on the device is a user information page based on the window mechanism provided by the operating system. This window mechanism can be configured with information such as application window switching, which window appears after switching, and which controls are included in the switched window.

[0175] Feasibly, the accessibility protection execution module can trigger accessibility services on demand to perform customized control or general system control, depending on whether the currently running application is a third-party open application or a general application, whether the third-party open application calls the accessibility protection interface, and whether the application's current user interface is a user information page.

[0176] The accessibility service can run in the background, receiving accessibility events from the system and distributing them to accessible applications. Based on a system blacklist, the service can prevent accessibility events from being sent to blacklisted applications, effectively prohibiting them from receiving any accessibility events. For accessible applications not on the blacklist, further adjustments can be made through customized controls and on-demand execution of general system controls to determine whether to send accessibility events to them.

[0177] The accessibility protection execution module can notify the accessibility service to begin customized control when it determines that customized control based on application whitelists needs to be started. This triggers the accessibility service to send accessibility events to accessibility applications that are on the application whitelist but not on the system blacklist, based on the system blacklist and application whitelists, instead of sending them to other accessibility applications. Furthermore, the accessibility protection execution module can notify the accessibility service to end customized control when it determines that customized control needs to be ended.

[0178] The accessibility protection execution module can notify the accessibility service to initiate system-wide general control based on the system whitelist when it determines that such control needs to be started. This triggers the accessibility service to send accessibility events related to changes in the user information page's interface state to accessibility applications that are on the system whitelist but not on the system blacklist, based on the system blacklist and system whitelist, instead of sending them to other accessibility applications. Furthermore, the accessibility protection execution module can notify the accessibility service to terminate system-wide general control when it determines that such control needs to be terminated.

[0179] When an accessibility application is prohibited from using accessibility services due to accessibility service control measures, the system notification module can output a corresponding notification message to inform the user of the control result. Based on the notification message, the user can request to modify the system blacklist / whitelist as needed. The system blacklist / whitelist management module can respond to user requests and update the system blacklist / whitelist accordingly to meet users' personalized control needs on top of the system's general control measures.

[0180] Figure 7In the illustrated embodiment, the general control module within the accessibility service management module can detect whether the user interface is a user information page during general system control periods (including the operation of general applications and the non-customized control periods of third-party open applications). In other embodiments, the accessibility service management module may not include the general control module, in which case the user information page is not identified by the accessibility service management module, but by the accessibility service based on a window mechanism.

[0181] Feasiblely, accessibility services can use a window mechanism to identify user information pages by judging window switching in the user interface. If a user information page is identified during general system control or non-customized control, accessibility service control based on the system whitelist can be automatically started, and accessibility service control based on the system whitelist can be automatically stopped when a non-user information page is identified.

[0182] A view system includes visual controls, such as password controls, controls for displaying text, and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0183] The notification manager allows applications to display notifications in the status bar. These can be used to convey informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users that an accessibility app is disabled from using its accessibility services, to announce download completion, or to send message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the device, or flashing indicator lights.

[0184] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and so on.

[0185] The window manager is used to manage windowed applications. It can retrieve screen size, determine if a status bar is present, lock the screen, and capture screenshots, among other things.

[0186] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0187] The system library and runtime layer includes the system library and the Android Runtime.

[0188] System libraries can include multiple functional modules. For example: 3D graphics processing libraries (e.g., OpenGL ES), surface managers, 2D graphics engines (e.g., SGL), media libraries, etc.

[0189] The 3D graphics processing library is used for 3D graphics drawing, image rendering, compositing, and layer processing. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The 2D graphics engine is the drawing engine for 2D graphics. The media library supports playback and recording of various common audio and video formats, as well as still image files. The media library supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0190] The Android runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries comprise two parts: one part contains the functionalities that Java calls, and the other part consists of the Android core libraries. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0191] The Hardware Abstraction Layer (HAL) acts as a bridge between software and hardware. It encapsulates the underlying hardware driver, providing a generic interface for the application framework to call the driver. The HAL is an abstract interface for the device kernel driver, providing application programming interfaces for accessing the underlying device to higher-level Java API frameworks. The HAL contains multiple library modules, each implementing an interface for a specific type of hardware component.

[0192] The kernel layer is the layer between hardware and software. It contains at least camera drivers, display drivers, Bluetooth drivers, audio drivers, and sensor drivers. Display drivers control the display screen to show images; sensor drivers control the operation of multiple sensors, such as pressure sensors and touch sensors.

[0193] Under the aforementioned multi-layer architecture, electronic devices also have a hardware layer, which may include the aforementioned electronic device hardware components, such as displays, sensors, indicators, etc.

[0194] It should be noted that, although the embodiments of this application are based on... The system is used as an example for explanation, but its basic principles also apply to systems based on... Electronic devices with operating systems, etc.

[0195] Understandable Figure 7 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0196] For example, based on Figure 7 The software structure of the electronic device shown above, taking the differentiation of different types of applications as an example, illustrates the implementation process of accessibility service management on the electronic device.

[0197] Assume an electronic device has the following applications installed: Accessibility Application A, Accessibility Application B, Accessibility Application C, Accessibility Application D, Instant Messaging Application, Banking Application, and Map Application. The Map Application is a general-purpose application, while the Instant Messaging Application and the Banking Application are both third-party open applications. The Instant Messaging Application provides an application whitelist, while the Banking Application does not.

[0198] Assume that accessibility application A is on the system blacklist, accessibility application B is on the application whitelist provided by the instant messaging application, accessibility application C is on the system whitelist, and accessibility application D is not on either list, and the user has authorized each accessibility application to use accessibility services. In this case, the following control example could exist.

[0199] 6.1 General control over general application execution systems

[0200] 6.1.1 Control over changes in interface state on non-user information pages of general applications

[0201] If a non-user information page in a map application experiences a change in its interface state, the system can generate a corresponding accessibility event 01 and notify the accessibility service. If the general management module detects that the currently displayed user interface is not a user information page, the accessibility protection execution module can determine that control based on the system whitelist is unnecessary. Therefore, the accessibility service can distribute accessibility event 01 to accessibility applications B, C, and D based on the system blacklist, but not to accessibility application A.

[0202] 6.1.2 Control over changes in the user information page of general applications

[0203] Assuming the user information page of a map application experiences a change in its interface state, the system can generate a corresponding accessibility event 02 and notify the accessibility service. The general management module detects that the currently displayed user interface is the user information page, and the accessibility protection execution module determines that management based on the system whitelist is required. Therefore, it can notify the accessibility service to begin management based on the system whitelist. In response to the notification, the accessibility service can, based on the system blacklist, not distribute accessibility event 02 to accessibility application A, and based on the system whitelist, distribute accessibility event 02 to accessibility application C, but not to accessibility applications B and D.

[0204] 6.2 Implement customized control over third-party open applications

[0205] 6.2.1 Third-party open applications provide control when a whitelist is available.

[0206] Assuming an instant messaging application calls the accessibility protection interface and there is a change in the interface state, the system can generate a corresponding accessibility event 03 and notify the accessibility service of accessibility event 03. The accessibility protection execution module can determine that control based on the application whitelist is required, and thus notify the accessibility service to begin control based on the application whitelist. In response to the notification, the accessibility service can, based on the system blacklist, not distribute accessibility event 03 to accessibility application A, and based on the application whitelist provided by the instant messaging application, distribute accessibility event 03 to accessibility application B, but not to accessibility applications C and D.

[0207] 6.2.2 Control when third-party open applications do not provide a whitelist

[0208] Assuming the banking application calls the accessibility protection interface and there is a change in the interface state, the system can generate a corresponding accessibility event 04 and notify the accessibility service of accessibility event 04. The accessibility protection execution module can determine that control based on the application whitelist is required, and thus notify the accessibility service to begin control based on the application whitelist. In response to the notification, the accessibility service can, based on the system blacklist, not distribute accessibility event 04 to accessibility application A, and use the system whitelist as the application whitelist provided by the banking application, distribute accessibility event 04 to accessibility application C, but not to accessibility applications B and D.

[0209] 6.3 Implement general system control for third-party open applications during non-customized control periods.

[0210] 6.3.1 Management of Interface State Changes in Non-User Information Pages of Third-Party Open Applications

[0211] Assuming the instant messaging application (or banking application) does not call the accessibility protection interface, and its non-user information page experiences interface state changes, the system can generate a corresponding accessibility event 05 and notify the accessibility service of accessibility event 05. If the general management module detects that the currently displayed user interface is not a user information page, the accessibility protection execution module can determine that no control based on the system whitelist is needed. Therefore, the accessibility service can distribute accessibility event 05 to accessibility applications B, C, and D based on the system blacklist, but not to accessibility application A.

[0212] 6.3.2 Management of User Information Page Changes in Third-Party Open Applications

[0213] Assuming an instant messaging application (or banking application) does not call the accessibility protection interface, and its user information page experiences interface state changes, the system can generate a corresponding accessibility event 06 and notify the accessibility service of accessibility event 06. The general management module detects that the currently displayed user interface is the user information page, and the accessibility protection execution module can determine that control based on the system whitelist is required. Therefore, it can notify the accessibility service to begin control based on the system whitelist. In response to the notification, the accessibility service can, based on the system blacklist, not distribute accessibility event 06 to accessibility application A, and based on the system whitelist, distribute accessibility event 06 to accessibility application C, but not to accessibility applications B and D.

[0214] As can be seen from the above examples, this application embodiment can prevent untrusted applications from using accessibility services by implementing accessibility service control, thus addressing the risk of user information leakage. Furthermore, based on different application types, whether the application provides a whitelist, and by combining customized control with general system control, this application embodiment can flexibly adjust the control strategy for accessibility services to meet the personalized control needs of application developers and the general system control needs to address the risk of user information leakage, while minimizing the impact on accessibility applications providing accessible user interaction functions.

[0215] Based on the above, other embodiments of this application can also achieve the purpose of barrier-free service management in scenarios such as not distinguishing between different types of applications, not subjecting third-party open applications to general system management, and not setting up system blacklists, in order to deal with the risk of user information leakage. This embodiment will not be elaborated here.

[0216] This application provides a management device for accessibility services, applied to an electronic device. The electronic device has a first application and a second application installed. The second application supports accessibility services and has accessibility service usage permissions. The management device includes: an acquisition module, used to acquire accessibility events describing the interface state change when the interface state of the first application changes; and a processing module, used to send the accessibility events to the second application through the accessibility service if the second application is in the first whitelist.

[0217] This application also provides a chip, which is installed in an electronic device. The chip includes a processor for executing computer program instructions stored in a memory, wherein when the computer program instructions are executed by the processor, the chip is triggered to execute the method steps provided in any method embodiment of this application.

[0218] This application also proposes a terminal device, which includes a communication module, a memory for storing computer program instructions, and a processor for executing the program instructions. When the computer program instructions are executed by the processor, the terminal device is triggered to execute the method steps provided in any method embodiment of this application.

[0219] This application also provides an electronic device, which includes multiple antennas, a memory for storing computer program instructions, a processor for executing the computer program instructions, and a communication device (such as a communication module that can implement 5G communication based on the NR protocol). When the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps provided in any method embodiment of this application.

[0220] Specifically, in the embodiments of this application, one or more computer programs are stored in the aforementioned memory, and the one or more computer programs include instructions that, when executed by the aforementioned device, cause the aforementioned device to perform the method steps described in the embodiments of this application.

[0221] Furthermore, the devices, apparatuses, and modules described in the embodiments of this application may be implemented by computer chips or physical entities, or by products with certain functions.

[0222] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0223] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0224] Specifically, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the method steps provided in this application.

[0225] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method steps provided in this application.

[0226] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0227] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0228] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0229] An integrated unit implemented as a software functional unit can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (such as a personal computer) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0230] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0231] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0232] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments of this application can be implemented using electronic hardware, computer software, or a combination of electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0233] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the same or similar parts between the various embodiments of this application can be referred to mutually. For example, the specific working processes of the systems, devices, and units described in the embodiments of this application can be referred to the corresponding processes in the method embodiments of this application, and will not be repeated here.

[0234] The above description is merely a specific embodiment of this application and is not intended to limit this application. The scope of protection of this application shall be determined by the claims.

Claims

1. A method for managing accessible services, characterized in that, Applied to an electronic device, the electronic device having a first application and a second application installed, the second application supporting accessibility services and having access to the accessibility services; The control methods include: When the interface state of the first application changes, an accessibility event describing the interface state change is obtained; If the second application is in the first whitelist, the accessibility event is sent to the second application through the accessibility service; The control method further includes at least one of the following features 1 and 2: Feature 1: Determine whether the user interface of the first application includes user information-related controls when the interface state changes; If the user interface includes user information-related controls, then the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist is executed. Feature 2: If the first application is an application that can request accessibility protection, determine whether the first application has requested accessibility protection when the interface state change of the first application exists; If the first application has requested accessibility protection, the step of sending the accessibility event to the second application through the accessibility service is executed if the second application is in the first whitelist.

2. The method according to claim 1, characterized in that, Based on feature 1, the control method further includes: using the system whitelist as the first whitelist, and performing the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

3. The method according to claim 1, characterized in that, Based on feature 1, the control method further includes: If the first application is not an application that can request accessibility protection, then the step of determining whether the user interface of the first application includes user information-related controls when the interface state changes is performed is executed.

4. The method according to claim 1, characterized in that, Based on feature 2, the control method further includes: If the first application does not request accessibility protection, determine whether the user interface of the first application includes user information-related controls when the interface state changes; If the user interface includes user information-related controls, then if the second application is included in the system whitelist, the accessibility event is sent to the second application through the accessibility service.

5. The method according to claim 1 or 4, characterized in that, Based on feature 2, if the first application provides a whitelist, then the whitelist provided by the first application is used as the first whitelist; and / or, Based on feature 2, if the first application does not provide a whitelist, then the system whitelist shall be used as the first whitelist.

6. The method according to claim 1 or 4, characterized in that, Based on feature 2, the time period during which the first application requests accessibility protection is either the time period during which the first application is running or the time period during which the first application displays the settings user interface.

7. The method according to any one of claims 1, 3-4, characterized in that, If the electronic device provides an accessibility protection interface to the first application, then the first application is an application that can request accessibility protection. If the electronic device does not provide an accessibility protection interface to the first application, then the first application is not an application capable of requesting accessibility protection. The accessibility protection interface is used for application calls to request accessibility protection.

8. The method according to any one of claims 1, 3-4, characterized in that, The control method also includes: Through the accessibility service or the module in the electronic device used to manage the accessibility service, the step of determining whether the user interface includes user information-related controls when the interface state of the first application changes is executed according to the set window mechanism; The window mechanism is used to describe the switching of the interface window of the first application and the controls in the interface window of the first application.

9. The method according to any one of claims 1-4, characterized in that, The control method also includes: If the second application is not in the system blacklist, perform the step described above: if the second application is in the first whitelist, send the accessibility event to the second application through the accessibility service.

10. A control device for accessible services, characterized in that, Applied to an electronic device, the electronic device having a first application and a second application installed, the second application supporting accessibility services and having access to the accessibility services; The control device includes: The acquisition module is used to acquire accessibility events that describe the interface state change when there is an interface state change in the first application. The processing module is configured to send the accessibility event to the second application through the accessibility service if the second application is in the first whitelist. The control device is also used to perform at least one of the following features 1 and 2: Feature 1: Determine whether the user interface of the first application includes user information-related controls when the interface state changes; If the user interface includes user information-related controls, then the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist is executed. Feature 2: If the first application is an application that can request accessibility protection, determine whether the first application has requested accessibility protection when the interface state change of the first application exists; If the first application has requested accessibility protection, the step of sending the accessibility event to the second application through the accessibility service is executed if the second application is in the first whitelist.

11. A chip, characterized in that, include: A processor for executing computer program instructions stored in memory, wherein when the computer program instructions are executed by the processor, the chip is triggered to perform the method of any one of claims 1-9.

12. An electronic device, characterized in that, The electronic device includes one or more memories for storing computer program instructions and one or more processors, wherein when the computer program instructions are executed by the one or more processors, the electronic device is triggered to perform the method as described in any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-9.

Images