Barrier-free service management and control method and device, chip, electronic equipment and medium

Abstract

The invention provides a barrier-free service management and control method and device, a chip, electronic equipment and a medium, the management and control method is applied to the electronic equipment, the electronic equipment is provided with a first application and a second application, and the second application supports barrier-free service and has the use permission of the barrier-free service; the management and control method comprises the following steps: when a first application has an interface state change, obtaining a barrier-free event for describing the interface state change; and if the second application is in the first white list, sending the barrier-free event to the second application through the barrier-free service. The risk of user information leakage can be avoided.

Description

Technical Field

[0001] The present application relates to the technical field of electronic equipment, and in particular to a method, device, chip, electronic device, and medium for managing and controlling accessibility services. Background Art

[0002] The operating system of an electronic device can provide users with accessibility services (AccessibilityService). Accessibility services can be used to provide accessible user interaction functions for users with disabilities or users who are temporarily unable to fully interact with the device. For example, a reading application that supports accessibility services can provide users with the function of reading screen content aloud.

[0003] When a user operates the user interface displayed on an electronic device, the accessibility service can obtain accessibility events (AccessibilityEvents) generated by the system. Accessibility events can be used to indicate changes in the state of the user interface. If the user has enabled accessibility applications (or applications that support accessibility services) to use accessibility services, the accessibility applications can receive accessibility events sent by the accessibility service and respond accordingly to provide accessible user interaction functions.

[0004] Because accessibility apps can obtain user interface status change information through accessibility services after users grant them permission, there is a risk of user information leakage. For example, even if an accessibility app is actually malicious, it can obtain user interface status change information indiscriminately with user authorization. Summary of the Invention

[0005] The present application provides a method, device, chip, electronic device and medium for managing and controlling accessibility services, which can avoid the risk of user information leakage.

[0006] The operating system of an electronic device (such as the Android system) can provide accessibility services for users. The accessibility service can run in the background and can receive accessibility events sent by the system. Accessibility events can represent a series of state changes in the user interface, such as focus changes, input content changes, button clicks, etc. For any changes in the user interface, corresponding accessibility events will be generated, and the accessibility events can be notified to the accessibility service by the system. If accessibility service control is not implemented (that is, the use of accessibility services by accessibility applications is not controlled), after the user opens the accessibility application's permission to use the accessibility service, the accessibility application can receive accessibility events distributed by the accessibility service, that is, the accessibility application can obtain all accessibility events of the system.

[0007] Because accessible applications can obtain user interface state change information through accessibility events, there is a risk of user information leakage. Therefore, to address the risk of user information leakage, it is possible to control the use of accessibility services by accessible applications.

[0008] In a first aspect, an embodiment of the present application provides a method for managing and controlling accessibility services, which is applied to an electronic device, wherein a first application and a second application are installed in the electronic device, and the second application supports accessibility services and has permission to use the accessibility services; the management and control method includes: when there is an interface state change in the first application, obtaining an accessibility event for describing the interface state change; if the second application is in the first whitelist, sending the accessibility event to the second application through the accessibility service.

[0009] In the embodiment of the present application, the second application can be any barrier-free application, and the user has enabled its permission to use barrier-free services, and the first application can be any common application (ie, non-barrier-free application).

[0010] When the electronic device runs the first application in the foreground, the user interface of the first application can be displayed, and the user can operate the electronic device as needed to use the first application. For example, the user can use the touch screen of the electronic device to click on a control on the user interface of the first application, enter user information (such as a login account password, payment password, or other sensitive user information) on the user interface, perform a sliding operation on the user interface, etc.

[0011] User operations on the user interface can trigger the operating system to generate accessibility events. The accessibility service can intercept the accessibility events generated by the operating system and distribute the intercepted accessibility events to the accessibility applications in the whitelist.

[0012] In one embodiment, the accessibility service can set the status value of user operations (such as clicking, double-clicking, sliding the screen, etc.) through setServiceinfo (service information setting function), so that when users operate the user interface, the accessibility service can intercept the accessibility events generated by the operating system (that is, obtain the event information of the accessibility events). The event information of the accessibility event can include the event type (EventType) and the event object (Source). The event type can indicate the type of user operation corresponding to the accessibility event; the event object can indicate the object of the user operation corresponding to the accessibility event, which can be a button, floating box, floating window, etc. on the user interface.

[0013] In addition to user actions on an application causing changes in the user interface state, non-user actions can also cause changes in the user interface state. For example, the user interface can display real-time information received by the application, triggering the system to generate corresponding accessibility events. In one embodiment, accessibility events can include application interface content such as control IDs (unique codes) and text to describe changes in the application's interface state.

[0014] Embodiments of the present application can implement accessibility service management based on a whitelist. The whitelist may contain the identifiers of one or more accessibility applications, or may not contain any accessibility application identifiers. This allows accessibility applications on the whitelist to receive accessibility events reflecting user interface state changes without sending accessibility events to accessibility applications not on the whitelist. In one embodiment, the whitelist may include accessibility applications commonly used by people with disabilities, thereby supporting the normal use of these accessibility applications while implementing accessibility service management.

[0015] It is feasible that one or more common applications and one or more accessibility applications may be installed on the electronic device. If multiple accessibility applications are installed and all are authorized by the user, then after receiving the above-mentioned accessibility event, the electronic device may determine in sequence or simultaneously whether each accessibility application is on the first whitelist, and distribute the accessibility event to each accessibility application on the whitelist based on the judgment result. After receiving the accessibility event, the accessibility application may not perform any operation or respond to the accessibility event to provide accessible user interaction functions.

[0016] In one example, an electronic device is installed with multiple accessibility applications such as a reading application, and the user allows these multiple accessibility applications to use accessibility services. When the electronic device uses a news application (here taking the news application as an example, it can also be other installed applications), the user interface of the electronic device can display news information and identification controls of the reading application, and the reading application provides the user with a service of reading the screen content in the set reading mode. If the user performs an upward swipe gesture on the touch screen to request an update of the news information, the system can generate a corresponding accessibility event and send the accessibility event to the accessibility service. Assuming that the whitelist corresponding to the news application only includes the reading application, the accessibility service will send the accessibility event to the reading application, and will not send the accessibility event to any other accessibility application.

[0017] In one embodiment, if the second application is not on the first whitelist, the step of sending the accessibility event to the second application via the accessibility service can be omitted. In other embodiments, while not sending the accessibility event to the accessibility application not on the whitelist, a corresponding prompt message can be displayed to inform the user of the accessibility service management results. In yet another embodiment, the user can also choose whether to request that the accessibility event be sent to the accessibility application not on the whitelist based on the prompt message.

[0018] It can be seen that the embodiments of the present application are based on the whitelist and can control the use of accessibility services by accessibility applications, so as to selectively allow accessibility applications in the whitelist to use accessibility services, while prohibiting (completely prohibiting or prohibiting within a certain range) accessibility applications outside the whitelist from using accessibility services, so as to differentially send accessibility events to trusted accessibility applications, thereby preventing application developers of malicious applications from obtaining user interface status change information, and posing a risk of user information leakage.

[0019] Taking into account that accessibility events do not always involve user information pages, and users install accessibility applications in order to use them to provide users with accessible user interaction functions, on the basis of addressing the risk of user information leakage, in order to support accessibility applications in providing users with accessible user interaction functions, system-wide control can be implemented to restrict the use of accessibility services by accessibility applications outside the system whitelist based on a whitelist, only for user information pages. For other types of pages, there is no restriction on the use of accessibility services by accessibility applications outside the system whitelist.

[0020] In other feasible implementations, if the first application provides an application whitelist, system-wide control can also be performed based on the application whitelist provided by the first application.

[0021] In an embodiment of the first aspect above, the management and control method also includes: determining whether the user interface of the first application includes user information-related controls when the interface state changes; if the user interface includes user information-related controls (that is, the user interface is a user information page), then executing the above-mentioned step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

[0022] During general system control, the system whitelist can be used as the whitelist corresponding to the first application, and accessibility events reflecting changes in the user information page status can be sent to the accessibility applications on the whitelist, but not to accessibility applications outside the whitelist. In one embodiment, if the user interface includes user information-related controls, the system whitelist is used as the first whitelist, and the above-mentioned steps of sending accessibility events to the second application via the accessibility service are performed if the second application is on the first whitelist.

[0023] In one embodiment of system general management, if the user interface does not include user information-related controls, so that the accessibility event does not involve a state change of the user information page, the accessibility event can be sent to the second application through the accessibility service, regardless of whether the second application is in the system whitelist.

[0024] For example, user information-related controls may be controls involving sensitive user information, such as password controls and account controls. If the user interface includes a password control, the user interface is a password control page, where users typically enter their password information (such as payment passwords, login passwords, etc.), causing accessibility events to involve state changes on the password control page.

[0025] Alternatively, the user information page can be identified by an accessibility service or a functional module independently developed by the developer. In one embodiment of the first aspect, the management method further includes: executing the aforementioned step of determining whether the user interface of the first application includes user information-related controls when the interface state of the first application changes, using the accessibility service or a module in the electronic device used to manage the accessibility service, according to a set window mechanism; the window mechanism is used to describe the switching of the interface window of the first application and the controls within the interface window of the first application.

[0026] The operating system of the electronic device may provide a window mechanism, which may include information such as application window switching, which window is the window after switching, and which controls are included in the window after switching. In this way, it is possible to identify whether the user interface currently displayed by the device is a user information page based on the window mechanism.

[0027] In one embodiment, the system whitelist can be a whitelist provided by the manufacturer of the electronic device, and the device manufacturer can update the provided whitelist in real time and send it to the electronic device. In a feasible implementation, the updated whitelist can be sent to the electronic device in real time via the system cloud.

[0028] In one embodiment, during general system control (i.e., accessibility service control based on a system whitelist), the electronic device can output a corresponding prompt message without sending the accessibility event to the second application. If the user requests to modify the system whitelist based on the prompt message, the system whitelist stored on the electronic device can be modified based on the user's needs. The modified system whitelist is a combination of the whitelist provided by the device manufacturer and the user's personalized whitelist. In this way, the system whitelists on different electronic devices can be different to meet not only the general system control requirements but also the personalized control requirements of the device user.

[0029] In one embodiment, the first application can be any application installed on the electronic device, that is, no distinction is made between individual applications, and system-wide control is always applied to all applications. In this way, when the interface state of any application changes, if the currently displayed user interface is a user information page, accessibility service control can be performed based on the system whitelist to restrict accessibility applications outside the system whitelist from using accessibility services. Otherwise, accessibility events can be sent to each accessibility application through the accessibility service, so that accessibility applications outside the system whitelist are not restricted from using accessibility services.

[0030] In another embodiment, different types of applications may also be differentiated. For example, applications may be divided into third-party open applications and general applications based on whether the applications can request customized accessibility protection (i.e., request customized control).

[0031] When distinguishing between different types of applications, for third-party open applications, we can respond to requests for customized accessibility protection from third-party open applications and implement customized controls based on the application whitelist to meet the individual needs of the application developer. For general applications, we implement general system controls based on the system whitelist.

[0032] During the customized control period, the application whitelist is used as the whitelist corresponding to the first application.

[0033] In an embodiment of the first aspect above, the management and control method also includes: if the first application is not an application that can request accessibility protection (that is, it is a general application, not a third-party open application), executing the above-mentioned step of determining whether the user interface of the first application includes user information-related controls when the interface state changes.

[0034] By implementing system-wide controls when running universal applications on electronic devices, it is possible to ensure that, without the need for specific controls by the developers of universal applications, there is at least no risk of user information being leaked when users use universal applications, and that accessible user interaction functions provided by accessible applications are minimally impacted.

[0035] As described above, general system control can be as follows: if the user interface includes user information-related controls, accessibility service control is performed based on the system whitelist, and accessibility events are sent to accessibility applications on the system whitelist through the accessibility service, but not to accessibility applications outside the system whitelist, thereby restricting accessibility applications outside the system whitelist from using the accessibility service. If the user interface does not include user information-related controls, accessibility applications outside the system whitelist are not restricted from using the accessibility service, and accessibility events can be sent to accessibility applications outside the system whitelist through the accessibility service.

[0036] In contrast, customized control can be: if a third-party open application requests accessibility protection, accessibility service control is performed based on the application whitelist, and accessibility events are sent to the accessibility applications in the application whitelist through the accessibility service, but not to the accessibility applications outside the application whitelist, so as to restrict the use of accessibility services by accessibility applications outside the application whitelist, regardless of whether the user interface at this time includes user information-related controls.

[0037] In one implementation method (denoted as implementation method 1) for distinguishing different types of applications, only customized control can be performed on third-party open applications.

[0038] That is, in implementation method 1, system general control is performed when running general applications, and when running third-party open applications, customized control is performed based on accessibility protection requests.

[0039] In another implementation method (implementation method 2) that distinguishes different types of applications, not only customized control can be implemented for third-party open applications, but also system-wide control can be implemented during non-customized control periods. In other words, customized control and system-wide control can be combined to implement accessibility service control when electronic devices run third-party open applications.

[0040] That is, in implementation method 2, system general control is performed when running general applications, and when running third-party open applications, customized control is performed based on accessibility protection requests, and system general control is performed during non-customized control.

[0041] By performing general system control on third-party open applications during the non-customized control period of third-party open applications, it is possible to ensure that there is no risk of user information leakage when users use third-party open applications during the non-customized control period, provided that the developers of the third-party open applications have no specific control requirements during the non-customized control period, and to minimize the impact on accessible applications in providing users with accessible user interaction functions.

[0042] It can be seen that by combining customized control and system general control, in addition to achieving the personalized control purpose required by application developers during the customized control period of third-party open applications, so as to avoid the risk of information leakage (that is, not limited to user information) when users use third-party open applications during this period, it is also possible to at least ensure that there is no risk of user information leakage when users use third-party open applications during the non-customized control period of third-party open applications, and to minimize the impact on accessible applications providing users with accessible user interaction functions.

[0043] Alternatively, developers of third-party open applications can choose to request accessibility protection during the operation of the third-party open application, or they can choose to request accessibility protection only when the third-party open application displays a certain user interface or user interfaces. Thus, in one embodiment of the first aspect above, the time period during which the first application requests accessibility protection (i.e., the customized control time period) is the time period during which the first application is running, or the time period when the first application displays a set user interface.

[0044] In an embodiment of the first aspect above, if the electronic device provides an accessibility protection interface to the first application, the first application is an application that can request accessibility protection (that is, the above-mentioned third-party open application); if the electronic device does not provide an accessibility protection interface to the first application, the first application is not an application that can request accessibility protection (that is, the above-mentioned general application); wherein the accessibility protection interface is used for the application to call to request accessibility protection.

[0045] It is feasible that electronic devices can provide accessibility protection interfaces to third-party open applications with customized management requirements, but not to general applications. In one implementation, third-party open applications can start / stop calling the accessibility protection interface to start / stop requesting accessibility protection.

[0046] In one embodiment, third-party open applications may be integrated with an accessibility protection interface, while general applications may not be integrated with the accessibility protection interface. Applications integrated with the interface are third-party open applications, and applications not integrated with the interface are general applications.

[0047] Corresponding to the above-mentioned implementation method 1 or implementation method 2, in an embodiment of the above-mentioned first aspect (recorded as embodiment a), the management and control method also includes: if the first application is an application that can request accessibility protection (that is, a third-party open application, not a general application), determining whether the first application has requested accessibility protection when there is a change in the interface state of the first application (that is, determining whether the current moment is in the customized management period); if the first application has requested accessibility protection, executing the above-mentioned step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

[0048] When an electronic device runs a third-party open application, the third-party open application can request accessibility protection to enable the electronic device to perform customized control. During the customized control period, for accessibility events generated when the interface state of the third-party open application changes, the accessibility events can be sent to the accessibility applications in the application whitelist instead of to the accessibility applications outside the application whitelist, so as to avoid the risk of information leakage when the user uses the third-party open application during the customized control period.

[0049] Alternatively, developers of third-party open applications can provide customized application whitelists, allowing different third-party open applications to have different application whitelists to meet the personalized control needs of different application developers. In this case, since the developer specifies not only the customized control period but also the control objects during the customized control period, the whitelist used during the customized control period can be the application whitelist. Thus, in one implementation of the above embodiment a, if the first application provides a whitelist, the first whitelist is the whitelist provided by the first application.

[0050] Alternatively, developers of third-party open applications may choose not to provide a customized application whitelist. In this case, since the developer specifies the customized control period but does not specify the control objects during the customized control period, the whitelist used during the customized control period can be the system whitelist, that is, the system whitelist is used as the application whitelist. Thus, in another implementation of the above embodiment a, if the first application does not provide a whitelist, the first whitelist is the system whitelist.

[0051] Corresponding to the above-mentioned implementation method 1, in an implementation method of the above-mentioned embodiment a, the management and control method also includes: if the first application does not request accessibility protection, the accessibility event can be sent to the second application through the accessibility service.

[0052] In implementation method 1, only customized control can be performed on third-party open applications. During the non-customized control period of third-party open applications, accessibility services can be used without restriction for accessibility applications outside the system whitelist, and accessibility events can be sent directly to accessibility applications outside the system whitelist.

[0053] Corresponding to the above-mentioned implementation method 2, in an implementation method of the above-mentioned embodiment a, the management and control method also includes: if the first application does not request accessibility protection, determining whether the user interface of the first application includes user information-related controls when the interface state changes; if the user interface includes user information-related controls, then when the system whitelist includes the second application, the accessibility event is sent to the second application through the accessibility service.

[0054] In Implementation Method 2, customized control can be implemented during the customized control period of third-party open applications, while general system control can be implemented during the non-customized control period of third-party open applications. By combining customized control with general system control, while meeting the personalized control needs of applications, general system control during the non-customized control period can be used to mitigate the risk of user information leakage during the non-customized control period.

[0055] In one embodiment, if the first application does not request accessibility protection and the user interface does not include user information-related controls, so that the accessibility events generated during non-customized control do not involve the user information page and there is no risk of user information leakage, the accessibility events can be sent to the second application through the accessibility service.

[0056] As described above, by setting up a whitelist, you can implement whitelist-based accessibility service management on demand, allowing accessibility apps on the whitelist to use accessibility services while prohibiting accessibility apps outside the whitelist from using accessibility services. The content of the whitelist can be set as needed by application developers and device manufacturers, and generally trusted accessibility apps are set to be on the whitelist, while untrusted accessibility apps are not on the whitelist.

[0057] During customized control, only whitelisted accessibility apps can use accessibility services; apps outside the whitelist cannot. During system control, when the device displays the user information page, only whitelisted accessibility apps can use accessibility services; apps outside the whitelist cannot. However, when the device displays non-user information pages, no control is applied. In this case, apps outside the whitelist can use accessibility services, even if they are known to be malicious.

[0058] To prevent malicious apps from using accessibility services under any circumstances, a blacklist can be set up. This blacklist can contain the identifiers of one or more accessibility apps, or it can contain no identifiers at all. This prevents blacklisted accessibility apps from receiving accessibility events reflecting user interface state changes, thereby preventing malicious apps from obtaining information about interface state changes.

[0059] In an embodiment of the first aspect above, the management and control method further includes: if the second application is not in the system blacklist, executing the above-mentioned step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

[0060] After generating an accessibility event, we can first determine whether the second application is on the system blacklist. If so, we won't send the accessibility event to the second application. Otherwise, we can then implement general system controls or customized controls based on the whitelist. This way, we can completely prohibit malicious applications from using accessibility services, while also restricting untrusted applications from using accessibility services in certain situations as needed.

[0061] The system blacklist can be a blacklist provided by the manufacturer of the electronic device, and the device manufacturer can update the provided blacklist in real time and send it to the electronic device. In a feasible implementation, the updated blacklist can be sent to the electronic device in real time via the system cloud.

[0062] In one embodiment, when an electronic device performs accessibility service control based on a system blacklist, it can output a corresponding prompt message without sending the accessibility event to the second application. If the user requests to modify the system blacklist based on the prompt message, the system blacklist stored on the electronic device can be modified based on the user's needs. That is, the modified system blacklist is a combination of the blacklist provided by the device manufacturer and the user's personalized blacklist. In this way, the system blacklists on different electronic devices can be different, so as to not only comply with the general system control but also meet the personalized control needs of the device user.

[0063] In the second aspect, an embodiment of the present application provides a device for controlling accessibility services, which is applied to an electronic device, wherein the electronic device is installed with a first application and a second application, the second application supports accessibility services and has permission to use accessibility services; the control device includes: an acquisition module, which is used to obtain an accessibility event for describing the interface state change when there is an interface state change in the first application; and a processing module, which is used to send the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

[0064] In a third aspect, an embodiment of the present application provides a chip, comprising: a processor for executing computer program instructions stored in a memory, wherein when the computer program instructions are executed by the processor, the chip is triggered to execute a method as described in any one of the first aspects.

[0065] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes one or more memories for storing computer program instructions, and one or more processors, wherein when the computer program instructions are executed by one or more processors, the electronic device is triggered to execute a method as described in any one of the first aspects.

[0066] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes any method as in the first aspect.

[0067] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is run on a computer, it enables the computer to execute any method as described in the first aspect.

[0068] The technical effects of the aforementioned aspects can be referenced with each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments.

[0070] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0071] Figure 2a and Figure 2b Schematic diagram of two implementation methods for enabling accessibility applications to access accessibility services, provided in embodiments of the present application;

[0072] Figure 3 A schematic diagram of an accessible application using an accessible service according to an embodiment of the present application;

[0073] Figure 4 A schematic diagram of a password control page provided in an embodiment of the present application;

[0074] Figure 5 A flowchart of a method for managing and controlling accessibility services provided in an embodiment of the present application;

[0075] Figure 6 A flowchart of another method for managing and controlling accessibility services provided in an embodiment of the present application;

[0076] Figure 7 A software structure block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0078] It should be clear that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0079] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0080] It should be understood that the term "at least one" used in this article refers to one or more, and "plurality" refers to two or more. The term "and / or" used in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. Among them, A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0081] It should be understood that although the terms "first," "second," etc. may be used to describe the set thresholds in the embodiments of the present application, these set thresholds should not be limited to these terms. These terms are merely used to distinguish the set thresholds from each other. For example, without departing from the scope of the embodiments of the present application, the first set threshold may also be referred to as the second set threshold, and similarly, the second set threshold may also be referred to as the first set threshold.

[0082] The method provided in any embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, personal digital assistants (PDAs), desktop computers, laptop computers, notebook computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, and wearable devices. The present application does not impose any special restrictions on the specific forms of the above-mentioned electronic devices.

[0083] The barrier-free service management and control method provided in any embodiment of the present application can be applied to Figure 1 In the electronic device 100 shown. Figure 1 A schematic structural diagram of the electronic device 100 is shown.

[0084] The electronic device 100 may include a processor 110, an internal memory 121, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, an earphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a display screen 194, etc. The sensor module 180 may include a pressure sensor, a touch sensor, etc.

[0085] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0086] In some embodiments, the processor 110 may be a system on chip (SOC), which may include a central processing unit (CPU) or other types of processors. In some embodiments, the processor 110 may be a PWM control chip.

[0087] The processor 110 may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the technical solution program. In addition, the processor 110 may have the function of operating one or more software programs, which may be stored in a storage medium.

[0088] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0089] In some embodiments, the memory of the electronic device 100 may be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), or any computer-readable medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer.

[0090] In some embodiments, the processor 110 and the memory may be combined into a single processing device or may be independent components. The processor 110 may be configured to execute program code stored in the memory. In specific implementations, the memory may also be integrated into the processor 110 or may be independent of the processor 110.

[0091] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0092] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0093] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0094] In one embodiment, the electronic device 100 can receive the system white / blacklist sent from the system cloud through the mobile communication module 150 and / or the wireless communication module 160 to obtain the system white / blacklist. If it is the first time to obtain the system white / blacklist, the obtained system white / blacklist can be stored in the internal memory 121. If it is not the first time to obtain the system white / blacklist, the system white / blacklist stored in the internal memory 121 can be updated in real time based on the obtained system white / blacklist. Similarly, the electronic device 100 can also obtain the application whitelist provided by the application developer of the installed application to achieve the first acquisition or real-time update of the application whitelist.

[0095] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0096] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0097] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0098] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0099] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0100] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0101] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0102] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0103] The electronic device 100 can implement audio functions such as screen reading, music playback, and recording through the audio module 170, speaker 170A, headphone jack 170D, and application processor. In one embodiment, the screen reading function can be implemented based on a reading application installed on the electronic device 100 that supports accessibility services.

[0104] The pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor can be provided on the display screen 194. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor. The electronic device 100 can also calculate the location of the touch based on the detection signal of the pressure sensor. In some embodiments, touch operations applied to the same touch location but with different touch operation intensities can correspond to different operation instructions.

[0105] A touch sensor, also known as a "touch-sensitive device," can be provided on the display screen 194. The touch sensor and the display screen 194 form a touch screen, also known as a "touch screen." The touch sensor is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor can also be provided on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0106] In one embodiment, when the electronic device 100 displays the user interface of an application through a touch screen, the user can use the functions of the application by performing touch actions such as clicking and sliding on the touch screen. In response to the user's operation, the application may have an interface state change, and the operating system of the electronic device 100 may generate a corresponding accessibility event to describe the interface state change. The electronic device 100 can distribute accessibility events to accessibility applications in the whitelist through the accessibility service provided by the operating system based on the whitelist, and not distribute accessibility events to accessibility applications outside the whitelist, so as to achieve the purpose of accessibility service management and control.

[0107] Buttons 190 include a power button and a volume button. They can be mechanical or touch-sensitive. Motor 191 can generate vibrations. Motor 191 can be used for incoming call vibrations or for touch vibration feedback. Indicator 192 can be an indicator light that can indicate charging status, battery level changes, messages, missed calls, notifications, and more.

[0108] In one embodiment, by executing accessibility service management, there may be a situation where accessibility events are not distributed to some accessibility applications, that is, these accessibility applications are prohibited from using accessibility services. In this way, corresponding prompt information can be output through vibration, indicator light, screen pop-up window, etc. through motor 191, indicator 192, display screen 194 to prompt the user.

[0109] In addition, the electronic device runs an operating system on top of the above components. operating system, operating system, Operating system, etc. Applications can be installed and run on the operating system.

[0110] Operating systems can provide users with accessibility services. These services can be used to enhance accessibility for applications through methods such as text-to-speech, haptic feedback, gesture navigation, trackballs, and directional navigation, making them more accessible. Accessibility services can run in the background and receive accessibility events from the system. These events indicate changes in the user interface state, such as focus changes, input content changes, and button clicks. Accessibility services can also retrieve the active window and search for content within it.

[0111] When the user interface of an application changes state, the system can generate an accessibility event describing the interface state change and notify the accessibility service. Accessibility applications can obtain accessibility events through the accessibility service and can feedback operation instructions through the accessibility service to provide accessible user interaction functions.

[0112] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0113] As described above, the operating system of an electronic device can provide users with accessibility services. Accessibility services can be used to provide accessible user interaction functions to users with disabilities or users who are temporarily unable to fully interact with the device. For example, a reading application that supports accessibility services can provide users with the function of reading screen content aloud.

[0114] App developers can implement their apps as accessible apps. After users install an accessible app on an electronic device, they can enable the app's access to accessibility services. After obtaining user authorization, the accessible app can typically run in the background and utilize accessibility services to provide accessible user interaction.

[0115] In one implementation, the user can enable the accessibility application to accessibility services in the settings application of the electronic device. In another implementation, the user can enable the accessibility application to accessibility services in the control center of the electronic device. Figure 2a and Figure 2b, taking the example of enabling the reading application's permission to use accessibility services, this article explains these two implementation methods.

[0116] refer to Figure 2a After the user installs the reading application 204 on the user's mobile phone, the user can click the icon of the setting application 201 to start the setting application 201. In response to the user operation, the mobile phone can display the setting application 201 page, which can display a button for the auxiliary function 202. If the user clicks the button for the auxiliary function 202, the mobile phone can display the page for the auxiliary function 202 in response to the user operation, which can display a button for accessibility (i.e., accessibility service) 203. If the user clicks the button for accessibility 203, the mobile phone can display the page for accessibility 203 in response to the user operation, which can display a button for the reading application 204.

[0117] After the user installs the reading application 204 , the accessibility service usage status of the reading application 204 is not used by default. The user needs to manually enable the reading application 204 to use the accessibility service. At this time, the button of the reading application 204 can be displayed as closed.

[0118] If the user clicks the button for the reading application 204, the phone may display a page 205 for the reading application 204 in response to the user's operation. This page 205 may display a toggle button 206 in the off state, allowing the user to enable or disable the reading application 204's access to accessibility services. The user can click the toggle button 206 to toggle its on / off state. Page 205 may also display a description 207 of the reading application 204 and a schematic diagram 208 of the phone's user interface after the user authorizes the reading application 204.

[0119] If the user clicks the switch button 206, the mobile phone can respond to the user operation by updating the page of the reading application 204 from page 205 to page 209. Page 209 can display the switch button 206 in the on state, as well as the interface diagram 208 and the floating window 210. The floating window 210 can include multiple controls for controlling the usage status of the reading application 204, such as controls for allowing the user to pause / start reading.

[0120] refer to Figure 2bThe control center page of the mobile phone may display a control switch 211 for the reading application 204. The control switch 211 is used to allow the user to turn on / off the reading application 204's permission to use accessibility services. If the user clicks the control switch 211, the mobile phone may respond to the user operation by displaying a page 212 of the reading application 204. The page 212 may display a switch button 213 in the off state. The switch button 213 is used to allow the user to turn on / off the reading application 204's permission to use accessibility services. The user can click the switch button 213 to toggle its on / off state.

[0121] Since both implementations are used to turn on / off the accessibility service permission of the reading application 204, the page 212 can be the same as the page 205, and the switch button 213 can be the switch button 206. In this way, if the user clicks the switch button 213, the mobile phone can update the page 212 of the reading application 204 in response to the user operation, and the display content of the updated page can be as follows: Figure 2a As shown on page 209.

[0122] After a user grants permission to an accessibility app, if accessibility service control is not implemented (i.e., the accessibility app's use of accessibility services is not controlled), the app can receive all accessibility events from the system. The following example illustrates how an accessibility app receives accessibility events.

[0123] Exemplarily, referring to FIG2 , application A, application B, and accessibility application C may be installed on the electronic device, and the user has enabled accessibility application C's permission to use accessibility services.

[0124] If an electronic device is running application A and the user performs an operation on application A's user interface, causing the interface state of application A to change, the system can generate a corresponding accessibility event 1 to reflect the interface state change and send accessibility event 1 to the accessibility service. If accessibility service management is not implemented, the accessibility service can send accessibility event 1 to accessibility application C, which can then obtain the interface state change information carried in accessibility event 1. After obtaining accessibility event 1, accessibility application C can either do nothing or provide accessible user interaction functions based on it.

[0125] For example, suppose accessibility application C has the function of automatically claiming an electronic red envelope. If application A experiences a change in interface state due to receiving the electronic red envelope, the system can generate a corresponding accessibility event and notify the accessibility service, which in turn sends the accessibility event to accessibility application C. Based on the received accessibility event, accessibility application C can issue an instruction to automatically claim the electronic red envelope through the accessibility service. In this way, the electronic device can automatically claim the electronic red envelope for the user without any user input.

[0126] Based on the same implementation principle, if the electronic device later switches to running application B, and the user performs an operation on the user interface of application B, causing the interface state of application B to change, the system can generate a corresponding accessibility event 2 to reflect the interface state change and send accessibility event 2 to the accessibility service. If accessibility service management is not performed, the accessibility service can send accessibility event 2 to accessibility application C, and accessibility application C can obtain the interface state change information carried in accessibility event 2. After obtaining accessibility event 2, accessibility application C can either do nothing or provide accessible user interaction functions based on it.

[0127] In some embodiments, the user interface of application A and the user interface of application B may both include a user information page (such as a password control page), and the user may enter user sensitive information such as account number and password in the user information page.

[0128] refer to Figure 4 , Figure 4 A password control page is shown, specifically a recharge page including a payment password control. Figure 4 As shown, an application installed on a user's mobile phone can provide a recharge service to the user by displaying a recharge page. While the user's mobile phone displays the recharge page, the user can enter a payment password in a payment pop-up window 401 on the recharge page to request payment. The user's mobile phone can display the payment password entered by the user in an encrypted form in the payment pop-up window 401.

[0129] If the user enters their payment password, the app usage action causes a state change on the top-up page. The system can generate a corresponding accessibility event to describe this interface state change and send the accessibility event to the accessibility service. If accessibility service control is not implemented, the accessibility service can distribute the accessibility event to each accessibility application on the user's phone, and the accessibility application can obtain information about the state change of the top-up page.

[0130] As can be seen above, if accessibility service controls are not implemented, even if users grant accessibility apps permissions, these apps can obtain user interface state change information through accessibility services. This poses a risk of user information leakage. For example, even if an accessibility app is actually malicious, it can obtain user interface state change information indiscriminately with user authorization.

[0131] To address the risk of user information leakage, see Figure 5 One embodiment of the present application provides a method for managing and controlling accessibility services, which is applied to an electronic device. The electronic device has a first application and a second application installed thereon. The second application supports accessibility services and has accessibility service usage rights. Based on this, the management and control method may include the following steps 501 and 502:

[0132] Step 501: When an interface state of a first application changes, an accessibility event for describing the interface state change is obtained.

[0133] Step 502: If the second application is in the first whitelist, the accessibility event is sent to the second application via the accessibility service.

[0134] It is feasible to place the identifiers of trusted accessibility applications in a whitelist. The whitelist can contain the identifiers of one or more accessibility applications, or it can contain no identifiers of any accessibility applications, thereby limiting the accessibility events that reflect changes in the user interface state to the accessibility applications in the whitelist. In this way, the accessibility service can send accessibility events to the accessibility applications in the whitelist as needed, while not sending them to accessibility applications outside the whitelist, thus achieving accessibility service management and control.

[0135] Figure 5 The control method shown can perform accessibility service control based on a whitelist, selectively allowing accessibility applications in the whitelist to use accessibility services, while prohibiting accessibility applications outside the whitelist from using accessibility services, and differentially sending accessibility events to trusted accessibility applications. This can prevent application developers of malicious applications from obtaining user interface status change information, which poses a risk of user information leakage.

[0136] Considering that accessibility events do not always involve user information pages, and users install accessibility applications in order to use them to provide users with accessible user interaction functions, on the basis of addressing the risk of user information leakage, in order to support accessible applications in providing users with accessible user interaction functions, the purpose of accessibility service control can be achieved through system general control.

[0137] Feasibly, the general system control can be: based on the system whitelist provided by the manufacturer of the electronic device, only perform accessibility service control on the user information page, while for other types of pages, there is no restriction on the use of accessibility services by accessibility applications outside the system whitelist.

[0138] Figure 5 In one embodiment of the control method, no distinction is made between individual applications, allowing universal system control to be applied to all applications. Thus, whenever any application experiences a state change, it is determined whether the currently displayed user interface is a user information page. If so, accessibility service control is applied based on the system whitelist. Otherwise, accessibility events can be sent to accessibility applications outside the system whitelist via the accessibility service.

[0139] Figure 5 In another embodiment of the control method shown, different types of applications can be distinguished. Specifically, applications can be divided into third-party open applications and general applications based on whether the application can request customized accessibility protection (i.e., request customized control). In this way, third-party open applications can be customized and controlled as needed, while general applications are not customized and are subject to system-wide control. By distinguishing between third-party open applications and general applications, the personalized control needs of application developers can be met.

[0140] A feasible approach is to customize accessibility control: if a third-party open application requests accessibility protection, accessibility control is performed based on an application whitelist, regardless of whether the user interface includes user information-related controls. The application whitelist can be provided by the application developer, or, if the application developer does not provide a whitelist, the system whitelist can be used as the application whitelist.

[0141] In one implementation method 1 of distinguishing different types of applications, only customized control may be performed on third-party open applications.

[0142] In another implementation method 2 for distinguishing different types of applications, not only can customized control be implemented for third-party open applications, but system-wide control can also be implemented during non-customized control periods. In other words, customized control and system-wide control can be combined to implement accessibility service control when electronic devices run third-party open applications.

[0143] By performing general system control on third-party open applications during the non-customized control period of third-party open applications, it is possible to ensure that there is no risk of user information leakage when users use third-party open applications during the non-customized control period, provided that the developers of the third-party open applications have no specific control requirements during the non-customized control period, and to minimize the impact on accessible applications in providing users with accessible user interaction functions.

[0144] Figure 5The specific technical implementation of the control method shown can be referred to the description in other embodiments of this application and will not be repeated here.

[0145] Based on the above content, taking the above distinction of different types of applications as an example, see Figure 6 An embodiment of the present application provides another method for controlling accessibility services. The method is applied to an electronic device and may include the following steps 601 to 608.

[0146] Step 601: When an interface state of a first application installed on an electronic device changes, an accessibility event for describing the interface state change is obtained.

[0147] Step 602 , determining whether the second application installed on the electronic device is in the system blacklist, and whether the second application supports accessibility services and has permission to use accessibility services; if so, not sending the accessibility event to the second application, otherwise executing step 603 .

[0148] In this step, accessibility events are not sent to accessibility applications in the system blacklist. For accessibility applications outside the blacklist, they are further managed based on the application whitelist / system whitelist and the specific operation status of the application.

[0149] Step 603 , determining whether the first application is an application that can request accessibility protection, if so, proceed to step 604 , otherwise proceed to step 608 .

[0150] This step is used to determine whether the currently running application is a third-party open application or a general application. If it is a third-party open application, step 604 is executed to further determine whether it is in the customized control period of the third-party open application. If it is a general application, step 608 is executed to perform general control on the general application based on the system whitelist.

[0151] Step 604 , determining whether the first application has requested accessibility protection when the interface state of the first application changes, if so, execute step 605 , otherwise execute step 608 .

[0152] This step is used to determine whether the system is currently in the customized control period of the third-party open application. If it is in the customized control period, step 605 is executed to perform customized control based on the application whitelist. If it is in the non-customized control period, step 608 is executed to perform general control based on the system whitelist during the non-customized control period of the third-party open application.

[0153] Step 605 , determining whether the first application provides a whitelist, if so, executing step 606 , otherwise executing step 607 .

[0154] This step is used to determine whether the currently running application is a third-party open application with a whitelist or a third-party open application without a whitelist. If it is the former, step 606 is executed to perform customized control based on the application whitelist provided by the third-party open application. If it is the latter, step 607 is executed to perform customized control using the system whitelist as the application whitelist provided by the third-party open application.

[0155] Step 606 , determining whether the second application is in the whitelist provided by the first application; if so, sending the accessibility event to the second application via the accessibility service; otherwise, not sending the accessibility event to the second application.

[0156] This step is used to perform customized management and control based on the application whitelist (i.e., the whitelist provided by the application) for third-party open applications that provide a whitelist, so as to limit the accessibility applications in the whitelist to use the accessibility service, while the accessibility applications outside the whitelist cannot use the accessibility service.

[0157] Step 607 : Determine whether the second application is in the system whitelist. If so, send the accessibility event to the second application through the accessibility service. Otherwise, do not send the accessibility event to the second application.

[0158] This step is used to perform customized management based on the application whitelist (replaced by the system whitelist) for third-party open applications that do not provide a whitelist, so as to limit the accessibility applications in the whitelist to use the accessibility service, while the accessibility applications outside the whitelist cannot use the accessibility service.

[0159] Step 608: Determine whether the user interface of the first application includes user information related controls when the interface state changes. If so, execute step 607; otherwise, send the accessibility event to the second application through the accessibility service.

[0160] This step is used to perform general control based on the system whitelist for general applications, or to perform general control based on the system whitelist during the non-customized control of third-party open applications, so as to limit the accessibility applications in the whitelist to obtain the status change information of the user information page, while the accessibility applications outside the whitelist cannot obtain the status change information of the user information page, but can obtain the status change information of non-user information pages.

[0161] Figure 6 The illustrated embodiment can flexibly adjust the control strategy for accessibility services to meet the personalized control needs of application developers and the system's general control needs to address the risk of user information leakage, while minimizing the impact on accessible applications providing users with accessible user interaction functions.

[0162] Figure 6The embodiment shown is only an example of the implementation of the technology of the present application, which may also include more or fewer steps. For example, if blacklist control is not involved, step 602 may not be performed, and step 603 may be performed directly after step 601.

[0163] The software system of electronic devices can adopt layered architecture, event-driven architecture, micro-core architecture, micro-service architecture or cloud architecture. Taking the layered architecture of Android system as an example, combined with Figure 7 The software structure block diagram of the electronic device shown takes the above-mentioned distinction of different types of applications as an example to illustrate the barrier-free service management process of the embodiment of the present application. In one embodiment, the electronic device can be the above-mentioned electronic device 100.

[0164] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers: from top to bottom, the application layer, the application framework layer (Framework), the system library and runtime layer, the hardware abstraction layer (HAL), and the kernel layer (Kernel).

[0165] The application layer can include a series of application packages, which can be divided into three categories: accessible applications such as reading applications, third-party open applications such as instant messaging applications and banking applications, and general applications such as cameras, galleries, calendars, calls, maps, etc.

[0166] After users install the accessible app, they can use Figure 2a or Figure 2b The method shown enables accessibility apps to use accessibility services. After obtaining user authorization, accessibility apps can run in the device background and, under controlled and permitted conditions, use accessibility services to obtain accessibility events that reflect changes in the user interface state. These apps can then provide accessible user interaction features without any action or by responding to accessibility events.

[0167] Electronic devices can provide accessibility protection interfaces to each third-party open application. When the third-party open application runs, it can call the accessibility protection interface to request accessibility protection, thereby triggering the electronic device to perform the above-mentioned customized control. However, if the accessibility protection interface is not provided to general applications, the electronic device can perform the above-mentioned system-wide control when the general application runs.

[0168] Feasibly, third-party open applications can call the accessibility protection interface as needed when they start running to request accessibility protection, or call the accessibility protection interface when displaying a specific page to trigger the electronic device to perform customized control during the interface call.

[0169] Different from the general system control that performs accessibility service control based on the system whitelist, customized control performs accessibility service control based on the application whitelist. Based on whether the application provides its own whitelist, third-party open applications can be divided into two categories: one is applications that provide a whitelist, so during the operation of such applications, customized control can be performed based on the whitelist provided by them; the other is applications that do not provide a whitelist, so during the operation of such applications, the system whitelist can be used as the whitelist provided by them for customized control. In an example, Figure 7 The instant messaging apps in provide an application whitelist, while the banking apps do not.

[0170] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 7 As shown, the application framework layer may include accessibility services, accessibility service management modules, view systems, notification managers, resource managers, window managers, content providers, and the like.

[0171] The accessibility service management module may include an application whitelist setting module, an accessibility protection execution module, a system notification prompt module, a general management module, and a system black / whitelist management module.

[0172] For any third-party open application that provides a whitelist, the application whitelist setting module can set the provided whitelist as the application whitelist of the third-party open application to support accessibility service management based on its application whitelist during the customized management of the third-party open application.

[0173] Electronic device developers can distribute system whitelists through the system cloud to support general system management. In addition to system whitelists, developers can also distribute system blacklists through the system cloud. Correspondingly, the system blacklist / whitelist management module manages the blacklists / whitelists distributed by the system cloud (either initially or through updates).

[0174] The general control module can identify whether the current page of the device is the user information page based on the window mechanism provided by the operating system. Among them, the window mechanism can be set with information such as the application window switching, the window after switching, and the controls included in the window after switching.

[0175] Feasibly, the accessibility protection execution module can trigger the accessibility service on demand to perform customized control or system general control based on whether the currently running application is a third-party open application or a general application, whether the third-party open application calls the accessibility protection interface, and whether the current user interface of the application is a user information page.

[0176] The accessibility service runs in the background, receiving accessibility events from the system and distributing them to accessibility apps. Based on the system blacklist, the accessibility service can prevent accessibility events from being sent to blacklisted apps, effectively prohibiting them from receiving any accessibility events. For accessibility apps not on the blacklist, the service can further combine customized controls with on-demand execution of general system controls to determine whether to send accessibility events to them.

[0177] When the accessibility protection execution module determines that customized control needs to be initiated based on the application whitelist, it can notify the accessibility service to initiate customized control, thereby triggering the accessibility service to send accessibility events to accessible applications that are on the application whitelist and not on the system blacklist, based on the system blacklist and application whitelist, and not to other accessible applications. Furthermore, when the accessibility protection execution module determines that customized control needs to be terminated, it can notify the accessibility service to terminate customized control.

[0178] When the accessibility protection execution module determines that general system control needs to be initiated based on the system whitelist, it can notify the accessibility service to initiate general system control, thereby triggering the accessibility service to send accessibility events involving changes in the interface state of the user information page to accessibility applications that are on the system whitelist and not on the system blacklist, rather than to other accessibility applications, based on the system blacklist and the system whitelist. Furthermore, when the accessibility protection execution module determines that general system control needs to be terminated, it can notify the accessibility service to terminate the general system control.

[0179] If an accessibility application is prohibited from using accessibility services due to accessibility service control, the system notification module can output a corresponding prompt message to inform the user of the control result. Based on the prompt message, the user can request to modify the system blacklist / whitelist as needed. The system blacklist / whitelist management module can respond to user requests and update the system blacklist / whitelist accordingly to support users' personalized control needs based on the system's general control.

[0180] Figure 7In the illustrated embodiment, the general control module within the accessibility service management module can detect whether the user interface is a user information page during the system's general control period (including during the operation of general applications and the non-customized control period of third-party open applications). In other embodiments, the accessibility service management module may not include the general control module. In this case, the accessibility service management module does not identify the user information page, and instead the accessibility service can identify the user information page based on a window mechanism.

[0181] The accessibility service can, based on the window mechanism, determine the window switching of the user interface to identify the user information page. If the user information page is identified during the system general control period and the non-customized control period, the accessibility service control based on the system whitelist can be automatically started. If a non-user information page is identified, the accessibility service control based on the system whitelist can be automatically stopped.

[0182] The view system includes visual controls, such as password controls, controls for displaying text, and controls for displaying images. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface containing a text notification icon can include a view for displaying text and a view for displaying an image.

[0183] The Notification Manager allows applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short period of time without user interaction. For example, the Notification Manager is used to notify that an accessibility application is prohibited from using accessibility services, to notify that a download is complete, and to provide message reminders. The Notification Manager can also be used to display notifications in the form of icons or scrolling text in the status bar at the top of the system, such as notifications from applications running in the background, or to display notifications on the screen in the form of dialog windows. For example, text messages can be displayed in the status bar, prompts can be sounded, the terminal device can vibrate, and indicator lights can flash.

[0184] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0185] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0186] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0187] The system library and runtime layer includes the system library and Android Runtime.

[0188] The system library may include multiple functional modules, such as a three-dimensional (3D) graphics processing library (e.g., OpenGL ES), a surface manager, a two-dimensional (2D) graphics engine (e.g., SGL), and media libraries.

[0189] The 3D graphics library implements 3D graphics drawing, image rendering, compositing, and layer processing. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The 2D graphics engine is the drawing engine for 2D drawing. The media library supports playback and recording of various common audio and video formats, as well as still image files. The media library supports a variety of audio and video codecs, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0190] The Android runtime consists of core libraries and a virtual machine (VM). The Android runtime is responsible for scheduling and management of the Android system. The core libraries consist of two parts: one containing the Java language's callable functions and the other the Android core library. The application layer and the application framework layer run in the VM. The VM executes the Java files in the application and framework layers as binary files. The VM performs functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0191] The Hardware Abstraction Layer (HAL) acts as a bridge between software and hardware, encapsulating the underlying hardware drivers and providing a common interface for application frameworks to call these drivers. The HAL abstracts the device kernel driver, providing the higher-level Java API framework with an application programming interface (API) for accessing the underlying device. The HAL consists of multiple library modules, each of which implements an interface for a specific type of hardware component.

[0192] The kernel layer is the layer between hardware and software. It includes at least the camera driver, display driver, Bluetooth driver, audio driver, and sensor driver. The display driver controls the display's image display; the sensor driver controls the operation of various sensors, such as pressure sensors and touch sensors.

[0193] Under the above multi-layer architecture, the electronic device is also provided with a hardware layer, which may include the aforementioned electronic device hardware components, such as display screens, sensors, indicators, etc.

[0194] It should be noted that although the embodiments of the present application are based on The system is used as an example, but the basic principles are also applicable to Electronic devices with other operating systems.

[0195] It is understandable that Figure 7 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer layers than shown, and each layer may include more or fewer components, and this application does not limit this.

[0196] For example, based on Figure 7 The software structure of the electronic device shown takes the above-mentioned distinction between different types of applications as an example to illustrate the implementation process of electronic devices performing accessibility service management and control.

[0197] Suppose an electronic device has accessibility apps A, B, C, and D installed, as well as an instant messaging app, a banking app, and a map app. The map app is a universal app, while the instant messaging app and the banking app are both third-party open apps. The instant messaging app provides an app whitelist, while the banking app does not.

[0198] Assume that accessibility application A is on the system blacklist, accessibility application B is on the application whitelist provided by the instant messaging application, accessibility application C is on the system whitelist, and accessibility application D is not on either list. The user has authorized each accessibility application to use accessibility services. This could result in the following control example.

[0199] 6.1 Implementing system-wide control over general applications

[0200] 6.1.1 Controlling Interface Status Changes on Non-User Information Pages of General Apps

[0201] Assuming that a non-user information page of a map application experiences an interface state change, the system can generate a corresponding accessibility event 01 and notify the accessibility service of accessibility event 01. If the general control module detects that the currently displayed user interface is not the user information page, the accessibility protection execution module can determine that control based on the system whitelist is not currently required. Therefore, the accessibility service can distribute accessibility event 01 to accessibility applications B, C, and D based on the system blacklist, but not to accessibility application A.

[0202] 6.1.2 Controlling UI Status Changes on User Information Pages of General Applications

[0203] Assuming that there is an interface state change on the user information page of the map application, the system can generate a corresponding accessibility event 02 and notify the accessibility service of accessibility event 02. When the general control module detects that the currently displayed user interface is the user information page, the accessibility protection execution module can determine that control based on the system whitelist is currently required, and thus can notify the accessibility service to start control based on the system whitelist. In response to the notification, the accessibility service can not distribute accessibility event 02 to accessibility application A based on the system blacklist, and distribute accessibility event 02 to accessibility application C based on the system whitelist, but not to accessibility application B and accessibility application D.

[0204] 6.2 Implementing Customized Control over Third-Party Open Applications

[0205] 6.2.1 Control when third-party open applications provide whitelists

[0206] Assuming that the instant messaging application calls the accessibility protection interface and there is a change in the interface state, the system can generate a corresponding accessibility event 03 and notify the accessibility service of accessibility event 03. The accessibility protection execution module can determine that control based on the application whitelist is currently required, and thus can notify the accessibility service to begin control based on the application whitelist. In response to the notification, the accessibility service can not distribute accessibility event 03 to accessibility application A based on the system blacklist, and distribute accessibility event 03 to accessibility application B based on the application whitelist provided by the instant messaging application, but not to accessibility application C and accessibility application D.

[0207] 6.2.2 Control when third-party open applications do not provide a whitelist

[0208] Assuming that the banking application calls the accessibility protection interface and there is a change in the interface state, the system can generate a corresponding accessibility event 04 and notify the accessibility service of accessibility event 04. The accessibility protection execution module can determine that control based on the application whitelist is currently required, and thus can notify the accessibility service to begin control based on the application whitelist. In response to the notification, the accessibility service can, based on the system blacklist, not distribute accessibility event 04 to accessibility application A. Instead, it can use the system whitelist as the application whitelist provided by the banking application and distribute accessibility event 04 to accessibility application C instead of to accessibility application B and accessibility application D.

[0209] 6.3 Implementing general system control for third-party open applications during non-customized control periods

[0210] 6.3.1 Controlling Interface Status Changes on Non-User Information Pages of Third-Party Open Applications

[0211] Assuming the instant messaging application (or banking application) does not call the accessibility protection interface and there is an interface state change on its non-user information page, the system can generate a corresponding accessibility event 05 and notify the accessibility service of accessibility event 05. If the general control module detects that the currently displayed user interface is not the user information page, the accessibility protection execution module can determine that there is no need for control based on the system whitelist. In this case, the accessibility service can distribute accessibility event 05 to accessibility applications B, C, and D based on the system blacklist, but not to accessibility application A.

[0212] 6.3.2 Controlling changes in the user information page of third-party open applications

[0213] Assuming that the instant messaging application (or banking application) does not call the accessibility protection interface, and there is an interface status change on its user information page, the system can generate a corresponding accessibility event 06 and notify the accessibility service of accessibility event 06. When the general control module detects that the currently displayed user interface is the user information page, the accessibility protection execution module can determine that it is currently necessary to perform control based on the system whitelist, so it can notify the accessibility service to start control based on the system whitelist. In response to the notification, the accessibility service can not distribute accessibility event 06 to accessibility application A based on the system blacklist, and distribute accessibility event 06 to accessibility application C based on the system whitelist, but not to accessibility application B and accessibility application D.

[0214] As can be seen from the above examples, the embodiments of the present application can prohibit untrusted applications from using accessibility services by implementing accessibility service control, thereby addressing the risk of user information leakage. In addition, based on the different types of applications, whether the application provides a whitelist, and by combining customized control with system-wide control, the embodiments of the present application can flexibly adjust the control strategy for accessibility services to meet the personalized control needs of application developers and the system-wide control needs to address the risk of user information leakage, while minimizing the impact on the accessible user interaction functions provided by accessible applications to users.

[0215] Based on the above content, other embodiments of the present application can also achieve the purpose of barrier-free service control in scenarios such as not distinguishing between different types of applications, not performing system-wide control on third-party open applications, and not setting up a system blacklist, so as to cope with the risk of user information leakage. This embodiment will not be elaborated here.

[0216] An embodiment of the present application provides a device for controlling accessibility services, which is applied to an electronic device. The electronic device is installed with a first application and a second application, and the second application supports accessibility services and has permission to use the accessibility services. The device includes: an acquisition module, which is used to obtain an accessibility event for describing the interface state change when there is an interface state change in the first application; and a processing module, which is used to send the accessibility event to the second application through the accessibility service if the second application is in a first whitelist.

[0217] An embodiment of the present application also provides a chip, which is installed in an electronic device. The chip includes: a processor, which is used to execute computer program instructions stored in a memory, wherein when the computer program instructions are executed by the processor, the chip is triggered to execute the method steps provided by any method embodiment of the present application.

[0218] An embodiment of the present application also proposes a terminal device, which includes a communication module, a memory for storing computer program instructions, and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the terminal device is triggered to execute the method steps provided by any method embodiment of the present application.

[0219] An embodiment of the present application also provides an electronic device, which includes multiple antennas, a memory for storing computer program instructions, a processor for executing the computer program instructions, and a communication device (such as a communication module that can implement 5G communication based on the NR protocol), wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps provided by any method embodiment of the present application.

[0220] Specifically, in an embodiment of the present application, one or more computer programs are stored in the above-mentioned memory, and the one or more computer programs include instructions. When the instructions are executed by the above-mentioned device, the above-mentioned device executes the method steps described in the embodiment of the present application.

[0221] Furthermore, the devices, apparatuses, and modules described in the embodiments of the present application may be implemented by computer chips or entities, or by products having certain functions.

[0222] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, apparatus, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0223] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, etc.) to execute all or part of the steps of the method described in each embodiment of this application.

[0224] Specifically, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, it enables the computer to execute the method steps provided in the embodiment of the present application.

[0225] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program product is run on a computer, it enables the computer to execute the method steps provided in the embodiment of the present application.

[0226] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.

[0227] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0228] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0229] The integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc., various media that can store program code.

[0230] In the embodiments of the present application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, commodity, or apparatus comprising the element.

[0231] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0232] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0233] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the same or similar parts between the various embodiments of this application can be referred to in detail. For example, the specific working processes of the systems, devices, and units described in the embodiments of this application can refer to the corresponding processes in the method embodiments of this application, and will not be repeated here.

[0234] The above description is merely a specific embodiment of the present application and is not intended to limit the present application. The scope of protection of the present application shall be subject to the claims.

Claims

1. A method for managing and controlling barrier-free services, characterized in that: Applied to an electronic device, the electronic device having a first application and a second application installed thereon, the second application supporting an accessibility service and having permission to use the accessibility service; The control method includes: When an interface state of the first application changes, obtaining an accessibility event for describing the interface state change; If the second application is in the first whitelist, the accessibility event is sent to the second application through the accessibility service.

2. The method according to claim 1, characterized in that The control method further includes: determining whether the user interface of the first application when the interface state changes includes a user information-related control; If the user interface includes a user information related control, the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist is performed.

3. The method according to claim 2, characterized in that The management and control method further includes: using the system whitelist as the first whitelist, executing the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

4. The method according to claim 2, characterized in that The control method further includes: If the first application is not an application that can request accessibility protection, the step of determining whether the user interface of the first application includes user information related controls when the interface state changes is performed.

5. The method according to claim 1, wherein The control method further includes: If the first application is an application capable of requesting accessibility protection, determining whether the first application has requested accessibility protection when the interface state of the first application changes; If the first application has requested accessibility protection, the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist is performed.

6. The method according to claim 5, characterized in that The control method further includes: If the first application does not request accessibility protection, determining whether the user interface of the first application when the interface state changes includes a user information-related control; If the user interface includes a user information related control, then when the system whitelist includes the second application, the accessibility event is sent to the second application through the accessibility service.

7. The method according to claim 5 or 6, characterized in that If the first application provides a whitelist, the whitelist provided by the first application is used as the first whitelist; and / or, If the first application does not provide a whitelist, the system whitelist is used as the first whitelist.

8. The method according to claim 5 or 6, characterized in that The time period during which the first application requests accessibility protection is a time period when the first application is running, or a time period when the first application displays a setting user interface.

9. The method according to any one of claims 4 to 6, characterized in that: If the electronic device provides an accessibility protection interface for the first application, the first application is an application capable of requesting accessibility protection; If the electronic device does not provide an accessibility protection interface for the first application, then the first application is not an application that can request accessibility protection; The accessibility protection interface is used by applications to call and request accessibility protection.

10. The method according to any one of claims 2, 4 and 6, characterized in that: The control method further includes: The step of determining whether the user interface of the first application includes a user information-related control when the interface state changes is performed by the accessibility service or a module for managing the accessibility service in the electronic device according to a set window mechanism; The window mechanism is used to describe the switching of the interface window of the first application and the controls in the interface window of the first application.

11. The method according to any one of claims 1 to 6, characterized in that: The control method further includes: If the second application is not in the system blacklist, perform the step of sending the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

12. A barrier-free service management and control device, characterized in that: Applied to an electronic device, the electronic device having a first application and a second application installed thereon, the second application supporting an accessibility service and having permission to use the accessibility service; The control device includes: an acquisition module, configured to obtain, when an interface state change occurs in the first application, an accessibility event describing the interface state change; A processing module is configured to send the accessibility event to the second application through the accessibility service if the second application is in the first whitelist.

13. A chip, characterized in that: include: A processor configured to execute computer program instructions stored in a memory, wherein when the computer program instructions are executed by the processor, the chip is triggered to execute the method according to any one of claims 1 to 11.

14. An electronic device, characterized in that: The electronic device comprises one or more memories for storing computer program instructions, and one or more processors, wherein when the computer program instructions are executed by the one or more processors, the electronic device is triggered to execute the method according to any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images