Screen reading method, system and terminal device of application control and medium

Abstract

The application discloses a screen reading method and system of application control, a terminal device and a computer readable storage medium. The screen reading method of the application control comprises the following steps: receiving a screen reading instruction, determining a current browsing application and an application control in the browsing application based on the screen reading instruction, acquiring a target application control parameter to be displayed in a voice mode in the application control, and reading the target application control parameter. The application can improve the screen reading efficiency of the application control, and further improve the user experience.

Description

Technical Field

[0001] This invention relates to the field of voice-assisted technology, and in particular to a screen reading method, system, terminal device, and computer-readable storage medium for application controls. Background Technology

[0002] When the screen reader function is enabled on the phone's system, it will read aloud the content on the app's page, including text, control types, and control states. For example, if there are buttons on the app's page after the screen reader function is enabled, it will read aloud "Login button".

[0003] However, because the app does not take into account the need for visually impaired users to use the screen reader function, when there are many operable buttons on the app page, visually impaired users will hear many readings ending with "button". This screen reader method is cumbersome for visually impaired users who are already familiar with the app, making the app's operation inefficient. Summary of the Invention

[0004] The main objective of this invention is to provide a screen reading method, system, terminal device, and computer-readable storage medium for application controls, aiming to improve the screen reading efficiency of application controls and thus enhance the user experience.

[0005] To achieve the above objectives, the present invention provides a screen reading method for application controls, wherein screen reading of the application controls includes:

[0006] Receive a screen reading instruction, and determine the current browsing application and the application controls in the browsing application based on the screen reading instruction;

[0007] Obtain the parameters of the target application control that will be displayed in a voice manner, and read the parameters of the target application control aloud.

[0008] Optionally, before the step of receiving the screen reading instruction, the method further includes:

[0009] Receive a request to modify the display method of application control parameters, and configure the application control parameters according to the request to perform screen reading operations based on the configured control parameters.

[0010] Optionally, the step of configuring the control parameters of the application control according to the display method modification request includes:

[0011] Based on the request to modify the display method, the application control specified by the user is determined, and the display method of the control parameters of the application control is determined to be either voice display or text display, so as to configure the control parameters.

[0012] Optionally, the step of reading aloud the parameters of the target application control includes:

[0013] Read aloud the target control parameters, including control state and control text.

[0014] Optionally, after the step of determining the current browsing application and the application controls in the browsing application based on the screen reading instruction, the method further includes:

[0015] Obtain the application control parameters displayed in text format from the application controls, and display the application control parameters on the page in text format. The application control parameters include: control type.

[0016] Optionally, the screen reading method for the application control further includes:

[0017] If no request to modify the display method is received, all control parameters of the application controls in the currently viewed application will be displayed in a voice manner.

[0018] Optionally, before the step of receiving the screen reading instruction, the method further includes:

[0019] Mark the focus of all application controls in each client application so that the control parameters of the marked application controls can be configured.

[0020] To achieve the above objectives, the present invention also provides a screen reader system for application controls, the screen reader system for application controls comprising:

[0021] The determination module is used to receive screen reading instructions and determine the current browsing application and the application controls in the browsing application based on the screen reading instructions.

[0022] The reading module is used to obtain the parameters of the target application control that will be displayed in a voice manner, and to read the parameters of the target application control aloud.

[0023] In this invention, each functional module of the screen reading system for application controls implements the steps of the screen reading method for application controls as described above during runtime.

[0024] To achieve the above objectives, the present invention also provides a terminal device, the terminal device comprising: a memory, a processor, and a screen reader program for application controls stored in the memory and executable on the processor, wherein the screen reader program for application controls, when executed by the processor, implements the steps of the screen reading method for application controls as described above.

[0025] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a screen reader program for an application control, wherein the screen reader program for the application control, when executed by a processor, implements the steps of the screen reading method for the application control as described above.

[0026] In addition, to achieve the above objectives, the present invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the screen reading method for the application control as described above.

[0027] This invention provides a screen reader, system, terminal device, computer-readable storage medium, and computer program product for application controls. By receiving a screen reading instruction, the system determines the current browsing application and the application controls within the browsing application based on the screen reading instruction; it obtains the parameters of the target application control that will be displayed in a voice manner, and reads the parameters of the target application control aloud.

[0028] Compared to existing screen reading methods in apps, this invention, after determining the currently viewed application and all application controls within that application based on the user-triggered screen reading command, obtains the parameters of the target application control displayed via voice and then reads these parameters aloud. Therefore, this invention can directly read aloud the parameters of application controls displayed via voice, avoiding the need to read aloud all control parameters. Furthermore, this invention reduces the complexity for users, especially visually impaired individuals, when operating the app, improves user efficiency, and ultimately enhances the user experience. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention;

[0030] Figure 2 This is a flowchart illustrating an embodiment of the screen reading method for the application control of the present invention;

[0031] Figure 3 This is a schematic diagram of the functional modules of a screen reader system using the control application of the present invention.

[0032] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0033] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0034] like Figure 1 As shown, Figure 1This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0035] It should be noted that the terminal device in the embodiments of the present invention can be a terminal device for implementing screen reading of multiple application controls, and the terminal device can specifically be a smartphone, a personal computer, and a server, etc.

[0036] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0037] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0038] like Figure 1 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a screen reader program for application controls. The operating system is a program that manages and controls the device's hardware and software resources, supporting the screen reader program for application controls and the operation of other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the screen reader program of the application control stored in the memory 1005 and perform the following operations:

[0039] Receive a screen reading instruction, and determine the current browsing application and the application controls in the browsing application based on the screen reading instruction;

[0040] Obtain the parameters of the target application control that will be displayed in a voice manner, and read the parameters of the target application control aloud.

[0041] Furthermore, prior to the step of receiving the screen reading instruction, the processor 1001 can also invoke the screen reading program of the application control stored in the memory 1005, and perform the following operations:

[0042] Receive a request to modify the display method of application control parameters, and configure the application control parameters according to the request to perform screen reading operations based on the configured control parameters.

[0043] Furthermore, the processor 1001 can also be used to call the screen reader program of the application controls stored in the memory 1005, and also perform the following operations:

[0044] Based on the request to modify the display method, the application control specified by the user is determined, and the display method of the control parameters of the application control is determined to be either voice display or text display, so as to configure the control parameters.

[0045] Furthermore, the processor 1001 can also be used to call the screen reader program of the application controls stored in the memory 1005, and also perform the following operations:

[0046] Read aloud the target control parameters, including control state and control text.

[0047] Furthermore, after the step of determining the current browsing application and the application controls in the browsing application based on the screen reading instruction, the processor 1001 can also be used to call the screen reading program of the application controls stored in the memory 1005, and further perform the following operations:

[0048] Obtain the application control parameters displayed in text format from the application controls, and display the application control parameters on the page in text format. The application control parameters include: control type.

[0049] Furthermore, the processor 1001 can also be used to call the screen reader program of the application controls stored in the memory 1005, and also perform the following operations:

[0050] If no request to modify the display method is received, all control parameters of the application controls in the currently viewed application will be displayed in a voice manner.

[0051] Furthermore, prior to the step of receiving the screen reading instruction, the processor 1001 can also invoke the screen reading program of the application control stored in the memory 1005, and perform the following operations:

[0052] Mark the focus of all application controls in each client application so that the control parameters of the marked application controls can be configured.

[0053] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the screen reading method for the application control of the present invention.

[0054] In this embodiment, an implementation of an interface switching method is provided. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0055] In this embodiment, basic screen reader functionality can assist visually impaired users in operating mobile apps smoothly. However, existing apps do not consider the browsing and operation efficiency issues for visually impaired users. For example, sighted users can quickly browse the entire app page, but visually impaired users need to listen to each focus point one by one until they obtain the desired page focus. Therefore, existing app screen reader functions greatly reduce the efficiency of app operation for visually impaired users, complicating the process and significantly reducing the user experience. Based on this, this invention marks all controls within the app and allows users to set whether to read out control types, such as whether to read out "buttons," enabling visually impaired users to browse and use the app more efficiently.

[0056] Step S10: Receive a screen reading instruction, and determine the current browsing application and the application controls in the browsing application based on the screen reading instruction;

[0057] After receiving a screen reading command triggered by the user, the terminal device will determine the APP that the user will access based on the screen reading command, and further determine all the controls in the APP, so as to display all the controls by voice according to the user's needs, that is, read them aloud, so that the visually impaired person can perform corresponding operations on each control.

[0058] It should be noted that in this embodiment, the screen reading command can be triggered when the user starts the APP, so as to reduce the complexity of the APP operation for visually impaired people and thus improve the user experience.

[0059] Step S20: Obtain the parameters of the target application control that will be displayed in a voice manner, and read the parameters of the target application control aloud.

[0060] After obtaining the APP currently being operated by the user and all application controls in the APP based on the screen reading command triggered by the user, the terminal device will further determine the target control parameters that need to be displayed in a voice manner from all application controls, and then read the target control parameters aloud.

[0061] It should be noted that in this embodiment, to simplify the operation of the APP for visually impaired users and improve operational efficiency, the control parameters of the controls within the APP will not be read aloud. Users may pre-determine the control parameters that need to be read aloud, and the terminal device can read the control parameters aloud according to the user's settings, thereby improving user operational efficiency. In this embodiment, control parameters include, but are not limited to, control type, control state, and control text.

[0062] Furthermore, in step S20 above, "reading aloud the parameters of the target application control" may include:

[0063] Step S201: Read aloud the target control parameters, including control state and control text.

[0064] After obtaining the parameters of the target application control that will be displayed in a voice manner, the terminal device reads the parameters aloud, enabling visually impaired users to perform corresponding operations on the APP based on the application control parameters read aloud by the terminal device.

[0065] It should be noted that in this embodiment, the application control parameters include control type, control state, and control text. The control type can include fixed buttons, sliding buttons, etc. Normally, when a terminal device performs screen reading, it will read aloud the "Login button." If there are many operable buttons on the app page, this will greatly reduce user efficiency. Therefore, in this embodiment, only "Login" can be read aloud, allowing users familiar with the app to perform quick operations. In addition to the control type, other control parameters can also be determined according to user needs to improve the flexibility of visually impaired users when using the app.

[0066] In this embodiment, after receiving a screen reading command triggered by the user, the terminal device determines the app the user will access based on the command, and further identifies all controls within that app. It then displays all controls aloud according to the user's needs, enabling the visually impaired user to perform corresponding operations on each control. Furthermore, it identifies the target control parameters that need to be displayed aloud from all application controls and reads these parameters aloud.

[0067] Compared to existing screen reading methods in apps, this invention, after determining the currently viewed application and all application controls within that application based on the user-triggered screen reading command, obtains the parameters of the target application control displayed via voice and then reads these parameters aloud. Therefore, this invention can directly read aloud the parameters of application controls displayed via voice, avoiding the need to read aloud all control parameters. Furthermore, this invention reduces the complexity for users, especially visually impaired individuals, when operating the app, improves user efficiency, and ultimately enhances the user experience.

[0068] Based on the first embodiment of the screen reading method for application controls of the present invention described above, a second embodiment of the screen reading method for application controls of the present invention is proposed.

[0069] In this embodiment, before step S10, "receiving screen reading instructions", the method further includes:

[0070] Step S30: Receive a request to modify the display method of the application control parameters, and configure the application control parameters according to the request to perform screen reading operations based on the configured control parameters.

[0071] Due to differences in user experience and familiarity with the app, this embodiment allows users to configure the parameters of application controls to suit their operating habits and improve efficiency. Upon receiving a user-triggered request to modify the display method, the terminal device will configure the user-specified application control parameters and then display the application controls according to the configured settings.

[0072] Further, step S30 above, "configuring the control parameters of the application control according to the display method modification request," may include:

[0073] Step S301: Determine the application control specified by the user according to the display mode modification request, and determine the display mode of the control parameters of the application control as voice display or text display, so as to configure the control parameters.

[0074] The terminal device receives a display mode modification request triggered by the user, determines the application control specified by the user based on the request, and modifies the display mode of the corresponding control parameters in the application control to either voice or text display, thus completing the configuration of the control parameters. In this embodiment, the user can modify all control parameters of the application control to suit their operating habits, thereby greatly improving the convenience and flexibility of the APP operation for visually impaired users.

[0075] Furthermore, after step S10 above, "determining the current browsing application and the application controls in the browsing application based on the screen reading instruction," the following may also be included:

[0076] Step S40: Obtain the application control parameters displayed in text format from the application controls, and display the application control parameters on the page in text format. The application control parameters include: control type.

[0077] In order to improve the efficiency of terminal devices in reading aloud the control parameters of the application controls in the current application, in this embodiment, the control parameters can be divided into: control parameters displayed in voice mode and control parameters displayed in text mode.

[0078] Specifically, for example, after the terminal device receives the screen reading command triggered by the user and determines the application the user is currently browsing and the controls in the application based on the screen reading command, it can obtain not only the application control parameters that will be displayed in voice mode, but also the application control parameters that will be displayed in text mode. This eliminates the need for the terminal device to read out all the control parameters, thereby saving the operating efficiency of users familiar with the APP and simplifying the user operation process.

[0079] It should be noted that, in this embodiment, the control parameters include, but are not limited to, control type, control state, and control text. Whether the above control parameters need to be displayed by the terminal device through voice, i.e., whether the user needs to read them aloud, can be set by the user, which improves the user's operational flexibility.

[0080] Furthermore, the screen reading method for the application control may also include:

[0081] Step S50: If no request to modify the display method is received, display all control parameters of the application controls in the currently browsing application in a voice manner.

[0082] If the terminal device does not receive a user-triggered request to modify the display method of application control parameters within a preset time, it will default to displaying all control parameters of the application controls via voice. However, users can change the display method of control parameters while using the app to improve the operational flexibility for visually impaired users.

[0083] Furthermore, prior to step S10, "receiving screen reading instructions," the following steps are also included:

[0084] Step S60: Mark the focus of all application controls of each application on the client to configure the control parameters of the marked application controls.

[0085] In order to enable users to configure the control parameters of application controls in various applications, the terminal device needs to pre-mark the focus of all controls in the APP, so that after receiving the user's screen reading command, the terminal device can quickly determine the application that the user wants to browse and all controls in the application.

[0086] In this embodiment, the terminal device receives a display mode modification request triggered by the user, determines the application control specified by the user based on the request, and modifies the display mode of the corresponding control parameters in the application control to either voice or text display, thus completing the configuration of the control parameters. After receiving a screen reading command triggered by the user and determining the application currently being viewed and the controls within that application based on the command, the terminal device can obtain not only the application control parameters to be displayed via voice but also the application control parameters to be displayed via text, eliminating the need for the terminal device to read all control parameters aloud. If the terminal device does not receive a user-triggered request to modify the display mode of application control parameters within a preset time, it defaults to displaying all control parameters of the application controls via voice. To enable users to configure the control parameters of application controls in various applications, the terminal device needs to pre-mark the focus of all controls in the APP, allowing the terminal device to quickly determine the application the user is currently viewing and all controls within that application after receiving the user's screen reading command.

[0087] This invention's screen reading method for application controls is particularly suitable for visually impaired users operating apps. Users can customize the display of control parameters—either via voice or text—based on their varying levels of app proficiency. Upon receiving a screen reading command from the user, the terminal device reads aloud the voice-displayed application control parameters, avoiding the need to read all control parameters aloud and thus reducing the operational burden on visually impaired users. In this way, this invention reduces the complexity of app operation for visually impaired users, improves user efficiency, and ultimately enhances the user experience.

[0088] Furthermore, embodiments of the present invention also propose a screen reader system for application controls, referring to... Figure 3 , Figure 3 This is a schematic diagram of the functional modules of a screen reader application control according to an embodiment of the present invention. Figure 3 As shown, the screen reader system for application controls of the present invention includes:

[0089] The determination module 10 is used to receive a screen reading instruction and determine the current browsing application and the application controls in the browsing application based on the screen reading instruction;

[0090] The reading module 20 is used to obtain the parameters of the target application control that will be displayed in a voice manner in the application control, and to read the parameters of the target application control aloud.

[0091] Furthermore, the screen reader system for the application control also includes:

[0092] The configuration module is used to receive a request to modify the display method of application control parameters, and configure the application control parameters according to the request to perform screen reading operations based on the configured control parameters.

[0093] Furthermore, the configuration module includes:

[0094] The display mode determination unit is used to determine the application control specified by the user according to the display mode modification request, and to determine the display mode of the control parameters of the application control as voice display or text display, so as to configure the control parameters.

[0095] Furthermore, the reading module 20 includes:

[0096] The reading unit is used to read aloud the target control parameters, including the control state and control text.

[0097] Furthermore, the screen reader system for the application control also includes:

[0098] The page display module is used to obtain the application control parameters displayed in text format from the application controls, and to display the application control parameters in text format on the page. The application control parameters include: control type.

[0099] Furthermore, the screen reading method for the application control also includes:

[0100] The voice display module is used to display all control parameters of the applied controls in the currently browsed application in a voice manner when no request to modify the display method is received.

[0101] Furthermore, the screen reading method for the application control also includes:

[0102] The tagging module is used to tag the focus of all application controls in each client application, so that the control parameters of the tagged application controls can be configured.

[0103] The specific implementation methods of each functional module of the screen reading system of the application control of the present invention are basically the same as those of the above-described embodiments of the screen reading method of the application control, and will not be repeated here.

[0104] Furthermore, this embodiment of the invention also proposes a computer-readable storage medium storing a screen reader program for an application control. When the screen reader program for the application control is executed by a processor, it implements the steps of the screen reading method for the application control as described above.

[0105] The various embodiments of the screen reading system and computer-readable storage medium of the application control of the present invention can be referred to the various embodiments of the screen reading method of the application control of the present invention, and will not be repeated here.

[0106] Furthermore, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of the screen reading method for application controls as described in any of the embodiments of the above-described screen reading method for application controls.

[0107] The specific embodiments of the computer program product of the present invention are basically the same as the embodiments of the screen reading method of the application control described above, and will not be repeated here.

[0108] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0109] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0111] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A screen reading method for application controls, characterized in that, The screen reading methods for the application controls include: Receive a screen reading instruction, and determine the current browsing application and the application controls in the browsing application based on the screen reading instruction; Obtain the parameters of the target application control that will be displayed in a voice manner, and read the parameters of the target application control aloud. Before the step of receiving the screen reading command, the method further includes: Receive a request to modify the display method of application control parameters, determine the application control specified by the user according to the request, and determine the display method of the control parameters of the application control as voice display or text display, so as to configure the control parameters, and perform screen reading operation based on the configured application control parameters; If no request to modify the display method is received within the preset time period, all control parameters of the application controls in the currently viewed application will be displayed in a voice manner.

2. The screen reading method for application controls as described in claim 1, characterized in that, The step of reading aloud the parameters of the target application control includes: Read aloud the target control parameters, including control state and control text.

3. The screen reading method for application controls as described in claim 1, characterized in that, After the step of determining the current browsing application and the application controls in the browsing application based on the screen reading instruction, the method further includes: Determine the control parameters that are displayed as text in the application controls, and display the control parameters as text on the page, wherein the control parameters include: control type.

4. The screen reading method for application controls as described in claim 1, characterized in that, Before the step of receiving the screen reading command, the method further includes: Mark the focus of all application controls in each client application so that the control parameters of the marked application controls can be configured.

5. A screen reader system for application controls, characterized in that, The screen reader system for the application controls includes: The determination module is used to receive screen reading instructions and determine the current browsing application and the application controls in the browsing application based on the screen reading instructions. The reading module is used to obtain the parameters of the target application control that will be displayed in a voice manner, and to read the parameters of the target application control aloud. Before the step of receiving the screen reading command, the method further includes: The configuration module is used to receive a request to modify the display method of application control parameters, determine the application control specified by the user according to the request, and determine the display method of the control parameters of the application control as voice display or text display, so as to configure the control parameters and perform screen reading operation based on the configured application control parameters. The voice display module is used to display all control parameters of the application controls in the currently browsed application in a voice manner when no request to modify the display method is received within a preset time period.

6. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a screen reader program for application controls stored in the memory and executable on the processor. When the screen reader program for application controls is executed by the processor, it implements the steps of the screen reading method for application controls as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a screen reader program for an application control, which, when executed by a processor, implements the steps of the screen reading method for the application control as described in any one of claims 1 to 4.

Images