Using large language models to generate view-based accessibility information
By applying a large language model to accessibility information, the system generates a summary of the graphical user interface content, enhancing accessibility for users with disabilities by providing a quicker and more intuitive experience.
WO2025071985A1 Publication Date: 2025-04-03GOOGLE LLC
1 Cites 1 Cited by
Patent Information
- Application Number
- PCT/US2024/047050
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-09-29
- Filing Date
- 2024-09-17
- Publication Date
- 2025-04-03
AI Technical Summary
Technical Problem
Existing accessibility services require users with disabilities to interact with each individual element on a screen to receive accessibility information, which is time-consuming and difficult, especially with unknown or unintuitive screen layouts.
Method used
A computing system applies a large language model to accessibility information associated with an application, generating a summary of the content included in a graphical user interface, which can be presented to the user via text-to-speech synthesis.
Benefits of technology
This approach provides users with a quicker and more user-friendly experience by summarizing the content of a graphical user interface, reducing the need for users to navigate through each element individually.
✦ Generated by Eureka AI based on patent content.
Abstract
A computing system retrieves, using an application programming interface, information associated with a plurality of user interface elements included in a graphical user interface associated with an application executing at a computing device. The information associated with the plurality of user interface elements may be generated by the application programming interface. The computing system converts the information into structured text. The computing system generates, using a machine learning model, a summary of content included in the graphical user interface by at least applying the machine learning model to the structured text, wherein the machine learning model is a large language model. The computing system then sends, to the computing device, the summary of the content included in the graphical user interface, wherein the summary of the content includes a description including general view structure and semantic information associated with the graphical user interface.
Need to check novelty before this filing date? Find Prior Art