A bank barrier-free service interaction method, device, equipment and medium

Abstract

This application discloses a method, apparatus, device, and medium for accessible banking services, relating to the field of computer technology. It includes: collecting multi-dimensional capability information of the current user and generating a user profile corresponding to the current user based on this information; receiving operation commands input by the current user using a preset banking application based on a preset input method, and performing a fusion decision based on the operation commands and the user profile to determine the target interaction channel; dynamically reconstructing the layout and components of the current interface based on the user profile and semantic analysis results to generate a personalized interface adapted to the current user; generating corresponding interactive feedback based on the operation commands and the personalized interface, and outputting the interactive feedback to the current user through the target interaction channel. This enables accessible banking services that cover multiple disability types, integrate multi-modal interaction methods, dynamically adapt to user capabilities, and ensure operational security and privacy.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for accessible banking services. Background Technology

[0002] With the advancement of digital transformation in the banking sector, online channels such as mobile banking are gradually becoming the mainstream service method. However, existing accessibility services in banks still mainly remain at a superficial level, such as "enlarging fonts" and "voice broadcasting," failing to truly meet the complex needs of visually impaired, hearing-impaired, and elderly users. For example, visually impaired users often encounter problems such as disorganized interface reading order and missed key information when using screen readers; hearing-impaired users cannot use voice customer service and lack visual interactive feedback; and elderly users are prone to getting lost or making mistakes in key steps due to complex operation processes and high information density on the interface. Currently, mainstream bank apps mainly adopt three modes for accessibility services: an elderly-friendly "senior mode" providing a large-font interface, basic accessibility adaptation compatible with some screen reader software, and human customer service as a backup. These solutions still have significant limitations in actual use and cannot achieve a targeted, dynamic, safe, and reliable accessibility interactive experience.

[0003] Existing technological modifications often focus on visual interface adjustments, lacking specialized services for the hearing impaired, resulting in this group being almost entirely ignored in practical use. Interaction methods are limited, primarily relying on touchscreen clicks and failing to integrate gestures, voice, Braille input, and other methods. There is a lack of step-by-step guidance and error-tolerant mechanisms for complex business processes, making it easy for users with weaker cognitive abilities to get lost. A balance between safety and convenience is difficult to achieve; independent operation by disabled users carries high risks, while seeking assistance from others can easily lead to the leakage of sensitive information. Accessibility features are configured fixedly, unable to dynamically adapt to changes in user abilities or environment. Furthermore, the lack of data tracking and evaluation mechanisms for the actual effectiveness of accessibility features hinders continuous service optimization.

[0004] As can be seen from the above, how to achieve accessible banking services that cover multiple types of barriers, integrate multimodal interaction methods, dynamically adapt to user capabilities, and ensure operational security and privacy is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for accessible service interaction in banking. The specific solution is as follows: Firstly, this application provides a method for interactive accessibility services in a bank, including: Collect multidimensional ability information of the current user, and generate a user profile corresponding to the current user based on the multidimensional ability information; the multidimensional ability information includes visual dimension information, hearing dimension information, cognitive dimension information, and motor dimension information; The system uses a pre-defined banking application to receive operation commands input by the current user based on a pre-defined input method, and performs a fusion decision based on the operation commands and the user profile to determine the target interaction channel. The pre-defined input method includes one or more combinations of voice input, gesture input, touch input, eye-tracking input, and peripheral device input. The target interaction channel includes a visually impaired channel, a hearing impaired channel, and a cognitively impaired channel. Based on the user profile and the semantic analysis results of the current operation interface, the layout and components of the current operation interface are dynamically reconstructed to generate a personalized interface adapted to the current user. Based on the operation instructions and the personalized interface, corresponding interactive feedback is generated and output to the current user through the target interaction channel.

[0006] Optionally, the step of collecting the current user's multidimensional capability information and generating a user profile corresponding to the current user based on the multidimensional capability information includes: The system collects multidimensional ability information of the current user based on a structured questionnaire, and uses a preset device compensation tool to generate a corresponding multidimensional ability score based on the multidimensional ability information. A user profile corresponding to the current user is generated based on the multidimensional ability score, and the user profile is dynamically updated based on the analysis results of the current user's operation behavior, anomaly detection results, and periodic feedback results.

[0007] Optionally, the step of receiving the operation command input by the current user based on a preset input method using a preset bank application, and making a fusion decision based on the operation command and the user profile to determine the target interaction channel includes: Utilize a pre-set bank application to receive operation instructions input by the current user based on a pre-set input method; For the operation commands input by the user via voice, a preset end-to-end ASR recognition and NLU semantic understanding model is used to process the operation commands input via voice, so as to output standardized intent and slot parameters; For the operation command input by the current user through gesture, a decision tree classifier is used to identify the operation command input by gesture based on predefined standard gestures to determine the gesture type, and the recognition sensitivity can be adjusted based on custom gestures and the current user's motion dimension information. For the current user's touch input operation commands, execute blind typing mode, large target touch area and fault-tolerant touch optimization to confirm the touch position and corresponding interface elements; For the current user's eye-tracking input commands, perform 9-point calibration and fixation time detection to determine the fixation area; For the operation commands input by the user through the peripheral device, the corresponding command code is output by receiving the routing key signal of the Braille display or the scanning selection signal of the switch controller; Based on the priority routing table in the user profile corresponding to the current user, conflict arbitration is performed on the standardized intent and slot parameters, gesture type, touch position and corresponding interface elements, gaze area and instruction code output within the same time window to obtain the corresponding conflict arbitration result, and collaborative instructions between different preset input methods are identified to obtain the corresponding collaborative instruction identification result. Based on the conflict arbitration result and the instruction recognition result, the target interaction channel in the current scenario is determined through fusion decision.

[0008] Optionally, the step of dynamically reconstructing the layout and components of the current user interface based on the user profile and the semantic analysis results of the current user interface to generate a personalized interface adapted to the current user includes: The current user interface is semantically parsed to generate a component tree with key labels; the key labels are pre-assigned based on the importance of the interface elements. Based on the component tree and the target interaction channel corresponding to the current user, perform the corresponding layout reorganization operation on the current operation interface, and adjust the element density level of the current operation interface based on the user profile of the current user. The component replacement operation is performed on the preset component types. The component replacement operation includes replacing multi-field forms with step-by-step filling wizards, replacing drop-down options with card grids, replacing graphic verification codes with audio verification or question-and-answer verification, and replacing standard numeric keypads with large-key keyboards that include quick amount options.

[0009] Optionally, outputting the interactive feedback to the current user through the target interaction channel includes: If the target interaction channel is a visually impaired channel, then the screen reader enhanced reading, the Braille display synchronous refresh, and tactile feedback signals are output to the current user in parallel; the screen reader enhanced reading supports custom pronunciation of financial terms and progressive speech speed, and the Braille display synchronous refresh only updates the changed areas and converts the amount into computer Braille; If the target interaction channel is a hearing-impaired channel, then real-time subtitles, visual prompts, sign language digital human translation driven by a 3D CNN model, and visual vibration mapping are output to the current user in parallel; the real-time subtitles support font size adjustment, speaker differentiation, and keyword highlighting, and the sign language digital human is used for risk disclosure and product description and supports interruption at any time; If the target interaction channel is a cognitive impairment channel, a simplified voice command is output to the current user, the complex process is broken down into atomic steps by the task decomposition engine, and a context-aware prompt pops up when the user stays for too long or when a repetitive operation occurs.

[0010] Optionally, the bank's accessible service interaction method further includes: If an assistance request is received from the current user, a one-time assistance code is generated and presented to the current user via TTS reading and large font display. The one-time assistance code is input into the assistance device to establish a P2P connection encrypted with AES-256, so that a synchronized interface corresponding to the current user operation interface is rendered on the assistance device, and sensitive information areas in the synchronized interface are covered, including the password box, balance and complete account. The operation permissions of the assistance device on the synchronization interface are restricted. The assistance device can only provide guidance through voice or text to complete the current assistance process, and is prohibited from responding to any click or input operation. If the current assistance process ends, the one-time assistance code is set to an invalid state, and the operation log is encrypted and stored. If the assisting device is detected to be inducing risky operations during the current assistance process, an alarm will be automatically triggered.

[0011] Optionally, the bank's accessible service interaction method further includes: The system performs risk assessments on the current user's risky actions and triggers corresponding intervention measures. Provides real-time validation, plain language error prompts, and history rollback during the input process; When a user experiences an error, a help menu will automatically pop up and support contacting emergency contacts. Automatically adjust interface parameters based on environmental conditions; Collect user interaction behavior data and calculate a health score based on the interaction behavior data, so as to dynamically update the user profile and system configuration corresponding to the current user according to the health score.

[0012] Secondly, this application provides a bank's accessible service interaction device, comprising: The information collection module is used to collect multi-dimensional ability information of the current user based on a preset bank application, and generate a user profile corresponding to the current user based on the multi-dimensional ability information; the multi-dimensional ability information includes visual dimension information, hearing dimension information, cognitive dimension information, and motor dimension information; The instruction acquisition module is used to receive operation instructions input by the current user based on a preset input method using a preset banking application, and to perform a fusion decision based on the operation instructions and the user profile to determine the target interaction channel; the preset input method includes one or more combinations of voice input, gesture input, touch input, eye-tracking input, and peripheral device input; the target interaction channel includes a visually impaired channel, a hearing impaired channel, and a cognitively impaired channel; The interface generation module is used to dynamically reconstruct the layout and components of the current operation interface based on the user profile and the semantic analysis results of the current operation interface, so as to generate a personalized interface adapted to the current user. An interactive feedback module is used to generate corresponding interactive feedback based on the operation command and the personalized interface, and output the interactive feedback to the current user through the target interaction channel.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned accessible service interaction method for banks.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned bank accessibility service interaction method.

[0015] This application provides a method for interactive accessibility services in a bank. It collects multi-dimensional capability information of the current user and generates a user profile corresponding to the current user based on this information. The multi-dimensional capability information includes visual, hearing, cognitive, and motor dimensions. A preset bank application is used to receive operation commands input by the current user based on preset input methods. A fusion decision is made based on the operation commands and the user profile to determine the target interaction channel. The preset input methods include one or more combinations of voice input, gesture input, touch input, eye-tracking input, and peripheral device input. The target interaction channel includes a visual impairment channel, a hearing impairment channel, and a cognitive impairment channel. Based on the user profile and the semantic analysis results of the current operation interface, the layout and components of the current operation interface are dynamically reconstructed to generate a personalized interface adapted to the current user. Based on the operation commands and the personalized interface, corresponding interactive feedback is generated and output to the current user through the target interaction channel.

[0016] As can be seen from the above, this application collects multi-dimensional ability information of users in terms of vision, hearing, cognition, and movement, and generates personalized user profiles. By combining multiple input methods such as voice, gestures, touch, eye tracking, and peripheral devices, it can intelligently determine the most suitable target interaction channel (visual impairment channel, hearing impairment channel, or cognitive impairment channel) for the current user. Based on semantic analysis, it dynamically reconstructs the user interface to generate a personalized interface adapted to the user's abilities. Finally, it outputs targeted interactive feedback through the selected channel, achieving full coverage support for disabled and elderly users. This breaks the limitations of traditional single-interaction modes and significantly improves the convenience, security, and accessibility of users with different abilities independently operating banking services. Thus, it enables accessible banking services that cover multiple types of disabilities, integrate multimodal interaction methods, dynamically adapt to user abilities, and ensure operational security and privacy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart of a bank's accessible service interaction method disclosed in this invention; Figure 2 This is a diagram of an accessible service interaction system architecture for banks disclosed in this invention. Figure 3 This is a schematic diagram of a bank's accessible service interaction device disclosed in this invention; Figure 4 This is a structural diagram of an electronic device disclosed in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] With the advancement of digital transformation in the banking sector, online channels such as mobile banking are gradually becoming the mainstream service method. However, existing accessibility services in banks still mainly remain at a superficial level, such as "enlarging fonts" and "voice broadcasting," failing to truly meet the complex needs of visually impaired, hearing-impaired, and elderly users. For example, visually impaired users often encounter problems such as disorganized interface reading order and missed key information when using screen readers; hearing-impaired users cannot use voice customer service and lack visual interactive feedback; and elderly users are prone to getting lost or making mistakes in key steps due to complex operation processes and high information density on the interface. Currently, mainstream bank apps mainly adopt three modes for accessibility services: an elderly-friendly "senior mode" providing a large-font interface, basic accessibility adaptation compatible with some screen reader software, and human customer service as a backup. These solutions still have significant limitations in actual use and cannot achieve a targeted, dynamic, safe, and reliable accessibility interactive experience. Existing technological modifications often focus on visual interface adjustments, lacking specialized services for the hearing impaired, resulting in this group being almost entirely ignored in practical use. Interaction methods are limited, primarily relying on touchscreen clicks and failing to integrate gestures, voice, Braille input, and other methods. There is a lack of step-by-step guidance and error-tolerant mechanisms for complex business processes, making it easy for users with weaker cognitive abilities to get lost. A balance between safety and convenience is difficult to achieve; independent operation by disabled users carries high risks, while seeking assistance from others can easily lead to the leakage of sensitive information. Accessibility features are configured fixedly, unable to dynamically adapt to changes in user abilities or environment. Furthermore, the lack of data tracking and evaluation mechanisms for the actual effectiveness of accessibility features hinders continuous service optimization. Therefore, this application provides a bank accessibility service interaction solution that can achieve bank accessibility services covering multiple disability types, integrating multimodal interaction methods, dynamically adapting to user capabilities, and ensuring operational security and privacy.

[0021] See Figure 1 As shown in the figure, this application discloses a method for interactive accessibility services in banks, including: Step S11: Collect the multi-dimensional capability information of the current user, and generate a user profile corresponding to the current user based on the multi-dimensional capability information.

[0022] In this embodiment, user information is collected through structured questionnaires and device testing. The multidimensional ability information collected includes, but is not limited to, visual acuity, hearing, cognition, and motor skills. Specifically, visual acuity information can be obtained by collecting visual acuity levels (total blindness / light perception / low light / color weakness, etc.), assistive tools (screen readers / Braille displays), and speech rate preferences. Hearing acuity information can be obtained by collecting hearing loss levels, communication methods (sign language / text / lip reading), and hearing aids. Cognitive acuity information can be assessed through optional memory card matching, reaction speed tests, and terminology comprehension tests. Motor skills information can be obtained by assessing fine and gross motor skills through touchscreen accuracy tests.

[0023] Furthermore, a comprehensive score is calculated for the current user based on their multidimensional ability information. Specifically, the process of collecting the current user's multidimensional ability information and generating a user profile corresponding to the current user based on this information can include: collecting the current user's multidimensional ability information based on a structured questionnaire, and generating a corresponding multidimensional ability score based on the information using a preset device compensation tool; generating a user profile corresponding to the current user based on the multidimensional ability score, and dynamically updating the user profile based on operational behavior analysis results, anomaly detection results, and periodic feedback results related to the current user. For example, in one specific implementation, a visual ability score can be calculated based on visual acuity level plus tool compensation; an auditory ability score can be calculated based on the degree of loss plus device compensation; a cognitive ability score can be weighted at 40% for memory, 30% for reaction, and 30% for comprehension; a motor ability score can be calculated based on the comprehensive evaluation results of fine and gross motor abilities, and finally a comprehensive profile including four-dimensional scores, recommendation channels, preference settings, device capabilities, emergency contacts, and safety parameters can be generated.

[0024] Furthermore, in some specific implementations, to maintain the timeliness of user profiles, the system dynamically updates them through three mechanisms. First, it continuously monitors user operation time, error rate, and number of repeated operations through operational behavior analysis, triggering profile reassessment when anomalies occur. Second, it pushes a lightweight feedback survey to users quarterly to update their preference settings. Third, the system defines five abnormal operation modes, including rapid combos, random swiping, and prolonged freezes; once these thresholds are exceeded, it proactively asks users if they need to adjust profile parameters.

[0025] Step S12: Receive the operation instructions input by the current user based on the preset input method using the preset bank application, and make a fusion decision based on the operation instructions and the user profile to determine the target interaction channel.

[0026] In this embodiment, the preset input methods include one or more combinations of voice input, gesture input, touch input, eye-tracking input, and peripheral device input; the target interaction channel includes, but is not limited to, visual impairment channel, hearing impairment channel, and cognitive impairment channel. Upon receiving the user's input command, processing is performed according to the different input methods. Specifically, receiving the user's input command based on the preset input method using a preset banking application, and performing a fusion decision based on the input command and the user profile to determine the target interaction channel, may include: receiving the user's input command based on the preset input method using a preset banking application; for voice input, using a finely tuned end-to-end ASR recognition model and NLU semantic understanding model in the financial field, outputting standardized intents (such as checking balance, transferring funds, paying bills, etc., more than twenty predefined intents) and corresponding slot parameters, and supporting referential resolution and omission completion. For gesture input, using a decision tree classifier to recognize eight standard gestures, determining the gesture type (such as single-finger tap for confirmation, two-finger tap for cancellation, left and right swipes for page turning, etc.), while also supporting user-recorded custom gestures, and adaptively adjusting the recognition sensitivity based on the user's motion dimension score. For touch input, the system implements blind typing mode, large target touch area, and error-tolerant touch optimization to accurately determine the touch position and its corresponding interface element. For eye-tracking input, a coordinate mapping matrix is ​​established through 9-point calibration, and gaze duration is detected to determine the area the user is looking at. For peripheral input, the system receives routing key signals from the Braille display or scan selection signals from the switch controller and outputs the corresponding instruction codes.

[0027] Furthermore, after obtaining the processing results of various inputs, the system uses a priority routing table in the user profile (e.g., the priority for a totally blind user can be set to voice 0.4, Braille 0.3, gesture 0.2, and touch 0.1) to arbitrate conflicts among multiple results within the same time window, obtaining the conflict arbitration result, and recognizing collaborative instructions between different input methods (e.g., saying "transfer money" while pointing to a contact). Finally, based on the conflict arbitration result and the collaborative instruction recognition result, the system determines the target interaction channel in the current scenario through fusion decision-making.

[0028] Step S13: Based on the user profile and the semantic analysis results of the current operation interface, dynamically reconstruct the layout and components of the current operation interface to generate a personalized interface adapted to the current user.

[0029] In this embodiment, by performing semantic parsing, layout reorganization, density adaptive adjustment, and component type replacement on the user interface, a personalized interface matching the user's capabilities is dynamically generated, thereby improving the autonomous operation capabilities of users with different disability types. Specifically, the dynamic reconstruction of the layout and components of the current user interface based on the user profile and the semantic analysis results of the current user interface to generate a personalized interface adapted to the current user may include: performing semantic parsing on the current user interface to generate a component tree with key labels; the key labels are pre-assigned based on the importance of interface elements; performing corresponding layout reorganization operations on the current user interface based on the component tree and the target interaction channel corresponding to the current user, and adjusting the element density level of the current user interface based on the user profile of the current user; and performing component replacement operations on preset component types, including replacing multi-field forms with step-by-step filling wizards, replacing drop-down options with card grids, replacing graphic verification codes with audio verification or question-and-answer verification, and replacing standard numeric keypads with large-key keyboards that include quick amount options.

[0030] For example, in one specific implementation, the system first performs semantic parsing on the current user interface to generate a component tree with key score annotations. Key score annotations are pre-assigned based on the importance of interface elements; for example, monetary numbers have a key score of 10, while decorative advertisements only have 1 to 2 points. Then, the system executes the corresponding layout reorganization rules according to the user's target interaction channel. If the target channel is for the visually impaired, the system removes all decorative advertisement elements, flattens the interface into a single-column flow layout, arranges elements in descending order of key score, fixes the operation buttons at the bottom, and collapses secondary items when there are more than 8 elements. If the target channel is for the hearing impaired, the system converts all voice prompts into visual text cards or animations, maintains a real-time subtitle area at the top of the interface, and uses triple feedback—color change, icon animation, and vibration—for operation confirmation. If the target channel is for the cognitively impaired, the system adopts an extremely simplified "one-screen-one-step" layout, displaying only 1 to 3 elements per screen, always showing a segmented progress indicator at the top, and retaining only two large navigation buttons at the bottom. Meanwhile, the system adjusts the interface density level based on the user's cognitive and behavioral scores, including four levels: ultra-sparse (1 to 2 elements), sparse (3 to 5 elements), standard (6 to 8 elements), and dense (close to the original interface). A background monitoring thread checks the user's dwell time on the page every 5 seconds; if it's too long, the system automatically downgrades the interface and displays a message saying "The page has been simplified for you"; if the user continues to perform correct actions, the system gradually upgrades the interface. In addition, the system performs replacement operations on preset typical component types, including replacing multi-field forms with step-by-step fill-in wizards, replacing dropdown options with more than 5 options with card grids, replacing image CAPTCHAs with audio or question-and-answer verification, and replacing standard numeric keypads with large-key keyboards that include quick amount options.

[0031] Step S14: Generate corresponding interactive feedback based on the operation command and the personalized interface, and output the interactive feedback to the current user through the target interaction channel.

[0032] In this embodiment, different multimodal output methods are adopted according to the user's different interaction channels (visual impairment, hearing impairment or cognitive impairment), including enhanced voice, Braille, subtitles, sign language digital human, tactile and visual cues, to ensure the perceptibility and comprehensibility of the feedback information. Specifically, the step of outputting the interactive feedback to the current user through the target interaction channel may include: if the target interaction channel is a visually impaired channel, then the screen reader's enhanced reading, Braille display's synchronous refresh, and tactile feedback signals are output to the current user in parallel; the screen reader's enhanced reading supports customized financial terminology pronunciation and progressive speech speed, and the Braille display's synchronous refresh only updates the changed areas and converts the amounts into computer Braille; if the target interaction channel is a hearing impaired channel, then real-time subtitles, visual prompts, a 3D CNN model-driven sign language digital human translation, and visual vibration mapping are output to the current user in parallel; the real-time subtitles support font size adjustment, speaker differentiation, and keyword highlighting, and the sign language digital human is used for risk disclosure and product explanation and supports interruption at any time; if the target interaction channel is a cognitively impaired channel, then simplified voice commands are output to the current user, complex processes are broken down into atomic steps through a task decomposition engine, and context-aware prompts pop up when the user stays for too long or repeats operations.

[0033] For example, in some specific implementations, the visually impaired channel (blind mode) input primarily uses voice and Braille, supplemented by gestures and keyboard. Output is handled in three parallel layers: screen reader enhancement (customized pronunciation of banking terminology, progressive reading, three-level speech speed mode, adaptive speech speed); synchronized Braille dot display (only refreshing changed areas, amounts displayed in computer Braille); tactile feedback (short vibration for success / long vibration for error / double short vibration at the end / gradually increasing load). A special "Exploration Mode" feature allows swiping without triggering any operation. The hearing impaired channel (deaf mode) input primarily uses touchscreen, supplemented by text and sign language recognition (a 3D CNN model supports 500 words). Output has four subsystems: real-time subtitles (adjustable font size, speaker differentiation, keyword highlighting); visual prompts (green checkmark = success, red exclamation mark = warning, red dot = new message, circle = countdown); digital sign language avatar (a core innovation, a 3D virtual avatar translating CSL sign language for risk disclosure and product explanation, supporting interruption at any time); and visual vibration mapping (unique call rhythm, timed reminders, continuous vibration for emergency alarms). The "Quiet Counter" feature enables end-to-end sign language translation for customer service conversations. The cognitive impairment channel (elderly mode) includes minimalist touch controls (80×80dp minimum button, ±15px tolerance, and anti-misoperation protection) plus simplified voice commands (20 commonly used instructions). The intelligent guidance system uses a task decomposition engine to break down complex processes into atomic steps (e.g., interbank transfers are broken down from 5 steps to 7 or more), and displays gentle bubble prompts when the user stays for more than 10 seconds, starting with a simple explanation and gradually increasing the detail. Simultaneously, the system remembers the user's frequent operating patterns (e.g., monthly transfers on fixed dates) and proactively reminds them.

[0034] Furthermore, this embodiment also includes assistance request and security protection functions. Specifically, if an assistance request initiated by the current user is received, a one-time assistance code is generated and presented to the current user via TTS reading and large font display. The one-time assistance code is input into the assistance device, establishing a P2P connection encrypted with AES-256, so that a synchronized interface corresponding to the current user's operation interface is rendered on the assistance device, and sensitive information areas in the synchronized interface, including password boxes, balance, and complete account information, are covered. The operation permissions of the assistance device on the synchronized interface are restricted; the assistance device can only provide guidance through voice or text to complete the current assistance process, and is prohibited from responding to any click or input operations. If the current assistance process ends, the one-time assistance code is set to an invalid state, and the operation log is encrypted and stored. If the assistance device is detected to induce risky operations during the current assistance process, an alarm is automatically triggered. That is, when a user initiates an assistance request, the system generates a 6-digit one-time assistance code (valid for 5 minutes) and informs the user via TTS reading and large font display. The user provides the assistance code to the assisting party. After the assisting party enters it on their own device, the system establishes an AES-256 encrypted P2P connection. An interface synchronized with the user's is rendered on the assisting party's device, but sensitive areas such as the password field, balance, and full account information are obscured. The assisting party can only provide guidance via voice or text and cannot perform any clicks or input operations. The assistance code becomes invalid immediately after the session ends, and the operation log is stored in encrypted form. If the system detects that the assisting party is inducing high-risk operations, it automatically triggers an alarm.

[0035] Furthermore, this application also includes intelligent guidance, fault tolerance, and effect evaluation mechanisms. Specifically, the system performs graded risk assessments for high-risk operations, triggering pop-up explanations, security verification, emergency contact confirmation, or operation freezing based on the assessment level, and suggesting a visit to a branch. During input, the system provides real-time verification, plain language error prompts (avoiding the use of technical terms such as "Luhn verification failed"), one-click repair of common errors, and a history rollback function for the last five operations. When the system detects that a user stays on the same page for more than a preset time or frequently switches tabs, it proactively pops up a help menu, providing options for homepage navigation, direct access to frequently used functions, or contacting emergency contacts. Simultaneously, the system automatically adjusts screen brightness based on ambient light intensity, adjusts volume and font size based on ambient noise levels, and enlarges the touch target and simplifies the interface when user movement is detected. Finally, the system collects user interaction behavior data (including channel usage distribution, task completion rate, operation time, error rate, heatmap, etc.), calculates a health score (covering four dimensions: accessibility, usability, satisfaction, and compliance), and dynamically updates user profiles and system configurations based on the score.

[0036] As can be seen from the above, this application's embodiments collect multi-dimensional ability information of users in the dimensions of vision, hearing, cognition, and movement, and generate dynamic user profiles. By combining multiple input methods such as voice, gestures, touch, eye tracking, and peripheral devices for fusion decision-making and conflict arbitration, it intelligently determines exclusive target interaction channels for visually impaired, hearing impaired, or cognitively impaired individuals. Based on semantic analysis, it reorganizes the layout of the operation interface, adapts its density, and replaces components to dynamically generate personalized interfaces. Then, through the output of screen readers from each channel, it enhances multimodal interactive feedback such as reading aloud, Braille synchronization, real-time subtitles, sign language digital humans, tactile feedback, visual prompts, and simplified voice. At the same time, it introduces privacy-preserving remote guidance with one-time assistance codes and encrypted P2P connections, graded risk intervention, instant verification and error repair, environmental adaptive adjustment, and full-process behavioral data collection and health scoring. This forms a closed-loop barrier-free service system that covers multiple types of impairments, integrates multimodal interaction methods, dynamically adapts to user capabilities, and balances safety and convenience. It significantly improves the accessibility, ease of use, and security of visually impaired, hearing impaired, and cognitively impaired individuals operating banking services independently.

[0037] See Figure 2 As shown, this application further provides an overall architecture for a bank's accessible service interaction system, which adopts a four-layer hierarchical architecture design.

[0038] In this implementation, the first layer is the user terminal layer, deployed within the mobile banking app client, providing five interaction mode entry points: visually impaired, hearing impaired, cognitively impaired, standard, and assisted. Users can actively choose the entry mode based on their disability type, or the system can automatically recommend one based on their user profile.

[0039] The second layer is the core engine layer, which includes a multimodal input fusion module, a dynamic interface reconstruction engine, and an output adapter. The multimodal input fusion module receives and integrates various input methods such as voice, gestures, touch, eye tracking, and peripheral input. The dynamic interface reconstruction engine generates a personalized interface in real time based on the user's ability score. The output adapter converts the processing results into channel-specific output formats, including TTS voice, subtitles, and sign language video. Above these three engines is an intelligent decision-making hub, which includes a channel router, a cognitive state perception module, and a security risk control engine, used to select the optimal interaction channel, monitor changes in user ability, and perform risk assessment and intervention, respectively. Below the core engine layer are three auxiliary modules: an intelligent guidance and fault tolerance module, a security proxy and assistance module, and an effect evaluation and data analysis module.

[0040] The third layer is the foundational support layer, providing underlying capabilities. This layer includes a user profile library for storing users' multidimensional capability scores and preference settings; a capability assessment library for storing assessment models and historical assessment data; a business logic layer for encapsulating various bank business operations; and the bank's core system APIs for connecting to the bank's back-end services.

[0041] The system's data flow is as follows: the user initiates an operation at the terminal layer, the operation command is transmitted to the core engine layer, and after intelligent decision-making, the business capabilities of the basic support layer are invoked. Finally, the processing result is returned to the terminal layer for presentation through the output adapter.

[0042] As can be seen from the above, the embodiments of this application, through the four-layer architecture and the collaborative work of each functional module, can achieve full coverage support for visually impaired, hearing impaired and cognitively impaired people, and complete a series of barrier-free service functions such as multimodal input fusion, dynamic interface adaptation, safe remote assistance, intelligent guidance and fault tolerance, and effect evaluation feedback.

[0043] See Figure 3 As shown in the figure, this application provides a bank accessibility service interaction device, including: The information collection module 11 is used to collect multi-dimensional ability information of the current user based on a preset bank application, and generate a user profile corresponding to the current user based on the multi-dimensional ability information; the multi-dimensional ability information includes visual dimension information, hearing dimension information, cognitive dimension information and motor dimension information; The instruction acquisition module 12 is used to receive operation instructions input by the current user based on a preset input method using a preset bank application, and to perform a fusion decision based on the operation instructions and the user profile to determine the target interaction channel; the preset input method includes one or more combinations of voice input, gesture input, touch input, eye-tracking input and peripheral device input; the target interaction channel includes a visually impaired channel, a hearing impaired channel and a cognitively impaired channel; The interface generation module 13 is used to dynamically reconstruct the layout and components of the current operation interface based on the user profile and the semantic analysis results of the current operation interface, so as to generate a personalized interface adapted to the current user. The interactive feedback module 14 is used to generate corresponding interactive feedback based on the operation command and the personalized interface, and output the interactive feedback to the current user through the target interaction channel.

[0044] In some specific embodiments, the information acquisition module 11 may specifically include: The ability scoring unit is used to collect multidimensional ability information of the current user based on a structured questionnaire, and to generate a corresponding multidimensional ability score based on the multidimensional ability information using a preset device compensation tool. The profile generation unit is used to generate a user profile corresponding to the current user based on the multi-dimensional ability score, and to dynamically update the user profile based on the operation behavior analysis results, anomaly detection results and periodic feedback results related to the current user.

[0045] In some specific embodiments, the instruction acquisition module 12 may specifically include: The instruction acquisition unit is used to receive operation instructions input by the current user based on a preset input method using a preset bank application. The first instruction recognition unit is used to process the operation instruction input by the current user via voice using a preset end-to-end ASR recognition and NLU semantic understanding model, so as to output standardized intent and slot parameters. The second instruction recognition unit is used to recognize the operation instructions input by the current user through gestures using a decision tree classifier based on predefined standard gestures, so as to determine the gesture type, and supports adjusting the recognition sensitivity based on custom gestures and the current user's motion dimension information. The third instruction recognition unit is used to perform blind typing mode, large target touch area and fault-tolerant touch optimization for the operation instructions input by the current user through touch, so as to confirm the touch position and the corresponding interface element; The fourth instruction recognition unit is used to perform 9-point calibration and fixation time detection for the operation instructions currently input by the user through eye movement, so as to determine the fixation area; The fifth instruction recognition unit is used to receive the routing key signal from the Braille display or the scanning selection signal from the switch controller to output the corresponding instruction code for the operation instruction input by the current user through the peripheral device. The recognition result generation unit is used to perform conflict arbitration on the standardized intent and slot parameters, gesture type, touch position and corresponding interface elements, gaze area and instruction code output within the same time window based on the priority routing table in the user profile corresponding to the current user, so as to obtain the corresponding conflict arbitration result, and to identify the collaborative instructions between different preset input methods so as to obtain the corresponding collaborative instruction recognition result. The channel determination unit is used to determine the target interaction channel in the current scenario based on the conflict arbitration result and the instruction recognition result through fusion decision.

[0046] In some specific embodiments, the interface generation module 13 may specifically include: The speech parsing unit is used to perform semantic parsing on the current operation interface to generate a component tree with key labels; the key labels are pre-assigned based on the importance of the interface elements. The interface layout reorganization unit is used to perform a corresponding layout reorganization operation on the current operation interface based on the component tree and the target interaction channel corresponding to the current user, and to adjust the element density level of the current operation interface based on the user profile of the current user. The component replacement unit is used to perform component replacement operations on preset component types. The component replacement operations include replacing multi-field forms with step-by-step filling wizards, replacing drop-down options with card grids, replacing graphic verification codes with audio verification or question-and-answer verification, and replacing standard numeric keypads with large-key keyboards that include quick amount options.

[0047] In some specific embodiments, the interactive feedback module 14 may specifically include: The first interactive feedback unit is used to output screen reader enhanced reading, Braille display synchronous refresh and tactile feedback signals to the current user in parallel if the target interactive channel is a visually impaired channel; the screen reader enhanced reading supports custom pronunciation of financial terms and progressive speech speed, and the Braille display synchronous refresh only updates the changed area and converts the amount into computer Braille. The second interactive feedback unit is used to output real-time subtitles, visual prompts, 3D CNN model-driven sign language digital human translation, and visual vibration mapping to the current user in parallel if the target interactive channel is a hearing-impaired channel. The real-time subtitles support font size adjustment, speaker differentiation, and keyword highlighting. The sign language digital human is used for risk disclosure and product description and supports interruption at any time. The third interactive feedback unit is used to output simplified voice commands to the current user if the target interactive channel is a cognitive impairment channel, break down complex processes into atomic steps through a task decomposition engine, and pop up context-aware prompts when the user stays for too long or repeats operations.

[0048] In some specific embodiments, the bank's accessible service interaction device may further include: The assistance request unit is used to generate a one-time assistance code if it receives an assistance request initiated by the current user, and to present the one-time assistance code to the current user through TTS reading and large font display. The interface synchronization unit is used to input the one-time assistance code into the assistance device, establish a P2P connection encrypted with AES-256, so as to render a synchronized interface corresponding to the current user operation interface on the assistance device, and cover sensitive information areas in the synchronized interface, including password box, balance and complete account. The permission restriction unit is used to restrict the operation permissions of the assistance device on the synchronization interface. The assistance device can only provide guidance through voice or text to complete the current assistance process and is prohibited from responding to any click or input operation. The log management unit is used to set the one-time assistance code to an invalid state and encrypt and store the operation log if the current assistance process ends. The risk alarm unit is used to automatically trigger an alarm if the assistance device is detected to induce risky operations during the current assistance process; The risk assessment unit is used to classify and assess the risky actions of the current user and trigger corresponding intervention measures. The error rollback unit provides real-time validation, plain language error messages, and historical rollback functionality during the input process. An exception handling unit is used to proactively pop up a help menu and support contacting emergency contacts when the user's operation is abnormal; An environment adaptive unit is used to automatically adjust interface parameters according to environmental conditions. The feedback unit is used to collect user interaction behavior data and calculate a health score based on the interaction behavior data, so as to dynamically update the user profile and system configuration corresponding to the current user according to the health score.

[0049] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the bank accessibility service interaction method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0050] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0051] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0052] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the bank accessibility service interaction method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0053] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for interactive accessibility services in banking. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0054] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0055] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0056] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0057] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for bank accessibility service interaction, characterized in that, include: Collect multidimensional capability information of the current user, and generate a user profile corresponding to the current user based on the multidimensional capability information; The multidimensional capability information includes visual ability information, hearing ability information, cognitive ability information, and motor ability information; The system uses a pre-set banking application to receive operation instructions input by the current user based on a pre-set input method, and performs a fusion decision based on the operation instructions and the user profile to determine the target interaction channel. The preset input methods include one or more combinations of voice input, gesture input, touch input, eye-tracking input, and peripheral input. The target interaction channels include visually impaired channels, hearing impaired channels, and cognitive impairment channels; Based on the user profile and the semantic analysis results of the current operation interface, the layout and components of the current operation interface are dynamically reconstructed to generate a personalized interface adapted to the current user. Based on the operation instructions and the personalized interface, corresponding interactive feedback is generated and output to the current user through the target interaction channel.

2. The bank accessible service interaction method of claim 1, wherein, The step of collecting the current user's multidimensional capability information and generating a user profile corresponding to the current user based on the multidimensional capability information includes: The system collects multidimensional ability information of the current user based on a structured questionnaire, and uses a preset device compensation tool to generate a corresponding multidimensional ability score based on the multidimensional ability information. A user profile corresponding to the current user is generated based on the multidimensional ability score, and the user profile is dynamically updated based on the analysis results of the current user's operation behavior, anomaly detection results, and periodic feedback results.

3. The bank accessible service interaction method of claim 1, wherein, The step of receiving operation instructions input by the current user based on a preset input method using a preset bank application, and making a fusion decision based on the operation instructions and the user profile to determine the target interaction channel includes: Utilize a pre-set bank application to receive operation instructions input by the current user based on a pre-set input method; For the operation commands input by the user via voice, a preset end-to-end ASR recognition and NLU semantic understanding model is used to process the operation commands input via voice, so as to output standardized intent and slot parameters; For the operation command input by the current user through gesture, a decision tree classifier is used to identify the operation command input by gesture based on predefined standard gestures to determine the gesture type, and the recognition sensitivity can be adjusted based on custom gestures and the current user's motion dimension information. For the current user's touch input operation commands, execute blind typing mode, large target touch area and fault-tolerant touch optimization to confirm the touch position and corresponding interface elements; For the current user's eye-tracking input commands, perform 9-point calibration and fixation time detection to determine the fixation area; For the operation commands input by the user through the peripheral device, the corresponding command code is output by receiving the routing key signal of the Braille display or the scanning selection signal of the switch controller; Based on the priority routing table in the user profile corresponding to the current user, conflict arbitration is performed on the standardized intent and slot parameters, gesture type, touch position and corresponding interface elements, gaze area and instruction code output within the same time window to obtain the corresponding conflict arbitration result, and collaborative instructions between different preset input methods are identified to obtain the corresponding collaborative instruction identification result. Based on the conflict arbitration result and the instruction recognition result, the target interaction channel in the current scenario is determined through fusion decision.

4. The bank accessible service interaction method of claim 1, wherein, The dynamic reconstruction of the layout and components of the current user interface based on the user profile and the semantic analysis results of the current user interface to generate a personalized interface adapted to the current user includes: The current user interface is semantically parsed to generate a component tree with key labels; the key labels are pre-assigned based on the importance of the interface elements. Based on the component tree and the target interaction channel corresponding to the current user, perform the corresponding layout reorganization operation on the current operation interface, and adjust the element density level of the current operation interface based on the user profile of the current user. The component replacement operation is performed on the preset component types. The component replacement operation includes replacing multi-field forms with step-by-step filling wizards, replacing drop-down options with card grids, replacing graphic verification codes with audio verification or question-and-answer verification, and replacing standard numeric keypads with large-key keyboards that include quick amount options.

5. The bank accessible service interaction method of claim 1, wherein, The step of outputting the interactive feedback to the current user through the target interaction channel includes: If the target interaction channel is a visually impaired channel, then the screen reader enhanced reading, the Braille display synchronous refresh, and tactile feedback signals are output to the current user in parallel; the screen reader enhanced reading supports custom pronunciation of financial terms and progressive speech speed, and the Braille display synchronous refresh only updates the changed areas and converts the amount into computer Braille; If the target interaction channel is a hearing-impaired channel, then real-time subtitles, visual prompts, sign language digital human translation driven by a 3D CNN model, and visual vibration mapping are output to the current user in parallel; the real-time subtitles support font size adjustment, speaker differentiation, and keyword highlighting, and the sign language digital human is used for risk disclosure and product description and supports interruption at any time; If the target interaction channel is a cognitive impairment channel, a simplified voice command is output to the current user, the complex process is broken down into atomic steps by the task decomposition engine, and a context-aware prompt pops up when the user stays for too long or when a repetitive operation occurs.

6. The bank accessible service interaction method of claim 1, wherein, Also includes: If an assistance request is received from the current user, a one-time assistance code is generated and presented to the current user via TTS reading and large font display. The one-time assistance code is input into the assistance device to establish a P2P connection encrypted with AES-256, so that a synchronized interface corresponding to the current user operation interface is rendered on the assistance device, and sensitive information areas in the synchronized interface are covered, including the password box, balance and complete account. The operation permissions of the assistance device on the synchronization interface are restricted. The assistance device can only provide guidance through voice or text to complete the current assistance process, and is prohibited from responding to any click or input operation. If the current assistance process ends, the one-time assistance code is set to an invalid state, and the operation log is encrypted and stored. If the assisting device is detected to be inducing risky operations during the current assistance process, an alarm will be automatically triggered.

7. The bank accessible service interaction method of claim 1, wherein, Also includes: The system performs risk assessments on the current user's risky actions and triggers corresponding intervention measures. Provides real-time validation, plain language error prompts, and history rollback during the input process; When a user experiences an error, a help menu will automatically pop up and support contacting emergency contacts. Automatically adjust interface parameters based on environmental conditions; Collect user interaction behavior data and calculate a health score based on the interaction behavior data, so as to dynamically update the user profile and system configuration corresponding to the current user according to the health score.

8. A bank accessible service interaction device, characterized by, include: The information collection module is used to collect multi-dimensional capability information of the current user based on a preset bank application, and generate a user profile corresponding to the current user based on the multi-dimensional capability information. The multidimensional capability information includes visual ability information, hearing ability information, cognitive ability information, and motor ability information; The instruction acquisition module is used to receive operation instructions input by the current user based on a preset input method using a preset bank application, and to make a fusion decision based on the operation instructions and the user profile to determine the target interaction channel; The preset input methods include one or more combinations of voice input, gesture input, touch input, eye-tracking input, and peripheral input. The target interaction channels include visually impaired channels, hearing impaired channels, and cognitive impairment channels; The interface generation module is used to dynamically reconstruct the layout and components of the current operation interface based on the user profile and the semantic analysis results of the current operation interface, so as to generate a personalized interface adapted to the current user. The interactive feedback module is used to generate corresponding interactive feedback based on the operation command and the personalized interface, and output the interactive feedback to the current user through the target interaction channel.

9. An electronic device, comprising: include: Memory, used to store computer programs; A processor for executing the computer program to implement the bank accessibility service interaction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the bank accessibility service interaction method as described in any one of claims 1 to 7.

Images