A self-service terminal voice navigation interaction method, a self-service terminal and a system

Abstract

The present application relates to the field of communication, and relates to a self-service terminal voice navigation interaction method, a self-service terminal and a system, wherein the method comprises the following steps: receiving a handshake message from a visual impairment auxiliary input device; extracting a device identifier and a function descriptor, performing device type identification according to the device identifier, and performing function capability identification according to the function descriptor; loading a corresponding interaction adaptation protocol according to the results of the device type identification and the function capability identification, and establishing a unified mapping channel between the self-service terminal and the visual impairment auxiliary input device; receiving original input data from the visual impairment auxiliary input device through the unified mapping channel, and mapping the original input data into standard control instructions; in response to the standard control instructions, executing corresponding business operation logic, and generating and playing voice feedback signals associated with the business operation logic. The method of the present application can reduce the adaptation cost of the self-service terminal.

Description

Technical Field

[0001] This invention relates to the field of communications. More specifically, this invention relates to a voice navigation interaction method for a self-service terminal, a self-service terminal, and a system. Background Technology

[0002] With the rapid development of smart cities and self-service technologies, self-service terminals (such as self-service ordering machines, vending machines, and government service kiosks) have been widely used in various public and commercial scenarios. To ensure the right of visually impaired people to access information and services equally, assistive input devices for self-service terminals have emerged. These devices are typically equipped with tactilely identifiable physical buttons and use voice navigation to assist visually impaired users in completing menu selections, information input, and payment confirmations.

[0003] However, in existing technological applications, the connection and adaptation between visually impaired assistive input devices and self-service terminals face numerous challenges. First, existing assistive input devices are often strongly tied to specific communication interfaces (such as USB, Bluetooth, or RS232 serial ports), resulting in poor device versatility. For example, a keyboard for the Braille that only supports a USB interface cannot be directly used in older self-service terminals that only have a serial port, limiting the widespread application of the device. Second, the underlying software architectures of different self-service terminal manufacturers vary significantly, typically requiring the development of dedicated drivers and interaction logic for specific hardware models. When assistive devices change from external to embedded integration, or when the communication protocol is changed, the self-service terminal's system software often needs to be redeveloped or heavily modified, leading to high adaptation costs and difficult maintenance.

[0004] In summary, existing communication methods between self-service terminals and visually impaired assistive input devices suffer from poor hardware compatibility, high adaptation costs, and difficult maintenance. Therefore, there is an urgent need for a communication and adaptation method that can mask underlying hardware differences and achieve unified recognition and interactive control. Summary of the Invention

[0005] To address the problems of poor hardware compatibility, high adaptation costs, and difficult maintenance in existing technologies regarding communication methods between self-service terminals and visually impaired assistive input devices, this invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a voice navigation interaction method for a self-service terminal, comprising: Establish a physical or wireless communication connection with the visually impaired assistive input device through a preset communication interface, and receive handshake messages from the visually impaired assistive input device; The handshake message is parsed to extract the device identifier and function descriptor. The device type is identified based on the device identifier, and the function capability is identified based on the function descriptor, so as to determine the interactive features and control permissions supported by the visually impaired assistive input device. Based on the results of device type identification and functional capability identification, the corresponding interaction adaptation protocol is loaded, and a unified mapping channel between the self-service terminal and the visually impaired assistive input device is established based on the interaction adaptation protocol. The system receives raw input data from the visually impaired assistive input device through the unified mapping channel and maps the raw input data into standard control commands. In response to the standard control command, the corresponding business operation logic is executed, and a voice feedback signal associated with the business operation logic is generated and played. The capability identification includes: parsing the key quantity definition, long-press support status, multi-key combination support status, and dedicated navigation key definition from the function descriptor; generating a capability configuration table for the visually impaired assistive input device based on the parsed content; and matching the capability configuration table with the current business system requirements of the self-service terminal to determine whether to enable the full-function navigation mode or the simplified blind operation mode.

[0007] Preferably, receiving the handshake message from the visually impaired assistive input device includes: sending a device query command to the preset communication interface and receiving a response data packet returned by the visually impaired assistive input device in response to the device query command; or directly receiving device description information actively broadcast by the visually impaired assistive input device when establishing the physical connection or wireless communication connection.

[0008] Preferably, the step of loading the corresponding interaction adaptation protocol and establishing a unified mapping channel between the self-service terminal and the visually impaired assistive input device based on the interaction adaptation protocol includes: Search the preset configuration database for the driver library path corresponding to the device identifier; Load the dynamic link library pointed to by the driver library path, and instantiate a low-level communication driver object adapted to the preset communication interface from the dynamic link library; Based on the results of the functional capability identification, configure an instruction parser that matches the underlying communication driver object; A data transmission stream is established between the underlying communication driver object and the instruction parser to construct the unified mapping channel, thereby shielding the transmission differences between different preset communication interfaces.

[0009] Preferably, mapping the raw input data to standard control commands includes: Extract hexadecimal key-value codes or pulse count values ​​from the original input data; The key-value code or pulse count value is used as an index to search in the mapping table associated with the interaction adaptation protocol; Obtain the logical operation semantics corresponding to the search results, and mark the logical operation semantics as the standard control instructions. The standard control instructions include cursor movement instructions, level confirmation instructions, and cancel return instructions.

[0010] Preferably, it further includes a compatibility processing step: when the visually impaired assistive input device is installed in the form of an embedded module, the physical connection is established through the board-level communication interface, and the handshake message is received; The process involves device type identification, functional capability identification, loading of interactive adaptation protocols, and instruction mapping to achieve unified software adaptation across different installation configurations.

[0011] Preferably, the method further includes an interface hot-switching step: During the operation after the unified mapping channel is established, if the current physical connection or wireless communication connection is detected to be disconnected, the connection will be re-established through the second communication interface within a preset time window and a handshake message with the same device identifier will be received. While maintaining the current business operation logic state, the underlying communication driver object corresponding to the second communication interface is invoked to switch the data receiving port of the unified mapping channel to the second communication interface.

[0012] Preferably, it further includes: monitoring the playback status of the voice feedback signal; If new raw input data is received from a visually impaired assistive input device while the voice feedback signal is playing, it is mapped to a new standard control command, the current voice feedback signal is immediately interrupted, and the next level of business process is entered according to the new standard control command.

[0013] In a second aspect, the present invention provides a self-service terminal, comprising: a communication interface module for establishing a physical connection or a wireless communication connection with an external device; a memory for storing a computer program; and a processor for executing the computer program to implement the method as described in any one of claims 1 to 8.

[0014] In a third aspect, the present invention provides a visually impaired assistive interaction system, characterized in that it includes a visually impaired assistive input device and a self-service terminal of the present invention; wherein, the self-service terminal acts as the execution subject, establishes a connection with the visually impaired assistive input device through the communication interface module and interacts with it, performing device identification, protocol adaptation and business control logic.

[0015] The beneficial effects of this invention are as follows: By establishing a unified mapping channel between the self-service terminal and the visually impaired assistive input device, and introducing a dual identification mechanism for device type and functional capabilities, the method of this invention decouples the upper-layer business logic of the self-service terminal from the lower-layer communication interface. Regardless of the communication interface (USB, serial port, or wireless) or access method used by the visually impaired assistive input device, the self-service terminal can be adapted through a unified process without requiring modifications to the upper-layer software for specific hardware. This significantly reduces the adaptation cost of the self-service terminal and improves the system's compatibility and versatility. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart illustrating a voice navigation interaction method for a self-service terminal according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the speech feedback signal generation process according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the structure of a self-service terminal according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the structure of a visually impaired assistive interactive system according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] Example of a voice navigation interaction method for self-service terminals: like Figure 1 As shown, the self-service terminal voice navigation interaction method of the present invention includes: S101. Receive a handshake message from the visually impaired assistive input device, specifically: establish a physical connection or wireless communication connection with the visually impaired assistive input device through a preset communication interface, and receive a handshake message from the visually impaired assistive input device. In this embodiment, the preset communication interface is a universal serial bus interface, an RS232 serial communication interface, or a wireless communication interface.

[0020] Receiving a handshake message from the visually impaired assistive input device includes: sending a device query command to the preset communication interface and receiving a response data packet returned by the visually impaired assistive input device in response to the device query command; or directly receiving device description information actively broadcast by the visually impaired assistive input device when establishing the physical connection or wireless communication connection.

[0021] By supporting two handshake methods—actively sending device query commands or passively receiving device broadcast information—this invention can flexibly adapt to external devices with different communication protocols and operating modes. This increases the flexibility of system connection establishment and ensures that device access identification can be completed quickly and stably under different hardware handshake strategies.

[0022] S102. Identify the device type and functional capabilities of the visually impaired assistive input device, specifically by parsing the handshake message to extract the device identifier and function descriptor, identifying the device type based on the device identifier, and performing functional capability identification based on the function descriptor, so as to determine the interactive features and control permissions supported by the visually impaired assistive input device. In this embodiment, the function capability identification includes: parsing the key quantity definition, long press support status, multi-key combination support status, and dedicated navigation key definition from the function descriptor; generating a capability configuration table for the visually impaired assistive input device based on the parsed content; and matching the capability configuration table with the current business system requirements of the self-service terminal to determine whether to enable the full-function navigation mode or the simplified blind operation mode.

[0023] Capability configuration tables typically contain the following four categories of information: 1. Basic equipment identity information Used by the system to identify the origin and legitimacy of a device, and to decide whether to trust the device.

[0024] Device ID: A unique identifier for a device.

[0025] Vendor ID: Used for whitelist verification to ensure that the connected device is a certified and compliant device.

[0026] Interface Type: Indicates the current connection method (such as USB HID, RS232 serial port, Bluetooth, etc.) so that the system can call the corresponding underlying communication channel.

[0027] 2. Hardware capabilities characteristics.

[0028] Hardware capability characteristics are used to describe the physical properties and interaction potential of a device, and are used by the system to decide which advanced functions to enable.

[0029] Number of buttons: For example, "5 buttons" or "12 buttons" determines which UI navigation template the system loads (whether it is a cross navigation or a nine-grid number input).

[0030] Long press support status: Boolean value (True / False), indicating whether the device can recognize and send a long press signal.

[0031] Key combination support status: Indicates whether the device supports pressing multiple keys simultaneously (such as "Confirm + Cancel" triggering a reset).

[0032] Dedicated navigation key indicator: Confirm whether the device has dedicated directional keys.

[0033] 3. Key-value mapping table.

[0034] The key-value mapping table defines the correspondence rules from "physical actions" to "logical instructions".

[0035] Index key: The original physical key value encoding (e.g., hexadecimal 0x31, or pulse count value).

[0036] Mapping values: System-standardized logical operation semantics (e.g., CMD_NAV_UP means "cursor up", CMD_CONFIRM means "confirm").

[0037] 4. System decision configuration, including: Activation mode: such as "full-featured navigation mode" or "simplified blind operation mode").

[0038] Voice feedback levels: such as "detailed mode" (reads out all prompts) or "compact mode" (reads out only the core results). Devices with fewer buttons usually require more detailed voice guidance.

[0039] By parsing function descriptors to identify functional capabilities, the system can accurately determine the specific interactive capabilities of input devices (such as the number of buttons and whether long presses are supported). Self-service terminals can dynamically adjust their interaction strategies based on the actual capabilities of the hardware (e.g., enabling simplified mode for basic devices and navigation mode for full-featured devices), thereby maximizing hardware performance and optimizing the user experience for visually impaired users.

[0040] S103. Establish a unified mapping channel between the self-service terminal and the visually impaired assistive input device, specifically: based on the results of device type identification and functional capability identification, load the corresponding interaction adaptation protocol, and establish a unified mapping channel between the self-service terminal and the visually impaired assistive input device based on the interaction adaptation protocol. S104. Mapping the raw input data of the visually impaired assistive input device into standard control commands, specifically: receiving raw input data from the visually impaired assistive input device through the unified mapping channel, and mapping the raw input data into standard control commands; S105. Generate and play a voice feedback signal, specifically: in response to the standard control command, execute the corresponding business operation logic, and generate and play a voice feedback signal associated with the business operation logic.

[0041] The method in this embodiment establishes a unified mapping channel between the self-service terminal and the visually impaired assistive input device, and introduces a dual identification mechanism for device type and functional capabilities, thereby decoupling the upper-layer business logic and the lower-layer communication interface of the self-service terminal. Regardless of the communication interface (USB, serial port, or wireless) or access method used by the visually impaired assistive input device, the self-service terminal can be adapted through a unified process without modifying the upper-layer software for specific hardware, significantly reducing the adaptation cost of the self-service terminal and improving the system's compatibility and versatility.

[0042] In one embodiment, loading the corresponding interaction adaptation protocol and establishing a unified mapping channel between the self-service terminal and the visually impaired assistive input device based on the interaction adaptation protocol includes: S201. Search for the driver library path corresponding to the device identifier in the preset configuration database; S202. Load the dynamic link library pointed to by the driver library path, and instantiate a low-level communication driver object adapted to the preset communication interface from the dynamic link library. S203. Based on the result of the functional capability identification, configure an instruction parser that matches the underlying communication driver object; S204. Establish a data transmission stream between the underlying communication driver object and the instruction parser to construct the unified mapping channel, thereby shielding the transmission differences of different preset communication interfaces.

[0043] A unified mapping channel is constructed using a series of steps including "path lookup, dynamic loading, object instantiation, and resolver configuration," enabling dynamic adaptation at the driver level. The technical advantages are: it shields the data transmission differences of the underlying physical interfaces, allowing the system to hot-load the corresponding driver at runtime, supporting new types of devices without recompiling or restarting the system, thus enhancing system scalability and maintenance convenience.

[0044] In one embodiment, mapping the raw input data to standard control commands includes: S301. Extract hexadecimal key value codes or pulse count values ​​from the original input data; S302. Using the key value code or pulse count value as an index, perform a retrieval in the mapping relationship table associated with the interaction adaptation protocol; S303. Obtain the logical operation semantics corresponding to the search results, and mark the logical operation semantics as the standard control instructions. The standard control instructions include cursor movement instructions, level confirmation instructions, and cancel return instructions.

[0045] By mapping the original key-value codes or pulse count values ​​to "standard control commands" (such as cursor movement and confirmation), the standardization of input data is achieved. This allows upper-layer business systems to handle only unified logical semantics without needing to concern themselves with specific hardware encoding formats, further deepening the decoupling of software and hardware and preventing business system failures due to changes in hardware parameters.

[0046] In one embodiment, a compatibility processing step is also included: S501. When the visually impaired assistive input device is installed in the form of an embedded module, the physical connection is established through the board-level communication interface, and the handshake message is received. S502, perform device type identification, function capability identification, interactive adaptation protocol loading and instruction mapping steps to achieve unified software adaptation under different installation modes.

[0047] By supporting embedded module form factors and implementing a unified adaptation process, the differences in software processing between external and embedded devices are eliminated. When a self-service terminal is upgraded from an external auxiliary keyboard to an integrated auxiliary module, there is no need to redevelop the software system; only the existing recognition and mapping logic needs to be reused, significantly reducing the software modification costs during hardware iteration.

[0048] In one embodiment, the method further includes an interface hot-switching step, comprising: S601. During the operation after the establishment of the unified mapping channel, if the current physical connection or wireless communication connection is detected to be disconnected, the connection is re-established through the second communication interface within a preset time window and a handshake message with the same device identifier is received. S602. Keep the current business operation logic state unchanged, call the underlying communication driver object corresponding to the second communication interface, and switch the data receiving port of the unified mapping channel to the second communication interface.

[0049] By using the interface hot-switching process, the system can maintain the upper-layer business logic state unchanged when the physical connection is broken and re-established (or even the interface is changed), thereby enhancing the robustness of the system and ensuring that the current interaction progress is not lost when the user accidentally plugs or unplugs the port or changes the port. This provides a continuous and seamless user experience and effectively prevents the situation of circumventing the technical solution by simply changing the interface.

[0050] In one embodiment, it also includes: S701. Monitor the playback status of the voice feedback signal; S702. If new raw input data from a visually impaired assistive input device is received while the voice feedback signal is playing, then the data is mapped to a new standard control command and the current voice feedback signal is immediately interrupted. The next level of business process is then initiated according to the new standard control command.

[0051] By monitoring the voice playback status and immediately interrupting the current playback upon receiving a new instruction, "exclusive" instant feedback is achieved. The method in this embodiment can solve the common problems of voice lag or stacking in visually impaired interaction, ensuring that auditory feedback is strictly synchronized with the user's current operation, thereby improving the accuracy and fluency of blind operation.

[0052] In one embodiment, such as Figure 2 As shown, in response to the standard control command, the corresponding business operation logic is executed, and a voice feedback signal associated with the business operation logic is generated and played, including: S401. Obtaining the context state and verifying the instruction: Specifically, after receiving the mapped standard control instruction through the unified mapping channel, the system first reads the context state parameters of its current user interface. These context state parameters include at least the current page identifier and the index value of the currently focused control. Then, the system performs logical association analysis between the standard control instruction and the read context state parameters to verify the validity of the standard control instruction under the current page logic.

[0053] For example, standard control commands can be cursor movement commands or other commands.

[0054] S402. Locate the target control and update the focus. Specifically, when the verification result is valid, the self-service terminal calculates and locates the next logically adjacent interactive control in the pre-stored interface control tree, starting from the index value of the control currently in focus, according to the semantic direction of the standard control instruction; performs a focus migration operation, cancels the highlight state of the current control and activates the highlight state of the next interactive control, thereby determining the target control object under the current business logic.

[0055] For example, the semantic direction of standard control commands can be "right", "left", "up" or "down", etc.

[0056] S403. Extract attributes and generate broadcast text, specifically: access the target control object, extract static description tags and real-time business data bound to them; according to the preset accessible voice template, dynamically concatenate the extracted static description tags and the real-time business data to generate a complete voice broadcast text string.

[0057] In this embodiment, the static description label can be the product name. Real-time business data can include information such as price and inventory status.

[0058] S404. Perform speech synthesis and mutually exclusive playback, specifically: call the speech synthesis engine integrated within the self-service terminal to convert the generated speech broadcast text string into a digitized PCM audio data stream; before playback, detect the occupancy status of the current audio output channel; if it is detected that the audio output channel is playing the feedback voice of the previous instruction, immediately send an interrupt signal to terminate the current playback and clear the buffer, and then load the PCM audio data stream generated this time into the buffer; finally, convert the PCM audio data stream into an analog signal through a digital-to-analog converter circuit, and output the voice feedback signal to the outside through a physical speaker or headphone jack.

[0059] Self-service terminal example: like Figure 3 As shown, the present invention also provides a self-service terminal, the self-service terminal comprising: a communication interface module for establishing a physical connection or wireless communication connection with an external device; a memory for storing a computer program; and a processor for executing the computer program to implement the self-service terminal voice navigation interaction method described in the above embodiments.

[0060] Example of an assistive interactive system for the visually impaired: like Figure 4 As shown, the present invention also provides a visually impaired assistive interaction system, including a visually impaired assistive input device and the self-service terminal described in the above embodiments; wherein, the self-service terminal acts as the execution subject, establishes a connection with the visually impaired assistive input device through the communication interface module and interacts with it, performing device identification, protocol adaptation and business control logic.

[0061] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A self-service terminal voice navigation interaction method, characterized in that, include: Establish a physical or wireless communication connection with the visually impaired assistive input device through a preset communication interface, and receive handshake messages from the visually impaired assistive input device; The handshake message is parsed to extract the device identifier and function descriptor. The device type is identified based on the device identifier, and the function capability is identified based on the function descriptor, so as to determine the interactive features and control permissions supported by the visually impaired assistive input device. Based on the results of device type identification and functional capability identification, the corresponding interaction adaptation protocol is loaded, and a unified mapping channel between the self-service terminal and the visually impaired assistive input device is established based on the interaction adaptation protocol. The system receives raw input data from the visually impaired assistive input device through the unified mapping channel and maps the raw input data into standard control commands. In response to the standard control command, the corresponding business operation logic is executed, and a voice feedback signal associated with the business operation logic is generated and played. The function capability identification includes: parsing the key quantity definition, long press support status, multi-key combination support status, and dedicated navigation key definition from the function descriptor; Based on the parsed content, a capability configuration table for the visually impaired assistive input device is generated; the capability configuration table is then matched with the current business system requirements of the self-service terminal to determine whether to enable the full-function navigation mode or the simplified blind operation mode.

2. The self-service terminal voice navigation interaction method as described in claim 1, characterized in that, Receiving a handshake message from the visually impaired assistive input device includes: sending a device query command to the preset communication interface and receiving a response data packet returned by the visually impaired assistive input device in response to the device query command; or directly receiving device description information actively broadcast by the visually impaired assistive input device when establishing the physical connection or wireless communication connection.

3. The self-service terminal voice navigation interaction method as described in claim 1, characterized in that, The step of loading the corresponding interaction adaptation protocol and establishing a unified mapping channel between the self-service terminal and the visually impaired assistive input device based on the interaction adaptation protocol includes: Search the preset configuration database for the driver library path corresponding to the device identifier; Load the dynamic link library pointed to by the driver library path, and instantiate a low-level communication driver object adapted to the preset communication interface from the dynamic link library; Based on the results of the functional capability identification, configure an instruction parser that matches the underlying communication driver object; A data transmission stream is established between the underlying communication driver object and the instruction parser to construct the unified mapping channel, thereby shielding the transmission differences between different preset communication interfaces.

4. The self-service terminal voice navigation interaction method as described in claim 1, characterized in that, The process of mapping raw input data into standard control commands includes: Extract hexadecimal key-value codes or pulse count values ​​from the original input data; The key-value code or pulse count value is used as an index to search in the mapping table associated with the interaction adaptation protocol; Obtain the logical operation semantics corresponding to the search results, and mark the logical operation semantics as the standard control instructions. The standard control instructions include cursor movement instructions, level confirmation instructions, and cancel return instructions.

5. The self-service terminal voice navigation interaction method as described in claim 1, characterized in that, It also includes a compatibility processing step: when the visually impaired assistive input device is installed in the form of an embedded module, the physical connection is established through the board-level communication interface, and the handshake message is received; The process involves device type identification, functional capability identification, loading of interactive adaptation protocols, and instruction mapping to achieve unified software adaptation across different installation configurations.

6. The self-service terminal voice navigation interaction method as described in claim 1, characterized in that, The method also includes an interface hot-switching step: During the operation after the unified mapping channel is established, if the current physical connection or wireless communication connection is detected to be disconnected, the connection will be re-established through the second communication interface within a preset time window and a handshake message with the same device identifier will be received. While maintaining the current business operation logic state, the underlying communication driver object corresponding to the second communication interface is invoked to switch the data receiving port of the unified mapping channel to the second communication interface.

7. The self-service terminal voice navigation interaction method as described in any one of claims 1 to 6, characterized in that, Also includes: Monitor the playback status of the voice feedback signal; If new raw input data is received from a visually impaired assistive input device while the voice feedback signal is playing, it is mapped to a new standard control command, the current voice feedback signal is immediately interrupted, and the next level of business process is entered according to the new standard control command.

8. A self-service terminal, characterized in that, include: The communication interface module is used to establish a physical connection or wireless communication connection with external devices. Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 7.

9. A visually impaired assistive interactive system, characterized in that, It includes a visually impaired assistive input device and a self-service terminal as described in claim 8; wherein, the self-service terminal acts as the execution entity, establishes a connection with the visually impaired assistive input device through the communication interface module and interacts with it to perform device identification, protocol adaptation and business control logic.

Images