Interactive system combining automated voice mechanism with visual feedback mechanism and interactive method thereof

Abstract

To provide an interactive system combining an automated voice mechanism with a visual feedback mechanism, and an interactive method thereof.SOLUTION: A first terminal device 11 and a telecommunications server 13 mutually transmit messages through a public switched telephone network 10. The first terminal device 11 transmits an input message to the telecommunications server 13 and receives a voice feedback message from the telecommunications server 13. The telecommunications server 13 converts the input message to a processing message and transmits the processing message to an information management server 23. On the basis of the processing message, the information management server 23 transmits a forward message to an information distribution server 25 through Internet 20. On the basis of the forward message's content, the information distribution server 25 transmits a control message to a second terminal device 21. According to the forward message, the second terminal device 21 transmits a request message to a screen content management server 27 to obtain a returned screen feedback message.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an interactive system, and more particularly to an interactive system and an interactive method that simultaneously combines an automatic voice mechanism and a visual feedback mechanism, which allows a user to view screen information that matches the voice information from a second terminal device while operating a first terminal device and listening to the voice information transmitted from the first terminal device. [Background technology]

[0002] Interactive Voice Response (IVR), also known as an interactive voice response system, allows users to interact with a dedicated telecommunication host on the vendor's side through telephone. After the user dials the dedicated telecommunication host to enter the IVR system, the IVR system will play a series of pre-recorded voice content, and the user can respond by pressing the telephone buttons according to the voice guidance he hears to obtain the required information or complete a certain operation (such as credit card authorization).

[0003] IVR systems and related technologies are used in many industries. For example, many banks and service providers use IVR systems to provide 24-hour customer service, allowing users to check account balances, pay bills, transfer funds and other operations; airlines, travel agencies and medical institutions also use IVR systems to allow users to check flight information, book tickets, confirm reservation times and so on. From this, we can see that IVR systems have many benefits, but the most obvious one is that IVR systems can greatly improve the efficiency and availability of services. Through automated flows, vendors can provide services anytime and anywhere without manual intervention, which not only reduces human resources but also ensures the accuracy and immediacy of information, shortens user waiting times, and improves customer service satisfaction.

[0004] However, although IVR systems have obvious advantages in terms of providing diverse and efficient services, they still have some disadvantages, especially for certain user groups. In fact, for the elderly and those with hearing impairments, IVR systems that rely only on voice may not be very user-friendly. Pre-recorded voice messages may have problems such as distortion or lack of clarity during transmission, and the aforementioned users may not be able to understand the voice content, as well as following the voice-guided operations may be inappropriate for the elderly or the response may be slow, causing them to miss important instructions. The aforementioned situations may annoy users and affect the user experience. In addition, most IVR systems have a certain response time limit, and if the user does not make a selection or respond within the time limit, the IVR system may automatically end the call. This will definitely bring pressure to elderly users who need more time to process information or make decisions, making them feel inconvenient and uncomfortable when using the IVR system.

[0005] Therefore, the industry and research institutions have begun to look for solutions, and some operators have adopted methods such as optimizing voice quality, adjusting playback speed, and adjusting response time according to user needs to improve user experience. However, for the elderly and people with hearing impairments, if voice guidance cannot fulfill its role, they usually choose to transfer to human customer service. Under the premise that the IVR system reduces the human resources of operators, the above-mentioned users may have to wait longer for human customer service, which may reduce user experience and satisfaction. Therefore, how to effectively solve the above-mentioned problem is an important task of the present invention. Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the fact that conventional IVR systems can only provide voice guidance functions, which are obviously not friendly enough for some users, the present inventor has made intensive research efforts and, after a long period of research and experiments, developed an interactive system and its interactive method of the present invention, which combines an automatic voice mechanism and a visual feedback mechanism to provide multiple auditory and visual guidance simultaneously, ensure that most users can clearly understand the guidance information, reduce misunderstandings, and help users make the right choice. [Means for solving the problem]

[0007] An object of the present invention is to provide an interactive system combining an automated voice mechanism and a visual feedback mechanism, comprising a first terminal device, a telecommunications server, an information management server, an information distribution server, a second terminal device, and a screen content management server. The first terminal device is connectable to the public switched telephone network and has a user input interface capable of generating an input message including at least one input information according to a user's operation. The telecommunications server can receive an input message sent from a first terminal device via a public switched telephone network, and generate a processing message based on the input message, the processing message including a first identification number representing the first terminal device and interaction information corresponding to the input information; the telecommunications server can also connect to a voice database, and based on the content of the input message, retrieve at least one pre-recorded voice segment information from the voice database to generate a voice feedback message, and return the voice feedback message to the first terminal device via the public switched telephone network. The information management server is connected to the telecommunications server for receiving a processing message sent from the telecommunications server, and the information management server can also be connected to a user database, and can search for corresponding user information from the user database based on the first identification number of the processing message, and generate a forwarding message including at least the user information and the interaction information. The information distribution server is connected to the information management server via the Internet to receive the forwarding message sent from the information management server, and can send a control message including at least interaction information to a corresponding second terminal device based on the user information of the forwarding message. The second terminal device can receive the control message transmitted from the information distribution server via the Internet and generate a request message including at least the second identification number and the interaction information. The screen content management server can receive a request message sent from a second terminal device via the Internet, and is connected to a screen database, and can obtain at least one default screen information from the screen database based on interaction information of the request message, generate a screen feedback message, and return the screen feedback message to the second terminal device via the Internet. In this way, guidance and hints such as audio and visual can be provided simultaneously to users with special needs, enabling the users to quickly obtain and understand said guidance and hint information, reducing the risk of the users making mistakes and enabling the users to make the right choices to obtain the information they need.

[0008] Alternatively, the interactive system further comprises a voice editing tool for editing each pre-recorded voice segment information in the voice database and the corresponding input information required for each pre-recorded voice segment information.

[0009] Alternatively, the interactive system further comprises a screen voice editing tool for editing each default screen information in the screen database and the corresponding interaction information required for each default screen information.

[0010] Alternatively, the first terminal device and the second terminal device are the same device.

[0011] Alternatively, the voice database is located in a telecommunications server.

[0012] Alternatively, the telecommunications server is installed with a voice recognition module.

[0013] Alternatively, the screen database resides in a screen content management server.

[0014] Another object of the present invention is to provide a method for use in a dialogue system that combines automated speech and visual feedback mechanisms. The interactive system includes a first terminal device, a telecommunications server, an information management server, an information distribution server, a second terminal device, and a screen content management server. After the first terminal device establishes a call with the telecommunication server, the interaction method makes the telecommunication server, the information management server, the information distribution server and the screen content management server perform the following steps: The telecommunications server performs the steps of receiving an input message sent from the first terminal device via the public switched telephone network, and generating a processing message including at least the first identification number and interaction information based on the input message. The telecommunications server performs the steps of obtaining at least one corresponding pre-recorded voice segment information from a voice database based on the processing message, generating a voice feedback message, returning the voice feedback message to the first terminal device via the public switched telephone network, and the telecommunications server also sends the processing message to the information management server via the Internet. The information management server performs the steps of, after receiving the processing message sent from the telecommunications server, searching for corresponding user information from a user database based on the first identification number of the processing message, generating a forwarding message, and sending the forwarding message including at least the user information and the interaction information to the information distribution server via the Internet. The information distribution server executes the steps of receiving a forwarding message sent from the information management server, generating a control message including at least interaction information based on user information in the forwarding message, and sending the control message to a corresponding second terminal device via the Internet, in which the second terminal device generates a request message based on the control message. The screen content management server executes the steps of receiving a request message sent from the second terminal device via the Internet, obtaining at least one default screen information from a screen database based on interaction information of the request message, generating a screen feedback message, and returning the screen feedback message to the second terminal device via the Internet. In this way, the interactive method of the present invention can simultaneously provide auditory and visual information to enhance the user's understanding of the guidance and hint information, and some guidance instructions are more intuitive visually and some guidance instructions are more concise in audio, thereby improving the overall efficiency of the user's use and operation.

[0015] Alternatively, the first identification number is used to correspond to the first terminal device, and the interaction information is used to correspond to input information in the processing message, and the input information is generated by the first terminal device after a user's operation.

[0016] Alternatively, the information management server stops generating and sending the forwarding message after it cannot search for the corresponding user information from the user database according to the first identification number of the processing message.

[0017] In order to facilitate a further appreciation and understanding of the objects, technical features and effects of the present invention, the present invention will now be described in detail with reference to the accompanying drawings, giving examples. [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of a system configuration of a dialogue system according to the present invention. [Diagram 2] FIG. 2 is a sequence diagram of the interaction method of the present invention. [Diagram 3] 4 is a schematic diagram of a screen of a second terminal device of the present invention. FIG. [Figure 4] 11 is another schematic diagram of a screen of the second terminal device of the present invention. FIG. [Diagram 5] FIG. 4 is a further schematic diagram of a screen of the second terminal device of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] In order to make the objectives, technical contents and advantages of the present invention clearer, the disclosed embodiments of the present invention will be described in more detail below in conjunction with specific embodiments and with reference to the drawings. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification, the present invention can be implemented or applied through other different specific embodiments, and various contents of this specification can be modified and changed in various ways based on different perspectives and applications without departing from the technical idea of ​​the present invention. In addition, it should be noted in advance that the drawings of the present invention are merely schematic and are not drawn based on actual size. In addition, unless otherwise clearly specified in the context or otherwise defined, the meaning of "a" and "the" in the present invention also includes plural shapes. In addition, in the following embodiments, the related technical contents of the present invention will be described in more detail, but the disclosures are not intended to limit the scope of the present invention.

[0020] It is to be understood that the terms used herein generally have their ordinary meaning in the art, and in the event of any conflict, the definitions given herein shall prevail. Because the same thing can be expressed in many ways, alternative words and synonyms may be used for the terms discussed or illustrated herein, and there is no particular limitation as to whether they are described or illustrated herein, and the use of one or more synonyms does not exclude other synonyms. The examples used anywhere in the description of the present invention, including the use of terms, are merely illustrative and in no way limit the scope and meaning of the present invention or terms. Similarly, the present invention is not limited to the various examples disclosed in the specification. In the present specification, terms such as first, second, third, etc. may be used to describe various elements, but each element should not be limited by the aforementioned terms, and the aforementioned terms are mainly used to distinguish one element from another element, and do not impose any substantial restrictions on any element, and do not limit the assembly or installation order of each element in actual application. It is to be noted that, as used herein, the term "or" includes any combination of one or more of the related items listed, depending on the actual situation.

[0021] The present invention relates to an interactive system (hereinafter referred to as an interactive system) that combines an automatic speech mechanism and a visual feedback mechanism, and an interactive method thereof. Referring to FIG. 1, the interactive system T is composed of an automated voice system 1 and a visual feedback system 2, the automated voice system 1 has an automated voice mechanism for providing voice feedback to the user, and the visual feedback system 2 has a visual feedback mechanism for providing visual feedback to the user. The automated voice system 1 may be an interactive voice response (IVR) system, where a user dials a telephone number designated by a vendor to access a dedicated telecommunication host on the vendor side, and the user can obtain required information by inputting a specific key and listening to pre-recorded voice content. Since the IVR system is a mature technology, only the necessary technical means related to the present invention will be described below, but any system equipped with IVR-related technology can be regarded as the automated voice system 1 of the present invention, and this point will be described in advance.

[0022] 1, the automated voice system 1 includes a first terminal device 11 and a telecommunications server 13, where the first terminal device 11 can be a landline or a mobile communication device (e.g., a mobile phone, a smartphone, etc.), and can directly or indirectly connect to a Public Switched Telephone Network (PSTN) 10 to exchange information (call data) with the telecommunications server 13. It is specifically mentioned here that the mobile communication device first establishes a wireless connection between the mobile communication device and a local telecommunications tower in a mobile communication network (e.g., 4G or 5G), and then the mobile communication device and the telecommunications server 13 also exchange information through the Public Switched Telephone Network 10.

[0023] Referring again to FIG. 1, the first terminal device 11 is provided with a user input interface 111 (e.g., physical buttons, touch panel, microphone, etc.), and the first terminal device 11 can generate an input message according to the result of the user operating the user input interface 111, and send the input message to the telecommunications server 13. When the first terminal device 11 has not established a call with the telecommunications server 13, the input message is a call request including a first identification number (e.g., telephone number) and input information, and the input information may be blank. In some embodiments, the call request may only have the first identification number (e.g., telephone number), which corresponds to the input information being blank in the above case. Also, when the first terminal device 11 has established a call with the telecommunications server 13, the input message only includes input information (e.g., button information) according to the specific embodiment (e.g., landline) of the first terminal device 11 and the technology used.

[0024] 1, the first identification number is used to identify the caller, so as to ensure that the first terminal device 11 establishes a correct call with the telecommunication server 13, and the telecommunication server 13 sends the call data (such as voice content) to the corresponding first terminal device 11. The first identification number may be a phone number, an International Mobile Subscriber Identity (IMSI), an International Mobile Equipment Identity (IMEI), or other unique identifier sufficient for the telecommunication server 13 to identify and maintain the call, and the input information may be button information, voice information, or blank. If the user input interface 111 is a physical button / virtual button, the input information is the button information of the user pressing the physical button / virtual button, and if the user input interface 111 is a microphone, the input information is the voice command dictated by the user.

[0025] It is worth noting that, referring again to FIG. 1, when a user uses the automated voice system 1, there are two scenarios: when the user calls the vendor (inbound) and when the vendor calls the user (outbound). Taking the user calling the vendor (inbound) as an example, the user first dials the vendor's telephone number through the first terminal device 11, and the above process corresponds to the first terminal device 11 sending an input message (call request) to the telecommunications server 13, the call request includes a first identification number (e.g., telephone number), and the telecommunications server 13 can establish a session based on the first identification number. After that, the user presses physical / virtual buttons one after another or dictates a voice command according to the voice content heard each time. If the call continues, the first terminal device 11 only sends button information or voice command to the telecommunications server 13, and does not repeatedly send the telephone number, IMSI or IMEI. However, in order to ensure the accuracy of the call between the first terminal device 11 and the telecommunications server 13, the call process between the first terminal device 11 and the telecommunications server 13 may include a unique identifier corresponding to the session. The unique identifier of the session is generated by the telecommunications server 13 and is an identifier for tracking the call; in other words, the first identification number is used to enable the telecommunications server 13 to accurately establish a call with the first terminal device 11. Since any information related to the above effects can be regarded as the first identification number, the specific form of the first identification number may change during the call process (e.g., from a telephone number to a unique identifier of the session), but their meanings remain unchanged (both represent the source of the first terminal device 11).

[0026] Referring again to FIG. 1, the usage scenario shows that when the vendor side makes an outbound call to the user, the telecommunications server 13 has previously obtained the first identification number (e.g., telephone number) of the first terminal device 11, and after the telecommunications server 13 establishes a call with the first terminal device 11, it can receive an input message sent from the first terminal device 11. The input message includes at least input information, such as the user pressing physical buttons / virtual buttons one after another according to the voice content heard each time, or dictating a voice command. It should be noted that in some embodiments, the user input interface 111 is not limited to a single device, and may include multiple devices, for example, the user input interface 111 may include both a touch panel and a microphone, and generate a corresponding input message according to the user's operation scenario. For example, in the same call stage, the user can first press a virtual button on the smartphone to generate button information, and then the user can verbally speak a voice command into the microphone.

[0027] Referring again to FIG. 1 , the telecommunications server 13 can receive an input message sent from the first terminal device 11 and read the content of the input message, and the telecommunications server 13 can obtain at least one pre-recorded voice segment information from the voice database 14 based on the content of the input message to generate a voice feedback message, and return the voice feedback message to the first terminal device 11 through the public switched telephone network 10. For example, if the first terminal device 11 has not established a call with the telecommunications server 13 and the telecommunications server 13 determines that the input information is blank, it can obtain pre-recorded voice segment information from the voice database 14, which may be “Thank you for calling. For Chinese service, please press ‘1’. For English service, please press 2”, and then the telecommunications server 13 sends a voice feedback message including the pre-recorded voice segment information to the first terminal device 11, so that the user can listen to the above-mentioned voice content.

[0028] Referring again to FIG. 1 , during a call, after the user hears voice content such as “If you want to inquire about your balance, press ‘1’; if you want to inquire about your transaction details, press ‘2’”, if the user presses the ‘1’ button on the physical / virtual button, or verbally speaks a voice command such as ‘1’ or ‘inquire about your balance’, the first terminal device 11 can send an input message including the above-mentioned input information to the telecommunications server 13, and the telecommunications server 13 retrieves relevant data from a corresponding database or other server (e.g., a financial database or server that can retrieve the user’s balance information) based on the input information (e.g., button information corresponding to ‘1’ or a voice command), and causes the voice database 14 to retrieve corresponding pre-recorded voice segment information according to the content of the relevant data, generate a voice feedback message, and then send the voice feedback message to the first terminal device 11, allowing the user to hear the voice content of the voice feedback message (e.g., balance XXXXX yuan) through the first terminal device 11.

[0029] Referring again to FIG. 1, in some embodiments, the voice database 14 may be located on the telecommunications server 13. In some embodiments, the voice database 14 may be located on another server. In some embodiments, the telecommunications server 13 or other device may also be provided with a voice editing tool 141 for editing the data in the voice database 14 and the corresponding input information required for each pre-recorded voice segment information (e.g., button information for physical button "1", voice information for "1" dictated by the user, etc.). In some embodiments, a voice recognition module 132 may be installed on the telecommunications server 13 to take the user's dictation and convert it to text, including but not limited to this.

[0030] Referring back to Fig. 1, the telecommunication server 13 generates a corresponding processing message according to the content of the input message, and sends the processing message to the visual feedback system 2 via the Internet 20, the processing message at least includes a first identification number (e.g., a phone number) and interaction information corresponding to the input information, and the content of the interaction information may be the same as the input information, or may be sufficient to represent the content of the input information. For example, the interaction information may be the button information of the physical button "1", the voice command of "1" dictated by the user, or the telecommunication server 13 may receive the voice command of "1" dictated by the user and then convert it into the information content of "balance inquiry" or "button 1", etc. In other words, the telecommunication server 13 may use the input information as the interaction information, or may process the input information first and change it into the corresponding interaction information.

[0031] Also, referring again to FIG. 1, the visual feedback system 2 includes a second terminal device 21, an information management server 23, an information distribution server 25, and a screen content management server 27, and the second terminal device 21 is a device having a display screen, such as an Internet TV, a smartphone, etc. In some embodiments, the second terminal device 21 may be a combination of multiple devices, such as a combination of a set-top box and a television, etc. In some embodiments, the second terminal device 21 and the first terminal device 11 may be the same device (e.g., a smartphone), mainly because the smartphone has a dial function and a display function, for example, its touch panel can be used as a display screen 211, and virtual buttons can also be displayed as a user input interface 111 (the smartphone as the automated voice system 1 in FIG. 1), and the smartphone also has the ability to connect to the public switched telephone network 10 and the Internet 20. Therefore, when playing the role of the automated voice system 1, the smartphone can transmit information directly or indirectly via the public switched telephone network 10. When playing the role of the visual feedback system 2, the smartphone transmits information via the Internet 20.

[0032] 1 again, the information management server 23 is connected to the telecommunication server 13 to receive processing messages sent from the telecommunication server 13, and the information management server 23 is provided with a telecommunication access interface (abbreviated as TAI) for enabling the information management server 23 and the telecommunication server 13 to establish communication and transfer data to each other. The information management server 23 can also be connected to a user database 24, which stores a plurality of user data tables, and the user data tables store user information such as at least a first identification number and a user identification number, and the user identification number is associated (bound) with at least one first identification number and can be associated with a second terminal device 21 so that the information management server 23 can identify the corresponding second terminal device 21 based on the user identification number and correctly transmit a message to the second terminal device 21. According to actual needs, the user identification number may be a unique system number (Embedded OS ID), such as Android ID, Apple ID, SoC unique ID (Linux), etc., or may be an IP address, a Media Access Control address (MAC address), a Universally Unique Identifier (UUID for short), IMEI or IMSI, etc. In other words, the user identification number corresponds to the second terminal device 21 so that the message of the information management server 23 can be correctly transmitted to the second terminal device 21. In addition, the information management server 23 can generate a forwarding message including at least some user information (e.g., user identification number) and interaction information (e.g., button information of physical button “1”) based on the processing message and the contents of the user data table.

[0033] Furthermore, referring again to FIG. 1, the information distribution server 25 may be connected to the information management server 23 via the Internet 20 to receive a forwarding message sent from the information management server 23 and generate a control message including at least the interaction information. In addition, the information distribution server 25 may send a control message to the corresponding second terminal device 21 via wired or wireless technology (e.g., wide area wireless technology such as 3G, 4G, 5G, or 6G) based on the content of the user information (e.g., user identification number). In some embodiments, the information management server 23, the information distribution server 25, and the second terminal device 21 may use (but are not limited to) a Message Queuing Telemetry Transport (MQTT) communication protocol, and the information management server 23 corresponds to a message publisher, the information distribution server 25 corresponds to a broker, and the second terminal device 21 corresponds to a message subscriber, thereby allowing them to send corresponding messages to each other. In addition, the second terminal device 21 can generate a request message based on the content of the control message, and the request message includes at least a second identification number and interaction information, and the second identification number may be the same as or different from the user identification number (but is not limited thereto), and may be a unique system number, an IP address, a Media Access Control address (MAC address), a Universally Unique Identifier (UUID for short), an IMEI or IMSI, etc. If the second terminal device 21 is a combination of a set-top box and an Internet TV, assuming that the power of the Internet TV is turned off, the set-top box will first send a power-on message after receiving the control message so that the Internet TV can be turned on. For example, the Internet TV can be switched to a power-on state by providing a power switch module in the set-top box, but is not limited thereto.In other embodiments of the present invention, according to different types of the second terminal device 21, it may have its own way of opening the display screen 211 after receiving the control message, or in some embodiments, the user needs to turn on the display screen 211 of the second terminal device 21 to complete the subsequent flow.

[0034] 1 again, the screen content management server 27 is connected to the second terminal device 21 via the Internet 20 and can receive a request message sent from the second terminal device 21. The screen content management server 27 can also obtain at least one default screen information from the screen database 28 based on the interaction information of the request message to generate a screen feedback message. The screen content management server 27 can then return a screen feedback message to the second terminal device 21 via the Internet 20 based on the second identification number so that the user can view a corresponding screen, for example, a screen of balance XXXXX yuan, on the display screen 211 of the second terminal device 21. From this, it can be seen that the visual feedback system 2 can obtain an input message (e.g., button information or voice command) from the first terminal device 11 via the telecommunications server 13 so as to provide a corresponding screen on the second terminal device 21. This allows users, especially the elderly, to operate in a more familiar way (e.g., pressing a button on a landline phone) and can not only hear the voice content but also see the screen directly, greatly improving usability. It is worth noting that the screen content management server 27 of the present invention is not limited to a single server, but can be a complex of multiple servers according to actual needs. For example, when the content the customer wants to know involves financial information, the screen content management server 27 can obtain relevant data from other corresponding servers (e.g., financial databases or servers that can obtain user balance information), and the screen database 28 can obtain corresponding default screen information according to the aforementioned relevant data content to generate a screen feedback message.

[0035] 1 again, in some embodiments, the screen database 28 can be located in the screen content management server 27. In some embodiments, the screen database 28 can be located in another server. In some embodiments, the screen content management server 27 or another server is also provided with a screen voice editing tool 281, which mainly edits the data in the screen database 28 and the corresponding interaction information required for each default screen information (e.g., button information of the physical button "1", voice information of "1" dictated by the user, etc.).

[0036] The interaction method involving the interactive system T of the invention is described below with reference to FIGS.

[0037] (S01) The first terminal device 11 (e.g., a landline phone) sends an input message to the telecommunications server 13. Before the first terminal device 11 establishes a call with the telecommunications server 13, the input message is a call request, and the first terminal device 11 and the telecommunications server 13 establish the call based on this. After the first terminal device 11 and the telecommunications server 13 establish the call, the input message can include input information such as button information or voice commands. (S02) The telecommunications server 13 sends a voice search request to the voice database 14 according to the content of the input message from the first terminal device 11. The voice search request may include the above-mentioned input information. (S03) The voice database 14 responds to the voice search request by returning corresponding pre-recorded voice segment information. If the input message is a call request, the pre-recorded voice segment information can be an opening greeting, such as, but not limited to, "Thank you for calling. For English service, please press '1'" (but not limited to). If the input message includes input information such as button information or voice command (for example, when the user selects the "Chinese service" option), the pre-recorded voice segment information is corresponding voice content, such as, but not limited to, "If you want to inquire about your balance, press '1'; if you want to inquire about your transaction details, press '2', etc." (S04) The telecommunications server 13 can convert the pre-recorded voice segment information obtained above to generate a corresponding voice feedback message, and send the voice feedback message to the first terminal device 11. Thus, the user can hear the voice content corresponding to the voice feedback message through the first terminal device 11, such as "Thank you for calling. For Chinese service, please press '1'. For English service, please press 2" or "If you want to inquire about your balance, please press '1'. For transaction details, please press '2', etc." (S05) The telecommunications server 13 can generate a corresponding processing message based on the input message, and send the processing message to the information management server 23. The processing message includes at least the first identification number and interaction information corresponding to the input information, and the processing order of step (S05) and step (S04) may be reversed or may be performed simultaneously. (S06) The information management server 23 can send a user search request to the user database 24 according to the contents of the processing message. The user search request can include at least the first identification number. (S07) In response to the user search request, the user database 24 returns a corresponding user data table. Each user data table is associated (bound) with the first identification number, and if the user database 24 cannot find a user data table corresponding to the first identification number, it responds with an error message to the information management server 23. (S08) The information management server 23 generates a corresponding forwarding message based on the processing message and the searched user data table contents, and transmits the forwarding message to the information distribution server 25. The forwarding message may include the user identification number and interaction information in the user data table. Note that when the information management server 23 receives an error message, it stops generating and transmitting the forwarding message, so that the user can thereafter only use services that can be provided and realized by the automated voice system 1. (S09) The information distribution server 25 generates a control message based on the contents of the forwarding message, and transmits the control message to the second terminal device 21 according to the user identification number in the forwarding message. The control message includes at least interaction information. (S10) The second terminal device 21 generates a request message based on the contents of the control message, and transmits the request message to the screen content management server 27. The request message includes at least the second identification number and the interaction information. (S11) The screen content management server 27 transmits a screen search request to the screen database 28 according to the contents of the request message. The screen search request may include interaction information. (S12) The screen database 28 responds to the screen search request and returns corresponding default screen information. Because the interaction information corresponds to the input information, the content of the default screen information can be matched to the pre-recorded voice segment information obtained in step (S04). (S13) The screen content management server 27 converts the default screen information acquired above into a corresponding screen feedback message and transmits it to the second terminal device 21 so that the second terminal device 21 can play the screen on the display screen 211.

[0038] In short, referring again to FIG. 1 and FIG. 2, according to the interactive system T and the interactive method of the present invention, the user can obtain the following experience results. First, when the user wants to inquire about his / her bank balance, he / she can use the landline (first terminal device 11) to dial the bank's phone number, and after the call is connected, the user hears the voice content such as "Thank you for calling. For Chinese service, please press '1'. For English service, please press 2" on the landline, and the user can also watch the screen shown in FIG. 3 on the Internet TV (second terminal device 21), and the image information W1, W2 of "Chinese service 1" and "English service 2" can be displayed on the screen. After that, the user can press "button 1" on the physical button (user input interface 111) or dictate "1" or "Chinese service 1". Also, the user can hear the voice content "If you want to inquire about your balance, press 1. If you want to inquire about your transaction details, press 2. If you want to transfer funds, press 3. If you want to use credit card services, press 4. If you want to use wealth management products, press 5. If you want to speak to a customer service representative, press 6." And the user can see the screen as shown in FIG. 4 through the Internet TV, and image information W3, W4, W5, W6, W7, W8 such as "Inquire about your balance 1," "Inquire about your transaction details 2," "Inquiry about your account details 3," "Inquiry about credit card services 4," "Wealth management products 5," and "Customer service representative 6" can be displayed on the screen. Finally, the user only needs to press the physical button "Button 1" or dictate "1" or "Inquire about your balance" to hear the voice content "Your account balance is XXXXXXX yuan" through the landline phone, and at the same time, the user can see the screen as shown in FIG. 5 on the Internet TV, and image information W9 "Your account balance is XXXXXXX yuan" can be displayed on the screen.

[0039] From this, it can be seen that the dialogue system T of the present invention can achieve the following effects. (1) Better user experience: Through the visual feedback mechanism, users can understand their choices and operation steps more clearly, reducing the risk of user confusion and frustration when operating traditional IVR systems. (2) Reducing user misunderstandings: For users who are hard of hearing or who have difficulty understanding voice commands, when the voice commands are not clear enough or are ambiguous, the visual feedback mechanism can provide additional on-screen assistance to help the user confirm their choice. (3) Improved user operation efficiency: Users can more quickly identify and select the options they need, reducing operational errors and shortening operation time. At the same time, telephone button operations provide a more familiar and convenient experience for seniors, allowing users to obtain the corresponding screen information by simply pressing a button. (4) Providing rich content: The visual feedback mechanism displays various contents such as photos, videos, and texts on the second terminal device 21, allowing the user to obtain information more clearly. (5) Reduced burden on customer service: When users can get their needs met more quickly and intuitively, they are less likely to need to be transferred to a human customer service representative, reducing the vendor's labor costs. (6) Sales and Marketing Opportunities: The visual feedback mechanism can also increase sales opportunities by displaying additional information such as product photos and promotional activities on the screen.

[0040] The above is merely a preferred embodiment of the present invention, and the scope of the rights claimed by the present invention is not limited thereto. Equivalent modifications that can be easily conceived by those skilled in the art based on the technical content disclosed in the present invention should fall within the protection scope of the present invention. [Explanation of symbols]

[0041] 1. Automated voice system 10 Public Switched Telephone Network 11 First terminal device 111 User Input Interface 13. Telecommunications Servers 132 Voice Recognition Module 14 Voice Database 141 Audio Editing Tools 2. Visual Feedback System 20. Internet 21 Second terminal device 211 Display screen 23 Information Management Server 24 User Database 25 Information distribution server 27 Screen Content Management Server 28 Screen Database 281 Screen Audio Editing Tool T Interactive Systems S01~S13 steps W1,W2,W3,W4,W5,W6,W7,W8,W9 Image information

Claims

1. a first terminal device that can be connected to a public switched telephone network and has a user input interface that can generate an input message including at least one piece of input information according to a user's operation; a telecommunications server that receives the input message sent from the first terminal device via the public switched telephone network, generates a processing message according to the input message, the processing message including a first identification number representing the first terminal device and interaction information corresponding to the input message, and can also connect to a voice database, and obtain at least one pre-recorded voice segment information from the voice database according to the content of the input message to generate a voice feedback message, and return the voice feedback message to the first terminal device via the public switched telephone network; an information management server connected to the telecommunications server to receive the processing message sent from the telecommunications server, and also capable of connecting to a user database, for searching corresponding user information from the user database according to the first identification number of the processing message, and generating a transfer message including at least the user information and the interaction information; an information distribution server connected to the information management server via the Internet to receive the forwarding message transmitted from the information management server, and capable of transmitting a control message including at least the interaction information to a corresponding second terminal device based on the user information of the forwarding message; the second terminal device capable of receiving the control message transmitted from the information distribution server via the Internet and generating a request message including at least a second identification number and the interaction information; a screen content management server that receives the request message sent from the second terminal device via the Internet and is connected to a screen database, and is capable of obtaining at least one default screen information from the screen database according to the interaction information of the request message to generate a screen feedback message, and returning the screen feedback message to the second terminal device via the Internet; An interactive system that combines an automated speech mechanism and a visual feedback mechanism, including:

2. 2. The interactive system combining automated speech and visual feedback mechanisms of claim 1, further comprising a voice editing tool for editing each of the pre-recorded voice segment information in the voice database and corresponding input information required for each of the pre-recorded voice segment information.

3. The interactive system combining the automated speech mechanism and the visual feedback mechanism of claim 1, further comprising a screen voice editing tool for editing each of the default screen information in the screen database and corresponding interaction information required for each of the default screen information.

4. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein the first terminal device and the second terminal device are the same device.

5. 2. The interactive system of claim 1, wherein the voice database is located within the telecommunications server.

6. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein the telecommunications server is installed with a speech recognition module.

7. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein said screen database is located within said screen content management server.

8. 1. An interaction method for an interactive system combining an automated voice mechanism and a visual feedback mechanism, comprising a first terminal device, a telecommunications server, an information management server, an information distribution server, a second terminal device, and a screen content management server, the method being used in an interactive system combining an automated voice mechanism and a visual feedback mechanism, the method causing the telecommunications server, the information management server, the information distribution server, and the screen content management server to execute the following steps after the first terminal device establishes a call with the telecommunications server: causing the telecommunications server to execute a step of receiving an input message sent from the first terminal device via a public switched telephone network, and generating a processing message including at least a first identification number and interaction information based on the input message; The telecommunications server executes the steps of: obtaining at least one corresponding pre-recorded voice segment information from a voice database according to the processing message, generating a voice feedback message, returning the voice feedback message to the first terminal device via the public switched telephone network, and the telecommunications server also sends the processing message to the information management server via the Internet; The information management server executes the steps of: after receiving the processing message sent from the telecommunication server, searching for corresponding user information from a user database according to the first identification number of the processing message, generating a transfer message, and sending the transfer message including at least the user information and the interaction information to the information distribution server via the Internet; the information distribution server receives the forwarding message transmitted from the information management server, generates a control message including at least the interaction information based on the user information of the forwarding message, transmits the control message to the corresponding second terminal device via the Internet, and the second terminal device generates a request message based on the control message; An interaction method for an interactive system combining an automatic voice mechanism and a visual feedback mechanism, the method comprising the steps of: receiving the request message sent from the second terminal device via the Internet; and causing the screen content management server to execute a step of obtaining at least one default screen information from a screen database based on the interaction information of the request message, generating a screen feedback message, and returning the screen feedback message to the second terminal device via the Internet.

9. 9. The method for an interactive system combining an automated voice mechanism and a visual feedback mechanism according to claim 8, wherein the first identification number is used to correspond to the first terminal device, the interaction information is used to correspond to input information in the processing message, and the input information is generated by the first terminal device after a user's operation.

10. The method for a dialogue system combining an automated voice mechanism and a visual feedback mechanism according to claim 8, wherein the first terminal device and the second terminal device are the same device.

11. The method for an interactive system combining an automated voice mechanism and a visual feedback mechanism according to claim 8, wherein the information management server stops generating and sending the forwarding message after it is unable to search for the corresponding user information from the user database based on the first identification number of the processing message.

Images