Dialogue system and dialogue method combining automated speech and visual feedback mechanisms

Abstract

To provide an interactive system combining an automated voice mechanism with a visual feedback mechanism, and an interactive method thereof.SOLUTION: A first terminal device 11 and a telecommunications server 13 mutually transmit messages through a public switched telephone network 10. The first terminal device 11 transmits an input message to the telecommunications server 13 and receives a voice feedback message from the telecommunications server 13. The telecommunications server 13 converts the input message to a processing message and transmits the processing message to an information management server 23. On the basis of the processing message, the information management server 23 transmits a forward message to an information distribution server 25 through Internet 20. On the basis of the forward message's content, the information distribution server 25 transmits a control message to a second terminal device 21. According to the forward message, the second terminal device 21 transmits a request message to a screen content management server 27 to obtain a returned screen feedback message.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an interactive system, and more particularly to an interactive system and an interactive method that simultaneously combine an automatic voice mechanism and a visual feedback mechanism, which allows a user to operate a first terminal device and listen to audio information transmitted from the first terminal device, and view screen information corresponding to the audio information from a second terminal device. [Background technology]

[0002] Interactive Voice Response (IVR), also known as an automated voice response system, allows users to interact with a dedicated telecommunications host on the vendor's side via telephone. After the user dials the dedicated telecommunications host and enters the IVR system, the IVR system plays a series of pre-recorded voice content, and the user responds by pressing the buttons on the telephone in accordance with the voice guidance they hear, to obtain the required information or complete a specific operation (such as credit card authorization).

[0003] IVR system-related technologies are used in many industries. For example, many banks and service providers use IVR systems to provide 24-hour customer service, allowing users to check account balances, pay bills, transfer funds, and perform other operations. Airlines, travel agencies, and medical institutions also use IVR systems to allow users to check flight information, book tickets, confirm appointment times, and more. This shows that IVR systems have many benefits, the most obvious of which is that they can significantly improve service efficiency and availability. Through automated processes, vendors can provide services anytime, anywhere without manual intervention, which not only reduces human resources but also ensures the accuracy and immediacy of information, shortens user waiting times, and improves customer service satisfaction.

[0004] However, while IVR systems have clear advantages in terms of providing diverse and efficient services, they still have some drawbacks, especially for certain user groups. In fact, IVR systems that rely solely on voice may not be very user-friendly for the elderly or those with hearing impairments. Pre-recorded voice messages can suffer from problems such as distortion or lack of clarity during transmission, making it difficult for such users to understand the content of the voice. Furthermore, following voice-guided operations can be inappropriate for the elderly, or the response may be slow, causing them to miss important instructions. These situations can be frustrating for users and affect the user experience. In addition, most IVR systems have a certain response time limit, and if the user does not make a selection or response within the time limit, the IVR system may automatically end the call. This will undoubtedly bring pressure to elderly users who need more time to process information or make decisions, making them feel inconvenienced and uncomfortable when using the IVR system.

[0005] Therefore, the industry and research institutions have begun to search for solutions, and some operators have used methods such as optimizing voice quality, adjusting playback speed, and adjusting response times according to user needs to improve user experience. However, for elderly people and those with hearing impairments, when voice guidance cannot fulfill their purpose, they usually choose to transfer to human customer service. Under the premise that IVR systems reduce operator human resources, such users may have to wait longer for human customer service, which may reduce user experience and satisfaction. Therefore, how to effectively solve the above problem is an important task of the present invention. Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the fact that conventional IVR systems can only provide voice guidance functions, which are obviously not friendly enough for some users, the inventor has made intensive research efforts and, after a long period of research and experimentation, developed an interactive system and its interactive method of the present invention that combines an automatic voice mechanism and a visual feedback mechanism to simultaneously provide multiple auditory and visual guidance, ensure that most users can clearly understand the guidance information, reduce misunderstandings, and help users make the right choice. [Means for solving the problem]

[0007] The object of the present invention is to provide an interactive system that combines an automated voice mechanism and a visual feedback mechanism, and that includes a first terminal device, a telecommunications server, an information management server, an information distribution server, a second terminal device, and a screen content management server. The first terminal device is connectable to the public switched telephone network and has a user input interface that can generate an input message including at least one piece of input information according to a user's operation. The telecommunications server can receive an input message sent from the first terminal device via the public switched telephone network, and generate a processing message based on the input message, the processing message including a first identification number representing the first terminal device and interaction information corresponding to the input information; the telecommunications server can also connect to a voice database, and based on the content of the input message, retrieve at least one pre-recorded voice segment information from the voice database to generate a voice feedback message, and return the voice feedback message to the first terminal device via the public switched telephone network. The information management server is connected to the telecommunications server to receive the processing message sent from the telecommunications server, and the information management server can also be connected to a user database, and can search for corresponding user information from the user database based on the first identification number of the processing message, and generate a forwarding message including at least the user information and interaction information. The information distribution server is connected to the information management server via the Internet to receive the forwarding message sent from the information management server, and can send a control message including at least interaction information to a corresponding second terminal device based on the user information in the forwarding message. The second terminal device can receive the control message transmitted from the information distribution server via the Internet and generate a request message including at least the second identification number and the interaction information. The screen content management server can receive a request message sent from the second terminal device via the Internet, and is connected to the screen database, and can obtain at least one default screen information from the screen database based on the interaction information of the request message, generate a screen feedback message, and return the screen feedback message to the second terminal device via the Internet. In this way, audio, visual and other guidance and hints can be provided simultaneously to users with special needs, allowing the users to quickly obtain and understand the aforementioned guidance and hint information, reducing the risk of users making mistakes and enabling the users to make the right choices to obtain the information they need.

[0008] Alternatively, the interactive system further comprises a voice editing tool for editing each pre-recorded voice segment information in the voice database and the corresponding input information required for each pre-recorded voice segment information.

[0009] Alternatively, the interactive system further comprises a screen audio editing tool for editing each default screen information in the screen database and the corresponding interaction information required for each default screen information.

[0010] Alternatively, the first terminal device and the second terminal device are the same device.

[0011] Alternatively, the voice database is located in a telecommunications server.

[0012] Alternatively, the telecommunications server is installed with a voice recognition module.

[0013] Alternatively, the screen database resides in a screen content management server.

[0014] Another object of the present invention is to provide a method for use in a dialogue system that combines automated speech and visual feedback mechanisms. The interactive system includes a first terminal device, a telecommunications server, an information management server, an information distribution server, a second terminal device, and a screen content management server. After the first terminal device establishes a call with the telecommunications server, the interaction method causes the telecommunications server, the information management server, the information distribution server, and the screen content management server to perform the following steps: The telecommunications server performs the steps of receiving an input message sent from the first terminal device via the public switched telephone network, and generating a processing message including at least the first identification number and interaction information based on the input message. The telecommunications server executes the steps of obtaining at least one corresponding pre-recorded voice segment information from a voice database based on the processing message, generating a voice feedback message, returning the voice feedback message to the first terminal device via the public switched telephone network, and the telecommunications server also transmitting the processing message to the information management server via the Internet. The information management server performs the steps of, after receiving the processing message sent from the telecommunications server, searching for corresponding user information from a user database based on the first identification number of the processing message, generating a forwarding message, and sending the forwarding message including at least the user information and interaction information to the information distribution server via the Internet. The information distribution server receives the forwarding message sent from the information management server, then generates a control message including at least interaction information based on the user information in the forwarding message, and sends the control message to a corresponding second terminal device via the Internet, and the second terminal device performs the steps of generating a request message based on the control message. The screen content management server executes the steps of receiving a request message sent from the second terminal device via the Internet, and then retrieving at least one piece of default screen information from the screen database based on interaction information of the request message, generating a screen feedback message, and returning the screen feedback message to the second terminal device via the Internet. In this way, the interactive method of the present invention can simultaneously provide auditory and visual information to enhance the user's understanding of the guidance and hint information, and some guidance instructions are more intuitive visually, while some guidance instructions are more concise in audio, thereby improving the overall efficiency of the user's use and operation.

[0015] Alternatively, the first identification number is used to correspond to the first terminal device, and the interaction information is used to correspond to input information in the processing message, and the input information is generated by the first terminal device after a user operation.

[0016] Alternatively, the information management server stops generating and sending the forwarding message after it is unable to search for corresponding user information from the user database based on the first identification number of the processing message.

[0017] In order to further appreciate and understand the objects, technical features and effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings by way of examples. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a schematic diagram of a system configuration of a dialogue system according to the present invention. [Figure 2] FIG. 2 is a sequence diagram of the interaction method of the present invention. [Figure 3] FIG. 4 is a schematic diagram of a screen of a second terminal device of the present invention. [Figure 4] 10 is another schematic diagram of a screen of the second terminal device of the present invention. FIG. [Figure 5] FIG. 10 is a further schematic diagram of a screen of the second terminal device of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] In order to make the objectives, technical contents and advantages of the present invention clearer, the disclosed embodiments of the present invention will be described in more detail below in conjunction with specific embodiments and with reference to the drawings. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification, and the present invention can be implemented or applied through other different specific embodiments. Various modifications and changes can be made to the various contents of this specification based on different perspectives and applications without departing from the technical spirit of the present invention. It should be noted in advance that the drawings of the present invention are merely schematic illustrations and are not drawn to actual scale. Unless otherwise clearly specified in the context or defined otherwise, the meanings of "a" and "the" in the present specification also include plural forms. Furthermore, the following embodiments will describe the technical contents related to the present invention in more detail, but the disclosures thereof are not intended to limit the scope of the present invention.

[0020] It should be understood that terms used herein generally have their ordinary meaning in the art, and in the event of any conflict, the definitions provided herein shall prevail. Because the same thing can be expressed in many ways, alternative words and synonyms may be used for terms discussed or illustrated herein, and there is no particular limitation as to whether they are described or illustrated herein. The use of one or more synonyms does not exclude other synonyms. Examples used anywhere in the description of the present invention, including the use of terms, are merely illustrative and in no way limit the scope and meaning of the present invention or terms. Similarly, the present invention is not limited to the various examples disclosed in the specification. Although terms such as "first," "second," and "third" may be used herein to describe various elements, each element should not be limited by the aforementioned terms. These terms are primarily used to distinguish one element from another, and do not impose any substantial limitations on any elements, nor do they limit the assembly or installation order of each element in actual applications. It should be noted that, as used herein, the term "or" includes any combination of one or more of the associated listed items, depending on the actual situation.

[0021] The present invention relates to an interactive system (hereinafter referred to as an interactive system) that combines an automated speech mechanism and a visual feedback mechanism, and an interactive method therefor. Referring to FIG. 1, the interactive system T is composed of an automated voice system 1 and a visual feedback system 2. The automated voice system 1 has an automated voice mechanism that provides voice feedback to the user, and the visual feedback system 2 has a visual feedback mechanism that provides visual feedback to the user. The automated voice system 1 may be an interactive voice response (IVR) system, in which a user dials a telephone number designated by a vendor to access a dedicated telecommunications host on the vendor's side, and the user enters specific keys to listen to pre-recorded voice content and obtain required information. Because the IVR system is a mature technology, only the necessary technical means related to the present invention will be described below. However, any system equipped with IVR-related technology can be considered as the automated voice system 1 of the present invention, and this point will be explained in advance.

[0022] 1 , the automated voice system 1 includes a first terminal device 11 and a telecommunications server 13. The first terminal device 11 can be a landline telephone or a mobile communication device (e.g., a mobile phone, a smartphone, etc.), and can directly or indirectly connect to a public switched telephone network (PSTN) 10 to exchange information (call data) with the telecommunications server 13. It is particularly noted here that the mobile communication device first establishes a wireless connection between the mobile communication device and a local telecommunications tower over a mobile communication network (e.g., 4G or 5G), and then exchanges information between the mobile communication device and the telecommunications server 13 over the public switched telephone network 10.

[0023] Referring again to FIG. 1 , the first terminal device 11 is provided with a user input interface 111 (e.g., physical buttons, a touch panel, a microphone, etc.), and the first terminal device 11 can generate an input message according to the result of a user operating the user input interface 111 and send the input message to the telecommunications server 13. When the first terminal device 11 has not established a call with the telecommunications server 13, the input message is a call request including a first identification number (e.g., a telephone number) and input information, or the input information may be blank. In some embodiments, the call request may only have the first identification number (e.g., a telephone number), which corresponds to the input information being blank. When the first terminal device 11 has established a call with the telecommunications server 13, the input message only includes input information (e.g., button information) depending on the specific aspect (e.g., a landline) of the first terminal device 11 and the technology used.

[0024] 1 , the first identification number is used to identify the caller to ensure that the first terminal device 11 establishes a correct call with the telecommunications server 13 and that the telecommunications server 13 sends call data (such as voice content) to the corresponding first terminal device 11. The first identification number may be a phone number, an International Mobile Subscriber Identity (IMSI), an International Mobile Equipment Identity (IMEI), or other unique identifier sufficient for the telecommunications server 13 to identify and maintain the call, and the input information may be button information, voice information, or blank. If the user input interface 111 is a physical button / virtual button, the input information is the button information of the physical button / virtual button pressed by the user, and if the user input interface 111 is a microphone, the input information is a voice command dictated by the user.

[0025] It is worth noting that, referring back to FIG. 1 , when a user uses the automated voice system 1, there are two scenarios: when the user calls a vendor (inbound) and when the vendor calls the user (outbound). Taking the user calling a vendor (inbound) as an example, the user first dials the vendor's telephone number via the first terminal device 11. This process corresponds to the first terminal device 11 sending an input message (call request) to the telecommunications server 13. The call request includes a first identification number (e.g., telephone number), and the telecommunications server 13 can establish a session based on the first identification number. The user then presses physical / virtual buttons or dictates voice commands according to the voice content heard each time. If the call continues, the first terminal device 11 only sends button information or voice commands to the telecommunications server 13, without repeatedly sending the telephone number, IMSI, or IMEI. However, to ensure the accuracy of the call between the first terminal device 11 and the telecommunications server 13, the call process between the first terminal device 11 and the telecommunications server 13 may include a unique identifier corresponding to the session. The unique identifier of the session is generated by the telecommunications server 13 and is an identifier for tracking the call; in other words, the first identification number is used to enable the telecommunications server 13 to accurately establish a call with the first terminal device 11. Since any information related to the above effects can be considered as the first identification number, the specific form of the first identification number may change during the call process (e.g., from a telephone number to a unique identifier of the session), but their meaning remains unchanged (both represent the source of the first terminal device 11).

[0026] Referring back to FIG. 1 , a usage scenario shows that when a vendor makes an outbound call to a user, the telecommunications server 13 obtains a first identification number (e.g., a telephone number) of the first terminal device 11 in advance. After establishing a call with the first terminal device 11, the telecommunications server 13 can receive an input message sent from the first terminal device 11. The input message includes at least input information, such as the user pressing physical / virtual buttons in succession according to the audio content heard each time or dictating a voice command. Note that in some embodiments, the user input interface 111 is not limited to a single device but may include multiple devices. For example, the user input interface 111 may include both a touch panel and a microphone, and generate a corresponding input message according to a user operation scenario. For example, during the same call, the user may first press a virtual button on the smartphone to generate button information, and then verbally speak a voice command into the microphone.

[0027] 1 , the telecommunications server 13 can receive an input message sent from the first terminal device 11 and read the content of the input message, and the telecommunications server 13 can obtain at least one pre-recorded voice segment information from the voice database 14 based on the content of the input message to generate a voice feedback message and return the voice feedback message to the first terminal device 11 via the public switched telephone network 10. For example, if the first terminal device 11 has not established a call with the telecommunications server 13 and the telecommunications server 13 determines that the input information is blank, it can obtain pre-recorded voice segment information from the voice database 14, which may be "Thank you for calling. For Chinese service, please press '1'. For English service, please press 2", and then the telecommunications server 13 sends a voice feedback message including the pre-recorded voice segment information to the first terminal device 11 so that the user can hear the voice content.

[0028] Referring back to FIG. 1 , during a call, if the user hears voice content such as "If you want to inquire about your balance, press '1'; if you want to inquire about your transaction details, press '2'", then if the user presses the '1' button on the physical / virtual button, or verbally speaks a voice command such as '1' or 'inquire about your balance', the first terminal device 11 can send an input message containing the above-mentioned input information to the telecommunications server 13. The telecommunications server 13 retrieves relevant data from a corresponding database or other server (e.g., a financial database or server that can obtain the user's balance information) based on the input information (e.g., button information corresponding to '1' or a voice command), and then the voice database 14 retrieves corresponding pre-recorded voice segment information according to the content of the relevant data to generate a voice feedback message, which is then sent to the first terminal device 11, allowing the user to hear the voice content of the voice feedback message (e.g., balance XXXXX yuan) through the first terminal device 11.

[0029] 1 , in some embodiments, the voice database 14 may be located on the telecommunications server 13. In some embodiments, the voice database 14 may be located on another server. In some embodiments, the telecommunications server 13 or other device may also include a voice editing tool 141 for editing the data in the voice database 14 and the corresponding input information required for each pre-recorded voice segment (e.g., button information for a physical "1" button, audio information for a "1" dictated by a user, etc.). In some embodiments, a voice recognition module 132 may be installed on the telecommunications server 13 to capture user dictation and convert it into text, including, but not limited to,

[0030] 1 , the telecommunications server 13 generates a corresponding processing message according to the content of the input message and sends the processing message to the visual feedback system 2 via the Internet 20. The processing message includes at least a first identification number (e.g., a phone number) and interaction information corresponding to the input information. The content of the interaction information may be the same as the input information or may be sufficient to represent the content of the input information. For example, the interaction information may be the button information of the physical button "1" or the voice command of "1" dictated by the user. Alternatively, the telecommunications server 13 may receive the voice command of "1" dictated by the user and convert it into the information content of "balance inquiry" or "button 1", etc. In other words, the telecommunications server 13 may use the input information as the interaction information or first process the input information to convert it into the corresponding interaction information.

[0031] Referring again to FIG. 1 , the visual feedback system 2 includes a second terminal device 21, an information management server 23, an information distribution server 25, and a screen content management server 27. The second terminal device 21 is a device having a display screen, such as an Internet TV or a smartphone. In some embodiments, the second terminal device 21 may be a combination of multiple devices, such as a set-top box and a television. In some embodiments, the second terminal device 21 and the first terminal device 11 may be the same device (e.g., a smartphone), primarily because a smartphone has dialing and display functions. For example, its touch panel can be used as a display screen 211, virtual buttons can be displayed as a user input interface 111 (the smartphone as the automated voice system 1 in FIG. 1 ), and the smartphone also has the ability to connect to the public switched telephone network 10 and the Internet 20. Therefore, when acting as the automated voice system 1, the smartphone can transmit information directly or indirectly via the public switched telephone network 10. When acting as the visual feedback system 2, the smartphone transmits information via the Internet 20.

[0032] 1 , the information management server 23 is connected to the telecommunications server 13 to receive processing messages sent from the telecommunications server 13, and is provided with a telecommunications access interface (TAI) for enabling the information management server 23 and the telecommunications server 13 to establish communication and transfer data between each other. The information management server 23 can also be connected to a user database 24, which stores a plurality of user data tables, and the user data tables store user information such as at least a first identification number and a user identification number, and the user identification number is associated (bound) with at least one first identification number and can be associated with a second terminal device 21 so that the information management server 23 can identify the corresponding second terminal device 21 based on the user identification number and correctly transmit a message to the second terminal device 21. According to actual needs, the user identification number may be a unique system number (Embedded OS ID) such as Android® ID, Apple® ID, SoC unique ID (Linux®), etc., or may be an IP address, a Media Access Control address (MAC address), a Universally Unique Identifier (UUID), an IMEI, or an IMSI, etc. In other words, the user identification number corresponds to the second terminal device 21 so that the message from the information management server 23 can be correctly transmitted to the second terminal device 21. In addition, the information management server 23 can generate a forwarding message including at least some user information (e.g., user identification number) and interaction information (e.g., button information of physical button "1") based on the processing message and the contents of the user data table.

[0033] 1 , the information distribution server 25 may be connected to the information management server 23 via the Internet 20 to receive a forwarding message sent from the information management server 23 and generate a control message including at least interaction information. Furthermore, the information distribution server 25 may transmit a control message to a corresponding second terminal device 21 via wired or wireless technology (e.g., wide-area wireless technology such as 3G, 4G, 5G, or 6G) based on the content of user information (e.g., user identification number). In some embodiments, the information management server 23, the information distribution server 25, and the second terminal device 21 may use (but are not limited to) the Message Queuing Telemetry Transport (MQTT) communication protocol, in which the information management server 23 corresponds to a message publisher, the information distribution server 25 corresponds to a broker, and the second terminal device 21 corresponds to a message subscriber, thereby transmitting corresponding messages to each other. Furthermore, the second terminal device 21 can generate a request message based on the content of the control message, where the request message includes at least a second identification number and interaction information. The second identification number may be the same as or different from (but is not limited to) the user identification number, a unique system number, an IP address, a MAC address (Media Access Control address), a universally unique identifier (UUID), an IMEI, an IMSI, etc. If the second terminal device 21 is a combination of a set-top box and an Internet TV, assuming that the Internet TV is turned off, after receiving the control message, the set-top box can first send a power-on message to turn on the Internet TV. For example, but not limited to, a power switch module can be provided in the set-top box to switch the Internet TV to a power-on state.In other embodiments of the present invention, depending on different types of the second terminal device 21, it may have its own way of opening the display screen 211 after receiving the control message, or in some embodiments, the user needs to turn on the display screen 211 of the second terminal device 21 to complete the subsequent process.

[0034] Referring back to FIG. 1 , the screen content management server 27 is connected to the second terminal device 21 via the Internet 20 and can receive a request message sent from the second terminal device 21. The screen content management server 27 can also generate a screen feedback message by retrieving at least one default screen from the screen database 28 based on the interaction information in the request message. The screen content management server 27 can then return a screen feedback message to the second terminal device 21 via the Internet 20 based on the second identification number so that the user can view a corresponding screen, such as a balance XXXXX, on the display screen 211 of the second terminal device 21. This indicates that the visual feedback system 2 can receive an input message (e.g., button information or a voice command) from the first terminal device 11 via the telecommunications server 13 to provide a corresponding screen on the second terminal device 21. This significantly improves usability for users, especially elderly users, because they can operate the system using a more familiar operation method (e.g., pressing buttons on a landline phone) and can not only hear the audio content but also see the screen directly. It is worth noting that the screen content management server 27 of the present invention is not limited to a single server, but can be a complex of multiple servers according to actual needs. For example, if the content the customer wants to know involves financial information, the screen content management server 27 can obtain related data from other corresponding servers (e.g., financial databases or servers that can obtain user balance information), and then, according to the content of the related data, the screen database 28 can obtain corresponding default screen information to generate a screen feedback message.

[0035] 1 , in some embodiments, the screen database 28 can be located in the screen content management server 27. In some embodiments, the screen database 28 can be located in another server. In some embodiments, the screen content management server 27 or another server also includes a screen audio editing tool 281, which is mainly used to edit the data in the screen database 28 and the corresponding interaction information required for each default screen information (e.g., button information for the physical button "1", audio information for "1" dictated by the user, etc.).

[0036] The interaction method involving the interactive system T of the present invention is described below with reference to FIGS.

[0037] (S01) The first terminal device 11 (e.g., a landline phone) sends an input message to the telecommunications server 13. Before the first terminal device 11 establishes a call with the telecommunications server 13, the input message is a call request, and the first terminal device 11 and the telecommunications server 13 establish the call based on this. After the first terminal device 11 and the telecommunications server 13 establish the call, the input message can include input information such as button information or voice commands. (S02) The telecommunications server 13 sends a voice search request to the voice database 14 according to the content of the input message from the first terminal device 11. The voice search request may include the above-mentioned input information. (S03) The voice database 14 responds to the voice search request and returns corresponding pre-recorded voice segment information. If the input message is a call request, the pre-recorded voice segment information may be an opening greeting, such as, but not limited to, "Thank you for calling. For Chinese service, please press '1'. For English service, please press 2." If the input message includes input information such as button information or voice commands (e.g., when the user selects the "Chinese service" option), the pre-recorded voice segment information may be, but is not limited to, corresponding voice content, such as, "To inquire about your balance, press '1'; for inquiring about transaction details, press '2'." (S04) The telecommunications server 13 can convert the pre-recorded voice segment information obtained above to generate a corresponding voice feedback message, and send the voice feedback message to the first terminal device 11. Thus, the user can hear the voice content corresponding to the voice feedback message through the first terminal device 11, such as "Thank you for calling. For Chinese service, please press '1'. For English service, please press 2" or "If you want to inquire about your balance, please press '1'. For transaction details, please press '2', etc." (S05) The telecommunications server 13 can generate a corresponding processing message based on the input message and send the processing message to the information management server 23. The processing message includes at least the first identification number and interaction information corresponding to the input information, and the processing order of step (S05) and step (S04) can be reversed or can be performed simultaneously. (S06) The information management server 23 can send a user search request to the user database 24 according to the content of the processing message. The user search request can include at least the first identification number. (S07) In response to the user search request, the user database 24 returns a corresponding user data table. Each user data table is associated (bound) with the first identification number, and if the user database 24 cannot find a user data table corresponding to the first identification number, it returns an error message to the information management server 23. (S08) The information management server 23 generates a corresponding forwarding message based on the processing message and the searched user data table contents, and sends the forwarding message to the information distribution server 25. The forwarding message may include the user identification number and interaction information in the user data table. Note that when the information management server 23 receives an error message, it stops generating and sending the forwarding message, so that the user can thereafter only use services that can be provided and realized by the automated voice system 1. (S09) The information distribution server 25 generates a control message based on the contents of the forwarding message, and transmits the control message to the second terminal device 21 according to the user identification number in the forwarding message. The control message includes at least interaction information. (S10) The second terminal device 21 generates a request message based on the content of the control message, and transmits the request message to the screen content management server 27. The request message includes at least the second identification number and interaction information. (S11) The screen content management server 27 sends a screen search request to the screen database 28 according to the contents of the request message. The screen search request may include interaction information. (S12) The screen database 28 responds to the screen search request and returns corresponding default screen information. Because the interaction information corresponds to the input information, the content of the default screen information can be matched with the pre-recorded voice segment information obtained in step (S04). (S13) The screen content management server 27 converts the default screen information acquired above into a corresponding screen feedback message and sends it to the second terminal device 21 so that the second terminal device 21 can play the screen on the display screen 211.

[0038] In summary, referring again to Figures 1 and 2, the interactive system T and the interactive method of the present invention can provide the following user experience: First, when a user wants to inquire about his or her bank balance, he or she can use a landline phone (first terminal device 11) to dial the bank's phone number. After the call is connected, the user hears a voice message on the landline such as, "Thank you for calling. For Chinese service, please press '1'. For English service, please press 2." The user can also view the screen shown in Figure 3 on the Internet TV (second terminal device 21), displaying image information W1 and W2 for "Chinese service 1" and "English service 2" on the screen. The user can then press "button 1" on the physical button (user input interface 111) or dictate "1" or "Chinese service 1." The user can also hear audio content via the landline phone, such as "Press 1 for balance inquiry, press 2 for transaction statement inquiry, press 3 for direct debit, press 4 for credit card service, press 5 for wealth management product, or press 6 for customer service representative." The user can also view a screen as shown in FIG. 4 on the Internet TV, which can display image information W3, W4, W5, W6, W7, and W8, such as "Balance Inquiry 1," "Transaction Statement Inquiry 2," "Direct Debit Service 3," "Credit Card Service 4," "Wealth Management Product 5," and "Customer Service Representative 6." Finally, the user simply presses "Button 1" on the physical button or dictates "1" or "Balance Inquiry," and hears audio content via the landline phone, such as "Your account balance is XXXXXXXXX." At the same time, the user can view a screen as shown in FIG. 5 on the Internet TV, which can display image information W9, such as "Your account balance is XXXXXXXXX."

[0039] From this, it can be seen that the interactive system T of the present invention can achieve the following effects. (1) Better user experience: Through visual feedback mechanisms, users can more clearly understand their choices and operation steps, reducing the risk of user confusion and frustration when operating traditional IVR systems. (2) Reducing user misunderstandings: For users who are hard of hearing or have difficulty understanding voice commands, if the voice command is not clear enough or is ambiguous, the visual feedback mechanism can provide additional on-screen assistance to help the user confirm their selection. (3) Improved user operation efficiency: Users can more quickly identify and select the required options, reducing operational errors and shortening operation time. At the same time, for the elderly, telephone button operation provides a more familiar and convenient operation feel, allowing users to obtain the corresponding screen information simply by pressing a button. (4) Providing rich content: The visual feedback mechanism displays various content such as photos, videos, and text on the second terminal device 21, allowing the user to obtain information more clearly. (5) Reduced burden on customer service: When users can meet their needs more quickly and intuitively, they are less likely to need to be transferred to a human customer service representative, reducing labor costs for vendors. (6) Sales and Marketing Opportunities: The visual feedback mechanism can also increase sales opportunities by displaying additional information such as product photos and promotional activities on the screen.

[0040] The above is merely a preferred embodiment of the present invention, and the scope of the rights claimed by the present invention is not limited thereto. Equivalent modifications that can be easily conceived by a person skilled in the art based on the technical content disclosed in the present invention should fall within the protection scope of the present invention. [Explanation of symbols]

[0041] 1. Automated voice system 10 Public Switched Telephone Network 11 First terminal device 111 User Input Interface 13 Telecommunications Servers 132 Voice Recognition Module 14 Voice Database 141 Audio Editing Tools 2. Visual Feedback System 20. Internet 21 Second terminal device 211 Display screen 23 Information Management Server 24 User Database 25 Information distribution server 27 Screen Content Management Server 28 Screen Database 281 Screen Audio Editing Tool T Interactive System S01~S13 steps W1,W2,W3,W4,W5,W6,W7,W8,W9 Image information

Claims

1. a first terminal device connectable to a public switched telephone network and provided with a user input interface capable of generating an input message including at least one piece of input information in accordance with a user's operation; a telecommunications server that receives the input message sent from the first terminal device via the public switched telephone network, generates a processing message based on the input message, the processing message including a first identification number representing the first terminal device and interaction information corresponding to the input information, and can also connect to a voice database, obtain at least one pre-recorded voice segment information from the voice database based on the content of the input message, and generate a voice feedback message, and return the voice feedback message to the first terminal device via the public switched telephone network; an information management server connected to the telecommunications server to receive the processing message sent from the telecommunications server, and also capable of connecting to a user database, for searching corresponding user information from the user database according to the first identification number of the processing message, and generating a transfer message including at least the user information and the interaction information; an information distribution server connected to the information management server via the Internet to receive the transfer message transmitted from the information management server, and capable of transmitting a control message including at least the interaction information to a corresponding second terminal device based on the user information in the transfer message; the second terminal device is capable of receiving the control message transmitted from the information distribution server via the Internet and generating a request message including at least a second identification number and the interaction information; a screen content management server that receives the request message sent from the second terminal device via the Internet and is connected to a screen database, and is capable of obtaining at least one default screen information from the screen database according to the interaction information of the request message to generate a screen feedback message, and returning the screen feedback message to the second terminal device via the Internet; An interactive system that combines an automated speech mechanism and a visual feedback mechanism, including:

2. 2. The interactive system combining automated speech and visual feedback mechanisms of claim 1, further comprising a speech editing tool for editing each of the pre-recorded speech segment information in the speech database and corresponding input information required for each of the pre-recorded speech segment information.

3. 2. The interactive system combining the automated voice mechanism and the visual feedback mechanism of claim 1, further comprising a screen voice editing tool for editing each of the default screen information in the screen database and corresponding interaction information required for each of the default screen information.

4. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein the first terminal device and the second terminal device are the same device.

5. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein said speech database is located within said telecommunications server.

6. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein the telecommunications server is installed with a speech recognition module.

7. 2. The interactive system combining automated speech and visual feedback mechanisms according to claim 1, wherein the screen database is located within the screen content management server.

8. 1. An interaction method for an interactive system that combines an automated voice mechanism and a visual feedback mechanism, the method comprising: a first terminal device, a telecommunications server, an information management server, an information distribution server, a second terminal device, and a screen content management server; the method being used in an interactive system that combines an automated voice mechanism and a visual feedback mechanism, the method causing the telecommunications server, the information management server, the information distribution server, and the screen content management server to perform the following steps after the first terminal device establishes a call with the telecommunications server: causing the telecommunications server to perform a step of receiving an input message sent from the first terminal device via a public switched telephone network, and generating a processing message including at least a first identification number and interaction information based on the input message; The telecommunications server executes the steps of: obtaining at least one corresponding pre-recorded voice segment information from a voice database according to the processing message, generating a voice feedback message, returning the voice feedback message to the first terminal device via the public switched telephone network, and the telecommunications server also transmitting the processing message to the information management server via the Internet; the information management server executes the steps of, after receiving the processing message sent from the telecommunications server, searching for corresponding user information from a user database based on the first identification number of the processing message, generating a transfer message, and sending the transfer message including at least the user information and the interaction information to the information distribution server via the Internet; the information distribution server receives the forwarding message transmitted from the information management server, generates a control message including at least the interaction information based on the user information in the forwarding message, transmits the control message to the corresponding second terminal device via the Internet, and the second terminal device generates a request message based on the control message; An interactive method for an interactive system combining an automatic voice mechanism and a visual feedback mechanism, the method comprising: causing the screen content management server to perform the steps of: after receiving the request message sent from the second terminal device via the Internet, retrieving at least one default screen information from a screen database based on the interaction information of the request message, generating a screen feedback message, and returning the screen feedback message to the second terminal device via the Internet.

9. 9. The method for an interactive system combining an automated voice mechanism and a visual feedback mechanism according to claim 8, wherein the first identification number is used to correspond to the first terminal device, the interaction information is used to correspond to input information in the processing message, and the input information is generated by the first terminal device after a user's operation.

10. 9. The method for a dialogue system combining an automated voice mechanism and a visual feedback mechanism according to claim 8, wherein the first terminal device and the second terminal device are the same device.

11. 9. The interactive method for an interactive system combining an automated voice mechanism and a visual feedback mechanism according to claim 8, wherein the information management server stops generating and sending the forwarding message after it is no longer able to search for the corresponding user information from the user database based on the first identification number of the processing message.

Images