A voice interaction method and system based on cloud computing

Abstract

The present invention discloses a voice interaction method and system based on cloud computing. The system comprises: a local function layer and a cloud computing layer; the local function layer comprises voice interaction terminals and processing servers connected to each other; the cloud computing layer comprises an API gateway, a load balancing unit, an interaction server, a data access unit, a database server connected in sequence, and a redundant backup unit connected to the interaction server. The present invention implements efficient voice interaction at low cost by performing streamlined interaction calculations through a cloud computing platform.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a voice interaction method and system based on cloud computing. Background Art

[0002] With the development of information technology, human-computer interaction based on natural scenes has become a research target in the field of research. Most of the human-computer voice interaction systems in the prior art are based on voice interaction requests, receiving audio data collected by the client included in the conversation, and semantically recognizing the voice information based on machine learning, searching in the instruction hierarchy diagram to generate interaction instructions. However, this method requires a lot of local computing power and a large amount of calculation, which reduces processing efficiency and increases costs. How to overcome the above defects has become a problem to be solved in this field. Summary of the invention

[0003] The present invention provides a voice interaction system based on cloud computing, the system comprising: a local function layer, a cloud computing layer;

[0004] The local function layer includes interconnected voice interaction terminals and processing servers;

[0005] The cloud computing layer includes an API gateway, a load balancing unit, an interactive server, a data access unit, a database server, and a redundant backup unit connected to the interactive server.

[0006] The voice interaction terminal is used to collect the user's voice and feed back the voice interaction information to the user;

[0007] The processing server is used for performing speech conversion and speech synthesis;

[0008] The API gateway is used to implement data transmission between the cloud computing layer and the local function layer;

[0009] The load balancing unit is used to achieve system load balancing at the cloud computing layer;

[0010] The interactive server is used for language recognition and content acquisition;

[0011] The redundant backup unit is used to implement disaster recovery redundant backup;

[0012] The data access unit is used to connect to the database server;

[0013] The database server is used to store content data and word data.

[0014] The present invention also relates to a cloud computing-based voice interaction method, which is implemented by the cloud computing-based voice interaction system, and comprises the following steps:

[0015] S1. The voice collection device collects the sound of the user's speech, converts it into audio information, and sends it to the output device;

[0016] S2. The output device sends the audio information to the processing server;

[0017] S3. The speech conversion module of the processing server is used to receive the audio information, convert it into sentence information, and send it to the interactive server;

[0018] S4. The language recognition module of the interactive server performs word segmentation on the sentence information, performs weight analysis on the word segmentation results, obtains word data corresponding to high-weight word segmentation from the word database of the database server, and sends it to the content acquisition module of the interactive server;

[0019] S5. The content acquisition module obtains the interactive information corresponding to the sentence data from the content database according to the received word data, and sends it to the processing server;

[0020] S6. The speech synthesis module of the processing server synthesizes the interactive information from the interactive server into audio data and sends it to the voice interaction terminal;

[0021] S7. The input device of the voice interaction terminal receives the audio data from the processing server and sends it to the voice playback device;

[0022] S8. The voice playing device converts the audio data into sound and plays it to the user.

[0023] The present invention also relates to a computer program product, which includes a computer program, and the computer program is executed by a processor to execute the above-mentioned cloud computing-based voice interaction method.

[0024] The present invention also relates to a computer-readable storage medium, which is used to store a computer program. The computer program is executed by a processor to execute the above-mentioned cloud computing-based voice interaction method.

[0025] The technical solution of the present invention performs streamlined interactive calculations through a cloud computing platform, thereby achieving high-efficiency voice interaction at low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a structural block diagram of a voice interaction system based on cloud computing in the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention. It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the present invention.

[0028] Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those generally understood by those of ordinary skill in the art to which the present invention belongs. It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0029] Embodiment 1 of the present invention relates to a voice interaction system based on cloud computing, the system comprising: a local function layer and a cloud computing layer. The local function layer comprises a voice interaction terminal and a processing server connected to each other. The cloud computing layer comprises an API gateway, a load balancing unit, an interaction server, a data access unit, a database server connected in sequence, and a redundant backup unit connected to the interaction server.

[0030] The voice interaction terminal is used to collect the user's voice and feed back voice interaction information to the user.

[0031] The processing server is used for performing speech conversion and speech synthesis.

[0032] The API gateway is used to realize data transmission between the cloud computing layer and the local function layer.

[0033] The load balancing unit is used to achieve system load balancing at the cloud computing layer.

[0034] The interactive server is used for language recognition and content acquisition.

[0035] The redundant backup unit is used to implement disaster recovery redundant backup.

[0036] The data access unit is used to connect to the database server.

[0037] The database server is used to store content data and word data.

[0038] The voice interaction terminal includes a voice collection device, a voice playback device, an input device, and an output device.

[0039] The voice collection device is used to collect the sound of the user speaking, convert it into audio information, and send it to the output device. The output device is used to send the audio information to the processing server. The input device receives the audio data from the processing server and sends it to the voice playback device. The voice playback device is used to convert the audio data into sound and play it to the user.

[0040] The processing server includes a speech conversion module and a speech synthesis module.

[0041] The voice conversion module is used to receive audio information from the voice interaction terminal, convert it into sentence information, and send it to the interaction server. The voice synthesis module is used to synthesize the interaction information from the interaction server into audio data and send it to the voice interaction terminal.

[0042] The speech conversion module includes a feature extraction unit, a feature matching unit, and a sentence generation unit.

[0043] The feature extraction unit is used to extract identification feature information from audio information, and specifically adopts Mel Frequency Cepstral Coefficient (MFCC) feature extraction algorithm.

[0044] The feature matching unit is used to obtain vocabulary information according to the recognition feature information matching, and specifically adopts Gaussian mixture model-hidden Markov model GMM-HMM for matching.

[0045] The sentence generating unit is used to generate sentence information based on the vocabulary information.

[0046] The interactive server includes a language recognition module and a content acquisition module.

[0047] The language recognition module is used to segment the sentence information, perform weight analysis on the segmentation results, and obtain word data corresponding to high-weight segmentations from the word database of the database server, and send it to the content acquisition module.

[0048] The language recognition module obtains m pieces of sentence information and converts the i-th sentence information S i Perform word segmentation operation and get

[0049] S i =(t i1 ,t i2 ,…,t in )

[0050] where t in Indicates the nth keyword in the i-th sentence information.

[0051] Calculate the word segmentation weight w ik , specifically:

[0052]

[0053] Among them, m represents the number of sentence information, n represents the number of keywords in each sentence information, and w ik represents the weight of the kth keyword in the i-th sentence information, g(t ik ) represents the number of times the kth keyword in the ith sentence information appears in the mth sentence information, e(t ik ) represents the number of times the kth keyword in the i-th sentence information appears in the i-th sentence information.

[0054] The word data corresponding to the first h keywords with the highest weights are obtained from the word database of the database server, and sent to the content acquisition module, where h=n / 2.

[0055] The word data is GB18030 encoded data.

[0056] The content acquisition module acquires the interaction information corresponding to the sentence data from the content database according to the received word data, and sends the interaction information to the processing server.

[0057] Specifically: vectorize the received word data to obtain the vector matrix J,

[0058] J=[j1…j h ]

[0059] Among them, j h is the vector of the hth word data;

[0060] The content acquisition module acquires the rth piece of interaction information d from the content database r The corpus vector matrix U,

[0061] U=[u r1 …u rj ]

[0062] Among them, u rj is the vector of the jth word in the rth interactive information;

[0063] The content acquisition module calculates the probability P(d r ),

[0064]

[0065] Among them, z is the number of interactive information, u rc is the rth interactive information d r The c-th word vector in N(u rc,d r ) is the rth interactive information d r The cth word vector in the rth interactive information d r The number of occurrences, u c is the vector of the cth word in the preset condition information, N(u c ,d r ) is the vector of the cth word in the preset condition information in the rth talent data information d r Number of occurrences;

[0066] The content acquisition module determines the conditional probability P(d r ) is greater than the preset probability value, the rth interactive information d r Sent to the processing server.

[0067] The database server includes a content database and a word database.

[0068] The content database stores interaction information, such as answers to questions, confirmation information to control instructions, and the like.

[0069] The word database stores common Chinese dictionary data.

[0070] Embodiment 2 of the present invention relates to a voice interaction method based on cloud computing, which is implemented by the system of embodiment 1 and includes the following steps:

[0071] S1. The voice collection device collects the sound of the user's speech, converts it into audio information, and sends it to the output device.

[0072] S2. The output device sends the audio information to the processing server.

[0073] S3. The speech conversion module of the processing server is used to receive the audio information, convert it into sentence information, and send it to the interactive server.

[0074] Specifically include:

[0075] S31. The feature extraction unit of the speech conversion module extracts identification feature information from the audio information.

[0076] Specifically, the Mel-frequency cepstral coefficient MFCC feature extraction algorithm is adopted.

[0077] S32. The feature matching unit of the speech conversion module obtains vocabulary information based on the recognition feature information.

[0078] Specifically, Gaussian mixture model-hidden Markov model GMM-HMM is used for matching.

[0079] S33. The sentence generation unit of the speech conversion module generates sentence information based on the vocabulary information.

[0080] S4. The language recognition module of the interactive server segments the sentence information, performs weight analysis on the segmentation results, obtains word data corresponding to high-weight segmentations from the word database of the database server, and sends it to the content acquisition module of the interactive server.

[0081] Specifically include:

[0082] S41. The language recognition module obtains m pieces of sentence information and converts the i-th sentence information S i Perform word segmentation operation and get

[0083] S i =(t i1 ,t i2 ,…,t in )

[0084] where t in Indicates the nth keyword in the i-th sentence information.

[0085] S42. Calculate word segmentation weight w ik , specifically:

[0086]

[0087] Among them, m represents the number of sentence information, n represents the number of keywords in each sentence information, and w ik represents the weight of the kth keyword in the i-th sentence information, g(t ik ) represents the number of times the kth keyword in the ith sentence information appears in the mth sentence information, e(t ik ) represents the number of times the kth keyword in the i-th sentence information appears in the i-th sentence information.

[0088] S43. Obtain word data corresponding to the first h keywords with the highest weights from the word database of the database server, and send them to the content acquisition module, where h=n / 2.

[0089] The word data is GB18030 encoded data.

[0090] S5. The content acquisition module acquires the interaction information corresponding to the sentence data from the content database according to the received word data, and sends it to the processing server.

[0091] Specifically include:

[0092] S51. The content acquisition module vectorizes the received word data to obtain a vector matrix J,

[0093] J=[j1…j h ]

[0094] Among them, j h is the vector of the hth word data.

[0095] S52. The content acquisition module acquires the rth interactive information d from the content database. r The corpus vector matrix U,

[0096] U=[u r1 …u rj ]

[0097] Among them, u rj is the vector of the jth word in the rth interactive information.

[0098] S53. The content acquisition module calculates the probability P(d r ),

[0099]

[0100] Among them, z is the number of interactive information, u rc is the rth interactive information d r The c-th word vector in N(u rc ,d r ) is the rth interactive information d r The cth word vector in the rth interactive information d r The number of occurrences, u c is the vector of the cth word in the preset condition information, N(u c ,d r ) is the vector of the cth word in the preset condition information in the rth talent data information d r Number of occurrences.

[0101] S54. The content acquisition module determines the conditional probability P(d r ) is greater than the preset probability value, the rth interactive information d r Sent to the processing server.

[0102] S6. The speech synthesis module of the processing server synthesizes the interaction information from the interaction server into audio data, and sends it to the voice interaction terminal.

[0103] S7. The input device of the voice interaction terminal receives the audio data from the processing server and sends it to the voice playback device.

[0104] S8. The voice playing device converts the audio data into sound and plays it to the user.

[0105] Embodiment 3 of the present invention relates to a computer program product, which includes a computer program. The computer program is executed by a processor to execute a cloud computing-based voice interaction method of embodiment 2.

[0106] Embodiment 4 of the present invention relates to a computer-readable storage medium, which is used to store a computer program. The computer program is executed by a processor to execute a cloud computing-based voice interaction method of embodiment 2.

[0107] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A voice interaction system based on cloud computing, characterized in that: The system comprises: a local function layer and a cloud computing layer; The local function layer includes interconnected voice interaction terminals and processing servers; The cloud computing layer includes an API gateway, a load balancing unit, an interactive server, a data access unit, a database server, and a redundant backup unit connected to the interactive server. The voice interaction terminal is used to collect the user's voice and feed back the voice interaction information to the user; The processing server is used for performing speech conversion and speech synthesis; The API gateway is used to implement data transmission between the cloud computing layer and the local function layer; The load balancing unit is used to achieve system load balancing at the cloud computing layer; The interactive server is used for language recognition and content acquisition; The redundant backup unit is used to implement disaster recovery redundant backup; The data access unit is used to connect to the database server; The database server is used to store content data and word data; The voice interaction terminal includes a voice acquisition device, a voice playback device, an input device, and an output device; The voice collection device is used to collect the sound of the user speaking, convert it into audio information, and send it to the output device; The output device is used to send the audio information to the processing server; The input device receives the audio data from the processing server and sends it to the voice playback device; The voice playing device is used to convert the audio data into sound and play it to the user; The processing server includes a speech conversion module and a speech synthesis module; The voice conversion module is used to receive audio information from the voice interaction terminal, convert it into sentence information, and send it to the interaction server; The speech synthesis module is used to synthesize the interaction information from the interaction server into audio data and send it to the voice interaction terminal; The interactive server includes a language recognition module and a content acquisition module; The language recognition module is used to segment the sentence information, perform weight analysis on the segmentation results, and obtain word data corresponding to high-weight segmentations from the word database of the database server, and send it to the content acquisition module; The language recognition module obtains m pieces of sentence information and converts the i-th sentence information S i Perform word segmentation operation and get S i =(t i1 ,t i2 ,...,t in ) where t in Indicates the nth keyword in the i-th sentence information; Calculate the word segmentation weight w ik , specifically: Among them, m represents the number of sentence information, n represents the number of keywords in each sentence information, and w ik represents the weight of the kth keyword in the i-th sentence information, g(t ik ) represents the number of times the kth keyword in the ith sentence information appears in the mth sentence information, e(t ik ) represents the number of times the kth keyword in the i-th sentence information appears in the i-th sentence information; Obtain word data corresponding to the first h keywords with the highest weight from the word database of the database server, and send the word data to the content acquisition module, where h=n / 2; The content acquisition module acquires the interaction information corresponding to the word data from the database server according to the received word data, and sends the interaction information to the processing server; Specifically: vectorize the received word data to obtain the vector matrix J, J=[j1...j h ] Among them, j h is the vector of the hth word data; The content acquisition module acquires the rth piece of interaction information d from the database server r The corpus vector matrix U, U=[u r1 …in rj ] Among them, u rj is the vector of the jth word in the rth interactive information; The content acquisition module calculates the probability P(d r ), Among them, z is the number of interactive information, u rc is the rth interactive information d r The c-th word vector in N(u rc , d r ) is the rth interactive information d r The cth word vector in the rth interactive information d r The number of occurrences, u c is the vector of the cth word in the preset condition information, N(u c , d r ) is the vector of the cth word in the preset condition information in the rth talent data information d r Number of occurrences; The content acquisition module determines the conditional probability P(d r ) is greater than the preset probability value, the rth interactive information d r Sent to the processing server.

2. A cloud computing-based voice interaction system according to claim 1, characterized in that: The speech conversion module includes a feature extraction unit, a feature matching unit, and a sentence generation unit; The feature extraction unit is used to extract identification feature information from the audio information; The feature matching unit is used to obtain vocabulary information according to the matching of the recognition feature information; The sentence generating unit is used to generate sentence information based on the vocabulary information.

3. A cloud computing-based voice interaction method, the method being implemented by the cloud computing-based voice interaction system according to claim 1 or 2, characterized in that: The following steps are involved: S1. The voice collection device collects the sound of the user's speech, converts it into audio information, and sends it to the output device; S2. The output device sends the audio information to the processing server; S3. The speech conversion module of the processing server is used to receive the audio information, convert it into sentence information, and send it to the interactive server; S4. The language recognition module of the interactive server performs word segmentation on the sentence information, performs weight analysis on the word segmentation results, obtains word data corresponding to high-weight word segmentation from the word database of the database server, and sends it to the content acquisition module of the interactive server; S5. The content acquisition module obtains the interactive information corresponding to the word data from the database server according to the received word data, and sends it to the processing server; S6. The speech synthesis module of the processing server synthesizes the interactive information from the interactive server into audio data and sends it to the voice interaction terminal; S7. The input device of the voice interaction terminal receives the audio data from the processing server and sends it to the voice playback device; S8. The voice playing device converts the audio data into sound and plays it to the user.

4. The cloud computing-based voice interaction method according to claim 3, characterized in that: The step S4 specifically includes: S41. The language recognition module obtains m pieces of sentence information and converts the i-th sentence information S i Perform word segmentation operation and get S i =(t i1 ,t i2 ,…,t in ) where t in Indicates the nth keyword in the i-th sentence information; S42. Calculate word segmentation weight w ik , specifically: Among them, m represents the number of sentence information, n represents the number of keywords in each sentence information, and w ik represents the weight of the kth keyword in the i-th sentence information, g(t ik ) represents the number of times the kth keyword in the ith sentence information appears in the mth sentence information, e(t ik ) represents the number of times the kth keyword in the i-th sentence information appears in the i-th sentence information; S43. Obtain word data corresponding to the first h keywords with the highest weights from the word database of the database server, and send them to the content acquisition module, where h=n / 2.

5. The cloud computing-based voice interaction method according to claim 3, characterized in that: The step S5 specifically includes: S51. The content acquisition module vectorizes the received word data to obtain a vector matrix J. J=[j1...j h ] Among them, j h is the vector of the hth word data; S52. The content acquisition module acquires the rth interactive information d from the content database. r The corpus vector matrix U, U=[u r1 ...in rj ] Among them, u rj is the vector of the jth word in the rth interactive information; S53. The content acquisition module calculates the probability P(d r ), Among them, z is the number of interactive information, u rc is the rth interactive information d r The c-th word vector in N(u rc ,d r ) is the rth interactive information d r The cth word vector in the rth interactive information d r The number of occurrences, u c is the vector of the cth word in the preset condition information, N(u c ,d r ) is the vector of the cth word in the preset condition information in the rth talent data information d r Number of occurrences; S54. The content acquisition module determines the conditional probability P(d r ) is greater than the preset probability value, the rth interactive information d r Sent to the processing server.

6. A computer program product, comprising a computer program, wherein the computer program is executed by a processor and is used to execute the cloud computing-based voice interaction method according to any one of claims 3 to 5.

7. A computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is executed by a processor to execute the cloud computing-based voice interaction method described in any one of claims 3 to 5.

Images