Intelligent question-answering method, device, equipment, and storage medium based on front and back nasal sounds

Abstract

This invention relates to artificial intelligence technology and discloses an intelligent question-answering method, apparatus, device, and storage medium based on nasal sounds. The method includes: splitting user-input speech audio data to obtain a phoneme sequence; converting the phoneme sequence into initials and finals according to nasal sound conversion rules to obtain a phoneme arrangement matrix, and quantizing it to obtain a phoneme vector matrix; performing a fully connected operation on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences; performing feature extraction and word segmentation recognition operations on each combined phoneme sequence in the combined phoneme sequence set to obtain a word segmentation arrangement matrix corresponding to each combined phoneme sequence; performing word segmentation and arrangement on each word segmentation arrangement matrix, identifying the completeness of the intent in the word segmentation and arrangement results to obtain a complete intent statement; and filtering the complete intent statement according to current business scenario information to obtain the speech intent text. This invention can increase the accuracy of speech recognition and improve the efficiency of intelligent question answering.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent question-and-answer method, apparatus, device, and computer-readable storage medium based on nasal sounds. Background Technology

[0002] With the development of the times, intelligent voice assistants have gradually expanded into all aspects of people's daily life and work, making life more convenient and improving work efficiency.

[0003] The regional differences in Mandarin between the North and South result in accents for both nasal and non-nasal sounds, which can significantly interfere with human-computer interaction and intelligent question answering. Summary of the Invention

[0004] This invention provides an intelligent question-and-answer method, device, equipment, and storage medium based on front and back nasal sounds. Its main purpose is to improve the efficiency of intelligent question-and-answer by increasing the accuracy of speech recognition.

[0005] To achieve the above objectives, the present invention provides an intelligent question-and-answer method based on nasal sounds, comprising:

[0006] The voice audio data input by the user is recognized to obtain the voice pinyin, and the voice pinyin is split into a phoneme sequence;

[0007] Using the input layer of a pre-trained speech recognition model, the phoneme sequence is converted into initials and finals according to preset rules for converting nasal initials and finals to obtain a phoneme arrangement matrix. The phoneme arrangement matrix is ​​then quantized for phonemes and position coordinates to obtain a phoneme vector matrix.

[0008] By performing a full connection on each phoneme vector in the phoneme vector matrix, a set of combined phoneme sequences is obtained;

[0009] The speech recognition model is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the set of combined phoneme sequences to obtain the word segmentation and permutation matrix corresponding to each combined phoneme sequence.

[0010] Each of the word segmentation and arrangement matrices is segmented and arranged, and the completeness of the intent of the word segmentation and arrangement results is identified. The word segmentation and arrangement statements with a completeness score greater than a preset qualified threshold are output to obtain the complete intent statements corresponding to each of the word segmentation and arrangement matrices.

[0011] The system obtains the current business scenario information corresponding to the voice audio data, and uses the output layer of the voice recognition model to perform scenario adaptability judgment on each complete intent statement based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information, and sends it to the pre-built intelligent question answering program.

[0012] Optionally, the step of converting the phoneme sequence into initials and finals according to preset nasal initial and final conversion rules to obtain a phoneme arrangement matrix includes:

[0013] The phoneme sequences are arranged horizontally in a pre-constructed table;

[0014] According to the preset conversion rules for front and back nasal initials and finals, each phoneme to be replaced in the phoneme sequence is identified, and according to the phoneme correspondence, the replaceable phonemes in the conversion rules for front and back nasal initials and finals are vertically filled into each phoneme to be replaced in the phoneme sequence to obtain a phoneme arrangement list.

[0015] The phoneme arrangement list is subjected to minimum external tangent matrix truncation and zero-value filling operations to obtain the phoneme arrangement matrix.

[0016] Optionally, the step of segmenting and arranging each of the word segmentation matrices, identifying the completeness of the intent in the word segmentation results, and outputting the word segmentation statements with a completeness score greater than a preset threshold, to obtain the complete intent statements corresponding to each of the word segmentation matrices, includes:

[0017] The elements in each column of the word segmentation and arrangement matrix are used as the neural nodes of each layer of the fully connected layer, and a fully connected operation is performed to obtain the word segmentation and arrangement sentence set.

[0018] The attention-based feature quantization operation is performed on each word segmentation and sorting statement in the word segmentation and sorting statement set, and the feature quantization result is subjected to feature extraction operation to obtain the attention-enhanced feature sequence;

[0019] The attention-enhanced feature sequence is subjected to integrity recognition and intent recognition to obtain a statement integrity score and an intent confidence score. The statement integrity score and the intent confidence score are then weighted and calculated to obtain an intent integrity score.

[0020] The word segmentation sentences with integrity scores greater than the preset qualified threshold are output to obtain the complete intent sentences corresponding to the combined phoneme sequences.

[0021] Optionally, the step of performing a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences includes:

[0022] The phoneme vector matrix is ​​split into a set of columns, and the elements in each column are used as neural nodes to construct a neural layer.

[0023] Based on the position vector of each column in the column set, the neural layers are sequentially sorted to construct a fully connected layer, and a fully connected operation is performed to obtain a set of combined phoneme sequences.

[0024] Optionally, before performing feature extraction and word segmentation recognition operations on each combined phoneme sequence in the set of combined phoneme sequences using the speech recognition model, the method further includes:

[0025] Obtain a set of statement samples containing integrity tags and intent type tags;

[0026] One sentence sample is extracted from the sentence sample set in sequence, and the sentence sample is propagated forward through the network using a pre-built speech recognition model to obtain the intent prediction result. The integrity prediction result is obtained through the output interface of the pre-built mid-layer network.

[0027] The comprehensive loss value of the integrity prediction result and the integrity label, and the intention prediction result and the intention type label are calculated using a preset integrity-intent cross-loss function.

[0028] Minimize the comprehensive loss value to obtain the network model parameters when minimized, and then use a forward feedback network to perform inverse model update on the network model parameters to obtain the updated speech recognition model;

[0029] Determine the convergence of the overall loss value;

[0030] When the comprehensive loss value fails to converge, return to the process described above of sequentially extracting one sentence sample from the sentence sample set, using the pre-built speech recognition model to perform network forward propagation on the sentence sample, and iteratively updating the updated speech recognition model;

[0031] When the comprehensive loss value converges, the last updated speech recognition model is obtained as the trained speech recognition model.

[0032] Optionally, the step of performing scenario adaptability judgment on each of the complete intent statements based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information includes:

[0033] Obtain the common intent types corresponding to the current business scenario information;

[0034] Determine whether the intent corresponding to each complete intent statement is in the common intent type, and output the complete intent statement whose intent is in the common intent type.

[0035] Optionally, after obtaining the voice intent text that conforms to the current business scenario information, the method may further include:

[0036] When the user is satisfied with the complete intent statement, the user's identity information is obtained, and the model gradient of the speech recognition model is transferred and downloaded. The model gradient and the identity information are then stored accordingly.

[0037] To address the aforementioned problems, the present invention also provides an intelligent question-and-answer device based on nasal sounds, the device comprising:

[0038] The voice data acquisition module is used to recognize the voice audio data input by the user, obtain the voice pinyin, and split the voice pinyin into a phoneme sequence;

[0039] The front and back nasal phoneme quantization module is used to utilize the input layer of a pre-trained speech recognition model to convert the phoneme sequence into initials and finals according to preset front and back nasal initials and finals conversion rules, to obtain a phoneme arrangement matrix, and to quantize the phoneme arrangement matrix and position coordinates to obtain a phoneme vector matrix.

[0040] The vector fully connected combination module is used to perform a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences.

[0041] The intent-complete sentence recognition module is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the combined phoneme sequence set using the speech recognition model, to obtain the word segmentation arrangement matrix corresponding to each combined phoneme sequence, to perform word segmentation arrangement on each word segmentation arrangement matrix, to identify the intent completeness of the word segmentation arrangement result, and to output the word segmentation arrangement sentences with a completeness score greater than a preset qualified threshold, so as to obtain the complete intent sentences corresponding to each word segmentation arrangement matrix;

[0042] The scenario-adaptive statement filtering module is used to obtain the current business scenario information corresponding to the voice audio data, and use the output layer of the speech recognition model to perform scenario-adaptive judgment on each complete intent statement according to the current business scenario information, so as to obtain the voice intent text that conforms to the current business scenario information and send it to the pre-built intelligent question answering program.

[0043] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0044] At least one processor; and,

[0045] A memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the intelligent question-and-answer method based on nasal sounds described above.

[0047] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the aforementioned intelligent question-and-answer method based on nasal sounds.

[0048] This invention acquires user voice audio data, first constructs a phoneme vector matrix using preset rules for converting nasal initials and finals, and then obtains a word segmentation ranking matrix through word segmentation and recognition. This invention employs two fully connected operations to consider all possible variations in nasal initials and finals, resulting in the optimal word segmentation ranking. Finally, this invention achieves optimal voice intent text output by matching sentence completeness and intent to current business scenario information, thereby increasing the efficiency of intelligent question answering. Therefore, this invention provides an intelligent question answering method, device, equipment, and storage medium based on nasal initials and finals, which can improve the efficiency of intelligent question answering by increasing the accuracy of speech recognition. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an intelligent question-and-answer method based on front and back nasal sounds provided in an embodiment of the present invention.

[0050] Figure 2 This is a detailed flowchart illustrating one step of an intelligent question-answering method based on front and back nasal sounds provided in an embodiment of the present invention.

[0051] Figure 3 This is a detailed flowchart illustrating one step of an intelligent question-answering method based on front and back nasal sounds provided in an embodiment of the present invention.

[0052] Figure 4 A functional block diagram of an intelligent question-and-answer device based on front and back nasal sounds provided in an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the intelligent question-and-answer method based on nasal sounds, according to an embodiment of the present invention.

[0054] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0056] This application provides an intelligent question-answering method based on nasal sounds. In this application, the executing entity of the intelligent question-answering method based on nasal sounds includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the intelligent question-answering method based on nasal sounds can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0057] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent question-answering method based on nasal sounds according to an embodiment of the present invention. In this embodiment, the intelligent question-answering method based on nasal sounds includes:

[0058] S1. Recognize the user-inputted voice audio data, obtain the voice pinyin, and split the voice pinyin into a phoneme sequence.

[0059] In this embodiment of the invention, a pre-built audio recognition model can be downloaded to convert a user's voice audio data into Chinese Pinyin. The audio recognition model is a neural network model based on sound feature recognition, which will not be elaborated upon here.

[0060] Furthermore, the phoneme is the smallest structural unit of speech pinyin, such as "a" being a monophonetic initial consonant and "ang" being a triphonetic initial consonant. In this embodiment of the invention, the speech pinyin is split according to the form of initials and finals using a sequential recognition method to obtain a phoneme sequence. For example, if the speech pinyin is

zhaoyixiawangwu

zh、ao、y、i、x、i、a、w、ang、w、u

[0061] S2. Using the input layer of the pre-trained speech recognition model, the phoneme sequence is converted into initials and finals according to the preset rules for converting nasal initials and finals to obtain a phoneme arrangement matrix. The phoneme arrangement matrix is ​​then quantized for phonemes and position coordinates to obtain a phoneme vector matrix.

[0062] In this embodiment of the invention, the speech recognition model includes an input layer, a processing layer, and an output layer. The input layer includes a fully connected network and a quantization network. The processing layer includes a fully connected network, an integrity recognition network, an intent recognition network, and a weighted calculation network. The output layer includes a fully connected network.

[0063] The first fully connected network is used to perform fully connected combination of different initials and finals, the second fully connected network is used to perform fully connected combination of each word segment, and the third fully connected network is used to perform multi-classification judgment on the intent recognition results to check whether the intent recognition results meet the current business scenario information.

[0064] For details, please refer to the following: Figure 2 As shown in this embodiment of the invention, the step of converting the phoneme sequence into initials and finals according to a preset nasal initial and final conversion rule to obtain a phoneme arrangement matrix includes:

[0065] S21. Arrange the phoneme sequences horizontally in a pre-constructed table;

[0066] S22. According to the preset conversion rules for front and back nasal initials and finals, identify each phoneme to be replaced in the phoneme sequence, and according to the phoneme correspondence, fill the replaceable phonemes in the conversion rules for front and back nasal initials and finals vertically into each phoneme to be replaced in the phoneme sequence to obtain a phoneme arrangement list.

[0067] S23. Perform minimum external tangent matrix truncation and zero-value filling operations on the phoneme arrangement list to obtain the phoneme arrangement matrix.

[0068] The rules for converting nasal initials and finals are based on the conversion of easily confused initials and finals obtained from big data statistics, such as (zh, z), (ch, c), (sh, s), (z, zh), (c, ch), (s, sh), (eng, en), (ang, an), (ing, in), (an, ang), (en, eng), (in, ing), etc.

[0069] Specifically, in this embodiment of the invention, the phoneme sequence is arranged horizontally in a blank table, resulting in:

[0070]

[0071] Then, using the aforementioned rules for converting nasal initials and finals, the phonemes to be replaced [zh, ang] in the phoneme sequence are identified, and the replaceable phonemes [z, an] are obtained. These replaceable phonemes are then vertically filled below each phoneme to be replaced, and zero-value filling and truncation operations are performed to obtain the phoneme arrangement matrix.

[0072] zh…ang…

[0073] z 0an 0

[0074] Then, in this embodiment of the invention, the initials and finals a, b, c, ... zh ... are uniquely quantized using the one-hot isoquantization method, and the row and column coordinates of each initial and final in the phoneme arrangement matrix are recorded to obtain the phoneme vector matrix.

[0075] S3. Perform a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences.

[0076] In detail, in this embodiment of the invention, the step of performing a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences includes:

[0077] The phoneme vector matrix is ​​split into a set of columns, and the elements in each column are used as neural nodes to construct a neural layer.

[0078] Based on the position vector of each column in the column set, the neural layers are sequentially sorted to construct a fully connected layer, and a fully connected operation is performed to obtain a set of combined phoneme sequences.

[0079] In this embodiment of the invention, the phoneme vector matrix is ​​transformed into a fully connected layer. The number of neural nodes in each layer of the fully connected layer corresponds to the number of neural nodes in the phoneme vector matrix, generally 1 to 2, or in other special cases, 3. The number of layers in the fully connected layer is the number of columns in the phoneme vector matrix. Then, through a fully connected operation, a set of combined phoneme sequences [zhaoyixiawangwu, zaoyixiawangwu, zhaoyixiawanwu, zaoyixiawanwu] is obtained.

[0080] S4. Using the speech recognition model, feature extraction and word segmentation recognition operations are performed on each combined phoneme sequence in the set of combined phoneme sequences to obtain the word segmentation arrangement matrix corresponding to each combined phoneme sequence.

[0081] In this embodiment of the invention, each pinyin is segmented by feature extraction and word segmentation recognition, and the corresponding Chinese characters for each segmented word are queried, such as [zhao: find, shine..., zao: make, chisel..., wangwu: Wang Wu..., wanwu: plaything, all things...]. Then, the segmented results are arranged, filled, and cropped using the same method as in step S2 above to obtain a word segmentation arrangement matrix.

[0082] Further reference Figure 3As shown in this embodiment of the invention, before performing feature extraction and word segmentation recognition operations on each combined phoneme sequence in the combined phoneme sequence set using the speech recognition model, the method further includes:

[0083] S401. Obtain a set of statement samples containing integrity tags and intent type tags;

[0084] S402. Extract a sentence sample from the sentence sample set in sequence, use a pre-built speech recognition model to perform network forward propagation on the sentence sample to obtain the intent prediction result, and obtain the integrity prediction result through the output interface of the pre-built mid-layer network.

[0085] S403. Calculate the combined loss value of the integrity prediction result and the integrity label, and the intention prediction result and the intention type label, using a preset integrity-intent cross-loss function;

[0086] S404. Minimize the comprehensive loss value to obtain the network model parameters when minimized, and update the network model parameters in reverse through a forward feedback network to obtain an updated speech recognition model.

[0087] S405. Determine the convergence of the comprehensive loss value;

[0088] If the comprehensive loss value does not converge, return to the process in S402 above and iteratively update the speech recognition model.

[0089] When the comprehensive loss value converges, S406, obtain the last updated speech recognition model as the trained speech recognition model.

[0090] In this embodiment of the invention, a set of sentence samples is obtained through the above operations. Using a pre-built intermediate network output interface, two intermediate data outputs are performed during the execution of the speech recognition model to determine the accuracy of the model parameters of the integrity recognition network and the intent recognition network. Since the other structures of the speech recognition model do not require training, the speech recognition model can be completed once the integrity recognition network and the intent recognition network are trained.

[0091] S5. Perform word segmentation and arrangement on each of the word segmentation and arrangement matrices, identify the completeness of the intent of the word segmentation and arrangement results, and output the word segmentation and arrangement statements with a completeness score greater than a preset qualified threshold to obtain the complete intent statements corresponding to each of the word segmentation and arrangement matrices.

[0092] In detail, in this embodiment of the invention, the step of segmenting and arranging each of the word segmentation matrices, identifying the completeness of the intent of the word segmentation results, and outputting the word segmentation statements with a completeness score greater than a preset qualified threshold to obtain the complete intent statements corresponding to each of the word segmentation matrices includes:

[0093] The elements in each column of the word segmentation and arrangement matrix are used as the neural nodes of each layer of the fully connected layer, and a fully connected operation is performed to obtain the word segmentation and arrangement sentence set.

[0094] The attention-based feature quantization operation is performed on each word segmentation and sorting statement in the word segmentation and sorting statement set, and the feature quantization result is subjected to feature extraction operation to obtain the attention-enhanced feature sequence;

[0095] The attention-enhanced feature sequence is subjected to integrity recognition and intent recognition to obtain a statement integrity score and an intent confidence score. The statement integrity score and the intent confidence score are then weighted and calculated to obtain an intent integrity score.

[0096] The word segmentation sentences with integrity scores greater than the preset qualified threshold are output to obtain the complete intent sentences corresponding to the combined phoneme sequences.

[0097] In this embodiment of the invention, the method in step S3 above is first used to transform the word segmentation matrix into data in a fully connected layer, and a fully connected operation is performed to obtain a set of word segmentation sentences. Then, this embodiment of the invention uses an attention mechanism to quantify the sequential relationships of words in each word segmentation sentence to obtain an attention-enhanced feature sequence. Next, through a trained integrity recognition network and intent recognition network, integrity recognition and intent recognition are performed on the attention-enhanced feature sequence to obtain a sentence integrity score and an intent confidence score. Finally, the scores are weighted using preset weighting coefficients to obtain an intent integrity score.

[0098] When the intent integrity score is greater than the preset qualified threshold, it indicates that the corresponding word segmentation and arrangement statement is a qualified statement in terms of expressing intent and semantic integrity.

[0099] S6. Obtain the current business scenario information corresponding to the voice audio data, and use the output layer of the voice recognition model to perform scenario adaptability judgment on each complete intent statement according to the current business scenario information, so as to obtain the voice intent text that conforms to the current business scenario information, and send it to the pre-built intelligent question answering program.

[0100] In detail, in this embodiment of the invention, the step of performing scenario adaptability judgment on each of the complete intent statements based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information includes:

[0101] Obtain the common intent types corresponding to the current business scenario information;

[0102] Determine whether the intent corresponding to each complete intent statement is in the common intent type, and output the complete intent statement whose intent is in the common intent type.

[0103] The commonly used intent types are based on different scenarios, such as "digital work scenario," "smart home control scenario," or "medical check-up scenario," and have different user needs. For example, common intent types in the "digital work scenario" are often [address book search intent, file search intent, etc.].

[0104] In this embodiment of the invention, based on whether the intent corresponding to each complete intent statement is in the common intent types, a complete intent statement that conforms to the semantics of the current scenario is selected and output, and then subsequent human-computer interaction, intelligent question answering and other operations are performed.

[0105] Furthermore, in another embodiment of the present invention, after obtaining the voice intent text that conforms to the current business scenario information, the method may further include:

[0106] When the user is satisfied with the complete intent statement, the user's identity information is obtained, and the model gradient of the speech recognition model is transferred and downloaded. The model gradient and the identity information are then stored accordingly.

[0107] It should be understood that different users have different speech patterns and habits. Through the above operations, the intelligent voice assistant can perform personalized training on the speech recognition model based on individual identity information, thereby gradually increasing the efficiency of the voice interaction process.

[0108] This invention acquires user voice audio data, first constructs a phoneme vector matrix using preset rules for converting nasal initials and finals, and then obtains a word segmentation ranking matrix through word segmentation and recognition. This invention employs two fully connected operations to consider all possible variations in nasal initials and finals, resulting in the optimal word segmentation ranking. Finally, this invention achieves optimal voice intent text output by matching sentence completeness and intent with current business scenario information, thereby increasing the efficiency of intelligent question answering. Therefore, this invention provides an intelligent question answering method based on nasal initials and finals, which improves the efficiency of intelligent question answering by increasing the accuracy of speech recognition.

[0109] like Figure 4The diagram shown is a functional block diagram of an intelligent question-and-answer device based on nasal sounds provided in an embodiment of the present invention.

[0110] The intelligent question-answering device 100 based on nasal sounds described in this invention can be installed in an electronic device. Depending on the functions implemented, the intelligent question-answering device 100 based on nasal sounds may include a speech data acquisition module 101, a nasal sound phoneme quantization module 102, a vector fully connected combination module 103, an intent-complete sentence recognition module 104, and a scene-adaptive sentence filtering module 105. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0111] In this embodiment, the functions of each module / unit are as follows:

[0112] The voice data acquisition module 101 is used to recognize the voice audio data input by the user, obtain the voice pinyin, and split the voice pinyin into a phoneme sequence;

[0113] The front and back nasal phoneme quantization module 102 is used to utilize the input layer of the pre-trained speech recognition model to convert the phoneme sequence into initials and finals according to the preset front and back nasal initials and finals conversion rules, to obtain a phoneme arrangement matrix, and to quantize the phoneme arrangement matrix and position coordinates to obtain a phoneme vector matrix.

[0114] The vector fully connected combination module 103 is used to perform a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences.

[0115] The intent-complete sentence recognition module 104 is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the combined phoneme sequence set using the speech recognition model, to obtain the word segmentation arrangement matrix corresponding to each combined phoneme sequence, to perform word segmentation arrangement on each word segmentation arrangement matrix, to identify the intent completeness of the word segmentation arrangement result, and to output the word segmentation arrangement sentences with a completeness score greater than a preset qualified threshold, so as to obtain the complete intent sentences corresponding to each word segmentation arrangement matrix;

[0116] The scenario-adaptive statement filtering module 105 is used to obtain the current business scenario information corresponding to the voice audio data, and use the output layer of the speech recognition model to perform scenario-adaptive judgment on each complete intent statement according to the current business scenario information, so as to obtain the voice intent text that conforms to the current business scenario information and send it to the pre-built intelligent question answering program.

[0117] In detail, the modules in the intelligent question-and-answer device 100 based on nasal sounds described in this application embodiment adopt the same usage as described above. Figures 1 to 3 The method uses the same technical means as the intelligent question-and-answer method based on nasal sounds described above, and can produce the same technical effect, so it will not be repeated here.

[0118] like Figure 5 The diagram shown is a structural schematic of an electronic device 1 that implements an intelligent question-and-answer method based on nasal sounds according to an embodiment of the present invention.

[0119] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as an intelligent question-and-answer program based on nasal sounds.

[0120] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device 1, connecting various components of the electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing an intelligent question-and-answer program based on nasal sounds) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0121] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of an intelligent question-and-answer program based on nasal sounds, but also to temporarily store data that has been output or will be output.

[0122] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0123] The communication interface 13 is used for communication between the electronic device 1 and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0124] Figure 5 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 5 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0125] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0126] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0127] The intelligent question-and-answer program based on nasal sounds stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0128] The voice audio data input by the user is recognized to obtain the voice pinyin, and the voice pinyin is split into a phoneme sequence;

[0129] Using the input layer of a pre-trained speech recognition model, the phoneme sequence is converted into initials and finals according to preset rules for converting nasal initials and finals to obtain a phoneme arrangement matrix. The phoneme arrangement matrix is ​​then quantized for phonemes and position coordinates to obtain a phoneme vector matrix.

[0130] By performing a full connection on each phoneme vector in the phoneme vector matrix, a set of combined phoneme sequences is obtained;

[0131] The speech recognition model is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the set of combined phoneme sequences to obtain the word segmentation and permutation matrix corresponding to each combined phoneme sequence.

[0132] Each of the word segmentation and arrangement matrices is segmented and arranged, and the completeness of the intent of the word segmentation and arrangement results is identified. The word segmentation and arrangement statements with a completeness score greater than a preset qualified threshold are output to obtain the complete intent statements corresponding to each of the word segmentation and arrangement matrices.

[0133] The system obtains the current business scenario information corresponding to the voice audio data, and uses the output layer of the voice recognition model to perform scenario adaptability judgment on each complete intent statement based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information, and sends it to the pre-built intelligent question answering program.

[0134] Specifically, the specific implementation method of the processor 10 for the above instructions can be referred to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, and will not be repeated here.

[0135] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0136] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0137] The voice audio data input by the user is recognized to obtain the voice pinyin, and the voice pinyin is split into a phoneme sequence;

[0138] Using the input layer of a pre-trained speech recognition model, the phoneme sequence is converted into initials and finals according to preset rules for converting nasal initials and finals to obtain a phoneme arrangement matrix. The phoneme arrangement matrix is ​​then quantized for phonemes and position coordinates to obtain a phoneme vector matrix.

[0139] By performing a full connection on each phoneme vector in the phoneme vector matrix, a set of combined phoneme sequences is obtained;

[0140] The speech recognition model is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the set of combined phoneme sequences to obtain the word segmentation and permutation matrix corresponding to each combined phoneme sequence.

[0141] Each of the word segmentation and arrangement matrices is segmented and arranged, and the completeness of the intent of the word segmentation and arrangement results is identified. The word segmentation and arrangement statements with a completeness score greater than a preset qualified threshold are output to obtain the complete intent statements corresponding to each of the word segmentation and arrangement matrices.

[0142] The system obtains the current business scenario information corresponding to the voice audio data, and uses the output layer of the voice recognition model to perform scenario adaptability judgment on each complete intent statement based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information, and sends it to the pre-built intelligent question answering program.

[0143] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0144] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0146] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0147] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0148] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0149] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0150] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent question-answering method based on nasal sounds, characterized in that, The method includes: The voice audio data input by the user is recognized to obtain the voice pinyin, and the voice pinyin is split into a phoneme sequence; Using the input layer of a pre-trained speech recognition model, the phoneme sequence is arranged horizontally in a pre-constructed table. According to the preset conversion rules for front and back nasal initials and finals, each phoneme to be replaced in the phoneme sequence is identified. According to the phoneme correspondence, the replaceable phonemes in the conversion rules for front and back nasal initials and finals are vertically filled into each phoneme to be replaced in the phoneme sequence to obtain a phoneme arrangement list. The phoneme arrangement list is subjected to minimum external tangent matrix truncation and zero-value filling operations to obtain a phoneme arrangement matrix. The phoneme arrangement matrix is ​​then subjected to phoneme quantization and position coordinate quantization to obtain a phoneme vector matrix. By performing a full connection on each phoneme vector in the phoneme vector matrix, a set of combined phoneme sequences is obtained; The speech recognition model is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the set of combined phoneme sequences to obtain the word segmentation and permutation matrix corresponding to each combined phoneme sequence. Each of the word segmentation and arrangement matrices is segmented and arranged, and the completeness of the intent of the word segmentation and arrangement results is identified. The word segmentation and arrangement statements with a completeness score greater than a preset qualified threshold are output to obtain the complete intent statements corresponding to each of the word segmentation and arrangement matrices. The system obtains the current business scenario information corresponding to the voice audio data, and uses the output layer of the voice recognition model to perform scenario adaptability judgment on each complete intent statement based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information, and sends it to the pre-built intelligent question answering program.

2. The intelligent question-answering method based on nasal sounds as described in claim 1, characterized in that, The process involves segmenting and arranging each of the segmentation matrices, identifying the completeness of the intent in the segmentation results, and outputting segmented sentences with a completeness score greater than a preset threshold, thereby obtaining the complete intent statements corresponding to each of the segmentation matrices. This includes: The elements in each column of the word segmentation and arrangement matrix are used as the neural nodes of each layer of the fully connected layer, and a fully connected operation is performed to obtain the word segmentation and arrangement sentence set. The attention-based feature quantization operation is performed on each word segmentation and sorting statement in the word segmentation and sorting statement set, and the feature quantization result is subjected to feature extraction operation to obtain the attention-enhanced feature sequence. The attention-enhanced feature sequence is subjected to integrity recognition and intent recognition to obtain a statement integrity score and an intent confidence score. The statement integrity score and the intent confidence score are then weighted and calculated to obtain an intent integrity score. The word segmentation sentences with integrity scores greater than the preset qualified threshold are output to obtain the complete intent sentences corresponding to the combined phoneme sequences.

3. The intelligent question-answering method based on nasal sounds as described in claim 1, characterized in that, The step of performing a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences includes: The phoneme vector matrix is ​​split into a set of columns, and the elements in each column are used as neural nodes to construct a neural layer. Based on the position vector of each column in the column set, the neural layers are sequentially sorted to construct a fully connected layer, and a fully connected operation is performed to obtain a set of combined phoneme sequences.

4. The intelligent question-answering method based on nasal sounds as described in claim 1, characterized in that, Before performing feature extraction and word segmentation recognition operations on each combined phoneme sequence in the set of combined phoneme sequences using the speech recognition model, the method further includes: Obtain a set of statement samples containing integrity tags and intent type tags; One sentence sample is extracted from the sentence sample set in sequence, and the sentence sample is propagated forward through the network using a pre-built speech recognition model to obtain the intent prediction result. The integrity prediction result is obtained through the output interface of the pre-built mid-layer network. The comprehensive loss value of the integrity prediction result and the integrity label, and the intention prediction result and the intention type label are calculated using a preset integrity-intent cross-loss function. Minimize the comprehensive loss value to obtain the network model parameters when minimized, and then use a forward feedback network to perform inverse model update on the network model parameters to obtain the updated speech recognition model; Determine the convergence of the overall loss value; When the comprehensive loss value fails to converge, return to the process described above of sequentially extracting one sentence sample from the sentence sample set, using the pre-built speech recognition model to perform network forward propagation on the sentence sample, and iteratively updating the updated speech recognition model; When the comprehensive loss value converges, the last updated speech recognition model is obtained as the trained speech recognition model.

5. The intelligent question-answering method based on nasal sounds as described in claim 1, characterized in that, The step of performing scenario adaptability judgment on each of the complete intent statements based on the current business scenario information to obtain voice intent text that conforms to the current business scenario information includes: Obtain the common intent types corresponding to the current business scenario information; Determine whether the intent corresponding to each complete intent statement is in the common intent type, and output the complete intent statement whose intent is in the common intent type.

6. The intelligent question-answering method based on nasal sounds as described in claim 1, characterized in that, After obtaining the voice intent text that conforms to the current business scenario information, the method may further include: When the user is satisfied with the complete intent statement, the user's identity information is obtained, and the model gradient of the speech recognition model is transferred and downloaded. The model gradient and the identity information are then stored accordingly.

7. An intelligent question-and-answer device based on nasal sounds, characterized in that, The device includes: The voice data acquisition module is used to recognize the voice audio data input by the user, obtain the voice pinyin, and split the voice pinyin into a phoneme sequence; The front and back nasal phoneme quantization module is used to use the input layer of a pre-trained speech recognition model to horizontally arrange the phoneme sequence in a pre-constructed table, identify each phoneme to be replaced in the phoneme sequence according to the preset front and back nasal initial and final conversion rules, and vertically fill each phoneme to be replaced in the phoneme sequence according to the phoneme correspondence relationship, so as to obtain a phoneme arrangement list. The phoneme arrangement list is subjected to minimum external tangent matrix truncation operation and zero value filling operation to obtain a phoneme arrangement matrix. The phoneme arrangement matrix is ​​then subjected to phoneme quantization and position coordinate quantization to obtain a phoneme vector matrix. The vector fully connected combination module is used to perform a full connection on each phoneme vector in the phoneme vector matrix to obtain a set of combined phoneme sequences. The intent-complete sentence recognition module is used to perform feature extraction and word segmentation recognition operations on each combined phoneme sequence in the combined phoneme sequence set using the speech recognition model, to obtain the word segmentation arrangement matrix corresponding to each combined phoneme sequence, and to perform word segmentation arrangement on each word segmentation arrangement matrix, and to identify the intent completeness of the word segmentation arrangement result, and to output the word segmentation arrangement sentences with a completeness score greater than a preset qualified threshold, so as to obtain the complete intent sentences corresponding to each word segmentation arrangement matrix; The scenario-adaptive statement filtering module is used to obtain the current business scenario information corresponding to the voice audio data, and use the output layer of the speech recognition model to perform scenario-adaptive judgment on each complete intent statement according to the current business scenario information, so as to obtain the voice intent text that conforms to the current business scenario information and send it to the pre-built intelligent question answering program.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intelligent question-and-answer method based on nasal sounds as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent question-and-answer method based on nasal sounds as described in any one of claims 1 to 6.

Images