Voice processing method, voice processing device and storage medium

Abstract

The invention discloses a voice processing method, voice processing equipment and a storage medium, and relates to the technical field of voice recognition, and the voice processing method comprises the steps that when a voice processing process is triggered, a query formula corresponding to a received voice signal is determined; acquiring historical interaction data based on a memory module; and performing query processing on the historical interaction data based on the query formula, and when a query result is not null, generating a voice processing result associated with the voice signal based on the query result. When voice processing is carried out, the voice signal is converted into the query formula, operation processing is carried out in the background based on the query formula, and existing solutions can be reused for processing when operation is carried out in the background, so that the processing result conforms to the pronunciation habit, the phrase habit and the actual demand of a user, and the accuracy of voice question and answer processing is improved.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech processing method, a speech processing device and a storage medium. Background Art

[0002] In the current voice question-answering system, although the automatic speech-to-text function has been realized, the existing speech recognition system may not be able to accurately capture the voice signal when faced with different accents, speaking speeds and intonations, resulting in misrecognition or missed recognition. At the same time, background noise and echoes will also affect the accuracy of speech recognition, resulting in low accuracy of the results obtained based on voice question-answering.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a speech processing method, a speech processing device and a storage medium, aiming to solve the technical problem of low accuracy of results obtained based on voice question and answer.

[0005] To achieve the above object, the present application proposes a speech processing method, the method comprising: When the voice processing process is triggered, a query formula corresponding to the received voice signal is determined; Obtain historical interaction data based on the memory module; The historical interaction data is queried based on the query formula, and when the query result is not empty, a speech processing result associated with the speech signal is generated based on the query result.

[0006] In one embodiment, when the voice processing process is triggered, the step of determining the query formula corresponding to the received voice signal includes: When the voice processing process is triggered, the received voice signal is converted into text information based on the deep learning model and the end-to-end model; Semantic parsing is performed on the keywords and grammatical structure of the text information, and the query formula is generated based on the query language or expression after the parsing.

[0007] In one embodiment, after the step of performing query processing on the historical interaction data based on the query formula and generating a speech processing result associated with the speech signal based on the query result when the query result is not empty, the step further includes: Generate query keywords and query tags based on the query results; Inputting the query keyword and the query tag into the display interface; In response to the feedback received by the display interface, the model parameters are adjusted, and the deep learning model and the end-to-end model are updated based on the feedback.

[0008] In one embodiment, before the step of performing semantic parsing on the keywords and grammatical structures of the text information and generating the query formula based on the query language or expression after the parsing, the step further includes: Extracting the keywords and the grammatical structure of the text information based on natural language processing technology; The step of performing semantic parsing on the keywords and grammatical structures of the text information and generating the query formula based on the query language or expression after the parsing includes: Perform semantic parsing on the keywords and grammatical structure of the text information based on a sequence-to-sequence model or a graph neural network to obtain the query language or the expression; The graph structure information of the query language or the expression is determined according to a preset knowledge graph, and the graph structure information is parsed based on a preset rule engine to obtain the query formula.

[0009] In one embodiment, when the voice processing process is triggered, the step of converting the received voice signal into text information based on the deep learning model and the end-to-end model includes: When the speech processing process is triggered, the received speech signal is preprocessed and feature extracted to obtain a feature vector; Inputting the feature vector into a deep learning model, and obtaining initial text information of the speech signal based on the deep learning model; The initial text information is subjected to text mapping processing based on an end-to-end model to obtain the text information.

[0010] In one embodiment, after the step of acquiring historical interaction data based on the memory module, the method further includes: When the query result is empty, obtaining user preference information based on the memory module; Query processing is performed in a database according to the user preference information and the query formula to obtain the voice processing result.

[0011] In one embodiment, before the step of acquiring historical interaction data based on the memory module, the method further includes: Constructing a long short-term memory network model, and training the long short-term memory network model with the historical interaction data and the user preference information to obtain the memory module; After the step of performing query processing on the historical interaction data based on the query formula and generating a voice processing result associated with the voice signal based on the query result when the query result is not empty, the method further includes: When a preset data update period is reached, the user preference information and the historical interaction data in the memory module are updated.

[0012] In one embodiment, before the step of acquiring historical interaction data based on the memory module, the method further includes: Constructing a knowledge graph corresponding to the historical interaction data; The step of acquiring historical interaction data based on the memory module includes: Based on the entity structure relationship of the knowledge graph, the user preference information and the historical interaction data are obtained in the memory module.

[0013] In addition, to achieve the above objectives, the present application also proposes a speech processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speech processing method described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the speech processing method described above are implemented.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: When performing voice processing, the voice signal is first converted into a query formula and background calculations are performed. During the calculation, past solutions are reused to reduce repeated calculations. The user historical interaction data stored in the memory module is used for query processing. During the query process, existing solutions are reused to avoid complex processing of the entire voice data and improve the accuracy of the voice processing results based on existing solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 A flowchart of a first embodiment of the speech processing method provided by the present application; Figure 2 A flowchart of a second embodiment of the speech processing method provided by the present application; Figure 3 A flowchart of a third embodiment of the speech processing method provided by the present application; Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the voice processing method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] The main solution of the embodiment of the present application is: when the voice processing process is triggered, the query formula corresponding to the received voice signal is determined; Obtain historical interaction data based on the memory module; The historical interaction data is queried based on the query formula, and when the query result is not empty, a speech processing result associated with the speech signal is generated based on the query result.

[0022] In this embodiment, for the convenience of description, the following description is made with the voice recognition device as the execution subject.

[0023] In the current voice question-answering system, although the automatic speech-to-text function has been realized, the existing speech recognition system may not be able to accurately capture the voice signal when faced with different accents, speaking speeds and intonations, resulting in misrecognition or missed recognition. At the same time, background noise and echoes will also affect the accuracy of speech recognition, resulting in low accuracy of the results obtained based on voice question-answering.

[0024] The present application provides a solution. When performing voice processing, the voice signal is first converted into a query formula, and background operations are performed based on the query formula. During the operation, historical interaction data is obtained through a memory module so that past solutions can be reused based on the historical interaction data to reduce repeated calculations. As a result, during the query process, the processing model is more adapted to the user's pronunciation habits, usage habits and actual needs, thereby improving the accuracy of the voice processing results.

[0025] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a voice processing device, etc. The following takes a voice processing device as an example to illustrate this embodiment and the following embodiments.

[0026] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0027] The present application embodiment provides a method for voice processing, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the speech processing method of the present application.

[0028] In this embodiment, the speech processing method includes steps S10 to S30: Step S10, when the voice processing process is triggered, determining the query formula corresponding to the received voice signal; In this embodiment, after the voice processing device runs and receives the voice signal, there is a lack of voice processing process, and then the received voice signal needs to be converted into text information. In this process, the voice signal needs to be adjusted so that the processing model can recognize the signal and query accurate results. Among them, the query formula can include a SQL (Structured Query Language) query of the database, or an API (Application Programming Interface) request, SPARQL (SPARQL Protocol and RDF Query Language) query or a custom query format.

[0029] As an optional implementation, taking SQL query as an example, in the process of determining the query formula corresponding to the voice signal, the deep learning speech recognition model can be used to extract features of the voice signal and convert these features into text information. In this process, an end-to-end model can be used to directly achieve audio-to-text mapping. After obtaining the text information, the content of the text information is parsed through natural language processing technology, and the query formula is generated based on the parsed content.

[0030] Optionally, speech signals can also be processed based on convolutional neural networks and recurrent neural networks to improve recognition performance by simultaneously capturing local features and global context information of speech.

[0031] In this embodiment, by converting speech into text and then converting text into query formulas, the speech processing device can perform calculations and processing in the background based on the query formulas, thereby improving the convenience and recognition efficiency of speech recognition operations. At the same time, when converting text into query formulas, deep learning, natural language processing (NLP), semantic analysis, and database query processing are integrated to form a highly intelligent processing system, which enables the speech processing device to handle complex speech processing tasks.

[0032] Step S20, acquiring historical interaction data based on the memory module; It should be noted that when obtaining historical interaction data, the system of the voice processing device needs to perform data query processing based on the knowledge graph constructed based on the historical interaction data. Among them, the memory module is a storage module in the voice processing device for storing relevant operation information, which assists the system in understanding and predicting user needs. Based on the memory module, the processing system of the voice processing device can remember the historical interaction data, and use the knowledge graph maintenance and query mode to select the historical query content associated with the query formula, so that past solutions can be reused to reduce repeated calculations. At the same time, more reasonable decisions can be made based on historical data to improve the accuracy of voice processing results. Among them, in addition to storing historical interaction data, the memory module also stores user preference information.

[0033] In this embodiment, the voice processing device also includes a knowledge graph for storing and retrieving historical data. Therefore, when obtaining historical interaction data based on the memory module, it is necessary to maintain and query the historical data through the knowledge graph. That is, before this step, it is also necessary to construct a knowledge graph corresponding to the historical interaction data, and then obtain the historical interaction data in the memory module based on the entity structure relationship in the knowledge graph.

[0034] Optionally, the memory module can be constructed by a long short-term memory network (LSTM), that is, after the long short-term memory network model is constructed, the long short-term memory network model is trained based on the historical interaction data and user preference information to obtain the memory module. It should be noted that after the long short-term memory network model is trained, it can be regarded as a memory module because it has learned the representation of historical interaction data and user preference information.

[0035] Optionally, in order to avoid outdated information from interfering with the voice processing device, it is also necessary to regularly update and optimize the content stored in the memory module. Based on this, after obtaining the voice processing results, when the preset data update period is reached, the user preference information and historical interaction data of the memory module need to be updated, wherein the updating method can be to optimize the data based on the voice signal received in real time.

[0036] It should be noted that in the process of constructing the knowledge graph, the entity structure relationship of the knowledge graph can be constructed based on the historical interaction data. Later, when it is necessary to determine whether the current query formula has a solution in a similar scenario, the entity structure relationship of the knowledge graph can be used to search and process from the corresponding node, and then the relevant historical interaction data and user preference information can be obtained. Then, when there is a target formula associated with the query formula in the queried historical interaction data, the existing solution can be reused to improve the accuracy of the speech processing results.

[0037] Step S30, performing query processing on the historical interaction data based on the query formula, and when the query result is not empty, generating a speech processing result associated with the speech signal based on the query result.

[0038] In this embodiment, query processing is performed on the historical interaction data based on the query formula. When the query result is not empty, it means that in the current voice processing process, the content to be queried corresponding to the voice signal already has a similar solution case. For example, the generated query formula is an SQL statement containing the query words such as "Lease Agreement", "Default", etc., and in the historical interaction data, there is a similar target formula in the query result, which is an SQL statement with words such as "Lease Agreement", "Default", "Consequence", etc., and the query result is not empty at this time. Based on this, it is necessary to generate voice processing based on the query result, that is, based on the existing solution, query results associated with the voice signal are queried in the database.

[0039] For example, suppose there is a company that provides legal consulting. The memory module has a large amount of historical case data, including customer questions, answers provided, and final legal solutions. At this time, the historical interaction data is first used to build a knowledge graph to improve service efficiency and quality. During the construction process, the historical interaction data is first analyzed to extract key entities such as case type, legal terms involved, customer problems, solutions, etc., and define the relationship between entities, such as "case type-involved-legal terms" or "customer problems-cause-solutions". Based on these entities and relationships, a knowledge graph in the legal field is constructed. Then, when the user raises a voice message for a legal query, the voice signal is converted into a corresponding query formula. Then the query formula is parsed to identify key entities such as case type, specific questions, etc., and then SQL statements are used to find cases similar to the new case in the knowledge graph, that is, the user asks about the legal consequences of breach of the lease contract.

[0040] When the system finds a target formula associated with the current query formula (similar cases identified using SQL) in the historical interaction data, that is, similar cases that have been processed before and the corresponding query results are not empty, the solutions for these cases can be reused. For example, the system may find that there have been similar cases of breach of lease contracts before, and successfully applied specific legal terms interpretation and solutions.

[0041] In this way, the voice processing device can quickly provide accurate suggestions, improving the accuracy of voice processing results. At the same time, by using the knowledge graph maintenance and query mode, the existing solutions can be reused to reduce repeated calculations. At the same time, more reasonable decisions can be made based on historical data to improve the accuracy of voice processing results.

[0042] In another optional implementation, when the query result is empty, the user preference information is obtained based on the memory module, and then the query processing is performed in the database according to the user preference information and the query formula to obtain the voice processing result. Specifically, there is no target formula corresponding to the query formula in the historical interaction data, which means that there is no relevant solution recorded in the current historical interaction data. At this time, the corresponding voice processing result can be directly queried and output according to the user preference information and the current query formula, thereby improving the accuracy of the voice processing result obtained by the voice processing device. In the process of query processing, SQL or other data access methods are used to retrieve data from the database, and then mathematical operations, statistical analysis, machine learning model prediction and other operations are performed. It should be noted that during the query process, it is also necessary to connect to the database to be queried. At the same time, if the amount of data query is large, tasks can be processed in parallel, or processed based on an asynchronous execution mechanism, thereby improving processing efficiency.

[0043] This embodiment provides a speech processing method. After the speech processing process is triggered, the speech information is first converted into a corresponding query formula. Then, based on the constructed knowledge graph, historical interaction data is obtained from the memory module. Then, based on the query formula, it is searched in the historical interaction data whether there is an existing solution. When there is an existing solution, the data calculation process is reduced by directly reusing the existing solution. At the same time, the existing solution is verified by the speech processing device, which can effectively improve the accuracy of the speech processing results.

[0044] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can refer to the above introduction, and will not be repeated in the following. Figure 2 , step S10 also includes steps S11-S12: Step S11, when the voice processing process is triggered, the received voice signal is converted into text information based on the deep learning model and the end-to-end model; In this embodiment, in the process of converting the voice signal into text information, the voice signal is firstly subjected to feature extraction processing using a deep learning speech recognition model, and these features are then converted into text information.

[0045] Specifically, when the speech processing process is triggered, the received speech signal needs to be preprocessed and feature extracted to obtain a feature vector, wherein the preprocessing process includes audio standardization, noise reduction, silence detection, etc., and the feature extraction process includes extracting Mel frequency cepstral coefficients (MFCCs), Mel spectrogram, linear spectrum logarithm, etc. Then the extracted feature vector is input into the deep learning model, and the initial text information of the speech signal is obtained based on the deep learning model, and then the initial text is subjected to text mapping processing based on the end-to-end model to obtain the text information.

[0046] Among them, during the training process of deep learning models, GPU can be used to accelerate processing when training based on large-scale labeled data.

[0047] Step S12, semantically analyzing the keywords and grammatical structures of the text information, and generating the query formula based on the query language or expression after the analysis.

[0048] In this embodiment, keywords and grammatical structures of text information can be extracted by natural language processing technology, and then semantic analysis processing can be performed on the keywords and grammatical structures of the text information, so as to convert the natural language into a structured query language or expression, and finally generate a query formula based on the obtained query language or expression. Wherein, when processing based on natural language processing technology, the processing process includes lexical analysis such as word segmentation and part-of-speech tagging, syntactic analysis such as data dependency analysis or component structure tree analysis, and in the semantic analysis process, the analysis result is converted into a query language such as SQL, SPARQL or a custom query format.

[0049] Furthermore, in the process of semantic processing, a sequence-to-sequence (Seq2Seq) model or a graph neural network (GNN) can be used for semantic parsing training. When determining the query formula, the context and relationship of the parsed analysis results can be understood based on the rule engine and knowledge graph to generate an accurate query formula.

[0050] Specifically, semantic parsing is performed on the keywords and grammatical structure of the text information based on a sequence-to-sequence model or a graph neural network to obtain the query language or the expression, and then the graph structure information of the query language or expression is determined according to a preset knowledge graph, and the graph structure information is parsed based on a preset rule engine to obtain the query formula.

[0051] For example, in an application in the financial field, a user asks for information about stocks through natural language. At this time, the processing system of the voice processing device needs to convert these inquiries into SQL query statements in order to extract the corresponding information from the database. Among them, the voice information input by the user is: "I want to know the performance of Company A's stocks in 20xx." Based on this, when performing lexical analysis, the sentence is decomposed into words or symbols, that is, "I" "want" "know" "Company A" "of" "stock" "in" "20xx" "of" "performance". Then, part-of-speech tagging is performed, that is, the part of speech of each word is identified, where "know" is a verb, "Company A" is a noun phrase, etc. Then, dependency analysis is performed to determine the grammatical structure between words, where "know" depends on "want" (indicating purpose), and the performance of "stock" is what the user wants to know.

[0052] When performing semantic processing, a tree diagram is constructed to express the sentence structure, clarifying the grammatical components such as subject, predicate, and object and the relationship between them. Then, the natural language is converted into a meaningful expression, and the above sentence is converted into an intent, i.e., a query language: "Query stock performance". The query language is further mapped to a structured query statement, SQL: "SELECT performance FROMstocks WHERE name = 'A Inc.' AND year = 20xx;" Furthermore, if there are specific domain rules (such as stock data only includes recent years), the query statement is adjusted according to the rules. At the same time, in this process, the relationships in the knowledge graph can be used to help understand complex queries, such as linking "Company A" to its stock code, historical data, etc.

[0053] During the model training process, the Seq2Seq model is used to receive input natural language sentences and then output the target query language (such as SQL). At the same time, a large number of annotated natural language and SQL query pairs are used for training to learn how to convert accurately. Optionally, GNN can also be used to handle complex dependencies and improve parsing accuracy.

[0054] This embodiment provides a voice processing method. After triggering the voice processing process, it integrates deep learning, natural language processing, semantic analysis and other processing processes to pre-process, extract features and map text to the voice signal. At the same time, it processes based on the knowledge graph and rule engine to generate corresponding query formulas to improve the accuracy of the query formulas. At the same time, starting from voice recognition to result feedback and optimization, the entire process is designed as a coherent chain, which can automatically complete the entire process from the voice command input by the user to the result feedback. The end-to-end design reduces manual intervention and improves the automation and efficiency of the system.

[0055] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction, and will not be repeated in the following. Figure 3 , after step S30, further comprising steps S40 to S60: Step S40, generating query keywords and query tags based on the query results; Step S50, inputting the query keyword and the query tag into the display interface; Step S60, adjusting model parameters in response to feedback received by the display interface, and updating the deep learning model and the end-to-end model based on the feedback adjusted model parameters.

[0056] In this embodiment, the processing system of the voice processing device can generate tags and keywords based on the query results, so as to facilitate users to quickly understand the current results, while increasing the flexibility and interactivity of the system, allowing users to intuitively filter information.

[0057] Therefore, after the query keywords and query tags are generated, the query keywords and query tags can be output to the display interface for the user to filter. When the model parameters are adjusted in response to the feedback input by the user in the display interface, the deep learning model and the end-to-end model are updated based on the feedback adjustment model parameters. That is, supervised or reinforcement learning algorithms are used to adjust the model parameters according to the user's selection feedback.

[0058] Based on this, by collecting user feedback information and using the feedback information for model retraining, the learning strategy can be effectively strengthened to optimize the long-term performance of the model.

[0059] This embodiment provides a speech processing method, which dynamically generates tags and keywords according to query results, and outputs the generated tags and keywords, so that users can quickly understand the current results, enhance the flexibility and interactivity of the system, and continuously adjust and improve the model through user feedback and historical data, so that the system can gradually understand and adapt to user habits and preferences, provide more personalized and accurate services, and thus improve the accuracy of speech processing results.

[0060] The present application provides a speech processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the speech processing method in the above-mentioned first embodiment.

[0061] Reference below Figure 4 , which shows a schematic diagram of the structure of a voice processing device suitable for implementing the embodiment of the present application. The voice processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The voice processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0062] like Figure 4As shown, the speech processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the speech processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the voice processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a voice processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0063] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0064] The speech processing device provided by the present application adopts the speech processing method in the above embodiment, which can solve the technical problem of low accuracy of the results obtained based on speech question and answer. Compared with the prior art, the beneficial effects of the speech processing device provided by the present application are the same as the beneficial effects of the speech processing method provided by the above embodiment, and the other technical features in the speech processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0065] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0066] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0067] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the speech processing method in the above-mentioned embodiment.

[0068] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0069] The computer-readable storage medium may be included in the speech processing device; or may exist independently without being assembled into the speech processing device.

[0070] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the speech processing device, the speech processing device: Determining a query formula corresponding to the received voice signal; Obtain historical interaction data based on the memory module; The historical interaction data is queried based on the query formula, and when the query result is not empty, a speech processing result associated with the speech signal is generated based on the query result.

[0071] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0072] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] The modules involved in the embodiments of the present application may be implemented by software or hardware, wherein the name of the module does not limit the unit itself in some cases.

[0074] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned speech processing method, and can solve the technical problem of low accuracy of the results obtained based on speech question and answer. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the speech processing method provided in the above-mentioned embodiment, and will not be described in detail here.

[0075] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned speech processing method when executed by a processor.

[0076] The computer program product provided by the present application can solve the technical problem of low accuracy of the results obtained based on voice question and answer. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the voice processing method provided by the above embodiment, which will not be repeated here.

[0077] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A speech processing method, characterized in that: The method comprises: When the voice processing process is triggered, a query formula corresponding to the received voice signal is determined; Obtain historical interaction data based on the memory module; The historical interaction data is queried based on the query formula, and when the query result is not empty, a speech processing result associated with the speech signal is generated based on the query result.

2. The method according to claim 1, characterized in that When the voice processing process is triggered, the step of determining the query formula corresponding to the received voice signal includes: When the voice processing process is triggered, the received voice signal is converted into text information based on the deep learning model and the end-to-end model; Semantic parsing is performed on the keywords and grammatical structure of the text information, and the query formula is generated based on the query language or expression after the parsing.

3. The method according to claim 2, characterized in that After the step of performing query processing on the historical interaction data based on the query formula and generating a voice processing result associated with the voice signal based on the query result when the query result is not empty, the method further includes: Generate query keywords and query tags based on the query results; Inputting the query keyword and the query tag into the display interface; In response to the feedback received by the display interface, the model parameters are adjusted, and the deep learning model and the end-to-end model are updated based on the feedback.

4. The method according to claim 2, characterized in that Before the step of semantically parsing the keywords and grammatical structures of the text information and generating the query formula based on the query language or expression after the parsing, the step further includes: Extracting the keywords and the grammatical structure of the text information based on natural language processing technology; The step of performing semantic parsing on the keywords and grammatical structures of the text information and generating the query formula based on the query language or expression after the parsing includes: Perform semantic parsing on the keywords and grammatical structure of the text information based on a sequence-to-sequence model or a graph neural network to obtain the query language or the expression; The graph structure information of the query language or the expression is determined according to a preset knowledge graph, and the graph structure information is parsed based on a preset rule engine to obtain the query formula.

5. The method according to claim 2, characterized in that When the voice processing process is triggered, the step of converting the received voice signal into text information based on the deep learning model and the end-to-end model includes: When the speech processing process is triggered, the received speech signal is preprocessed and feature extracted to obtain a feature vector; Inputting the feature vector into a deep learning model, and obtaining initial text information of the speech signal based on the deep learning model; The initial text information is subjected to text mapping processing based on an end-to-end model to obtain the text information.

6. The method according to claim 1, characterized in that After the step of acquiring historical interaction data based on the memory module, the method further includes: When the query result is empty, obtaining user preference information based on the memory module; Query processing is performed in a database according to the user preference information and the query formula to obtain the voice processing result.

7. The method according to any one of claims 6, characterized in that Before the step of acquiring historical interaction data based on the memory module, the method further includes: Constructing a long short-term memory network model, and training the long short-term memory network model with the historical interaction data and the user preference information to obtain the memory module; After the step of performing query processing on the historical interaction data based on the query formula and generating a voice processing result associated with the voice signal based on the query result when the query result is not empty, the method further includes: When a preset data update period is reached, the user preference information and the historical interaction data in the memory module are updated.

8. The method according to claim 6, characterized in that Before the step of acquiring historical interaction data based on the memory module, the method further includes: Constructing a knowledge graph corresponding to the historical interaction data; The step of acquiring historical interaction data based on the memory module includes: Based on the entity structure relationship of the knowledge graph, the user preference information and the historical interaction data are obtained in the memory module.

9. A speech processing device, characterized in that: The speech processing device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speech processing method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the speech processing method according to any one of claims 1 to 8 are implemented.

Images