Voice processing method, voice processing device and storage medium

Abstract

The present application discloses a speech processing method, a speech processing device, and a storage medium, and relates to the field of speech recognition technology. The speech processing method includes: when triggering a speech processing process, determining a query formula corresponding to a received speech signal; obtaining historical interaction data based on a memory module; performing query processing on the historical interaction data based on the query formula, and when the query result is not empty, generating a speech processing result associated with the speech signal based on the query result. When performing speech processing, the present application converts the speech signal into a query formula, and performs calculation processing in the background based on the query formula. During the background calculation, the existing solution can be reused for processing, so that the processing result conforms to the user's pronunciation habits, usage habits, and actual needs, thereby improving the accuracy of speech question and answer processing.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech processing method, a speech processing device, and a storage medium. Background Art

[0002] In current voice question-answering systems, although automated speech-to-text conversion has been implemented, existing speech recognition systems may not be able to accurately capture speech signals when faced with different accents, speaking speeds, and intonations, resulting in misrecognition or missed recognition. Background noise and echoes can also affect the accuracy of speech recognition, resulting in low accuracy in the results obtained based on voice question-answering.

[0003] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a speech processing method, a speech processing device and a storage medium, aiming to solve the technical problem of low accuracy of the results obtained based on voice question and answer.

[0005] To achieve the above objectives, the present application proposes a speech processing method, which includes:

[0006] When the voice processing process is triggered, the query formula corresponding to the received voice signal is determined;

[0007] Obtain historical interaction data based on the memory module;

[0008] Query processing is performed on the historical interaction data based on the query formula, and when a query result is not empty, a voice processing result associated with the voice signal is generated based on the query result.

[0009] In one embodiment, when the voice processing process is triggered, the step of determining the query formula corresponding to the received voice signal includes:

[0010] When the voice processing process is triggered, the received voice signal is converted into text information based on the deep learning model and the end-to-end model;

[0011] Semantic parsing is performed on the keywords and grammatical structure of the text information, and the query formula is generated based on the query language or expression after the parsing.

[0012] In one embodiment, after the step of performing query processing on the historical interaction data based on the query formula and generating a speech processing result associated with the speech signal based on the query result when the query result is not empty, the method further includes:

[0013] generating query keywords and query tags based on the query results;

[0014] Input the query keyword and the query tag into the display interface;

[0015] Adjust model parameters in response to feedback received by the display interface, and update the deep learning model and the end-to-end model based on the feedback-adjusted model parameters.

[0016] In one embodiment, before the step of performing semantic parsing on the keywords and grammatical structure of the text information and generating the query formula based on the query language or expression after the parsing, the method further includes:

[0017] extracting the keywords and the grammatical structure of the text information based on natural language processing technology;

[0018] The step of performing semantic parsing on the keywords and grammatical structure of the text information and generating the query formula based on the query language or expression after the parsing includes:

[0019] Performing semantic parsing on the keywords and grammatical structure of the text information based on a sequence-to-sequence model or a graph neural network to obtain the query language or the expression;

[0020] The graph structure information of the query language or the expression is determined according to a preset knowledge graph, and the graph structure information is parsed based on a preset rule engine to obtain the query formula.

[0021] In one embodiment, when the voice processing process is triggered, the step of converting the received voice signal into text information based on the deep learning model and the end-to-end model includes:

[0022] When the voice processing process is triggered, the received voice signal is preprocessed and feature extracted to obtain a feature vector;

[0023] Inputting the feature vector into a deep learning model, and obtaining initial text information of the speech signal based on the deep learning model;

[0024] The initial text information is subjected to text mapping processing based on an end-to-end model to obtain the text information.

[0025] In one embodiment, after the step of acquiring historical interaction data based on the memory module, the method further includes:

[0026] When the query result is empty, obtaining user preference information based on the memory module;

[0027] Query processing is performed in a database according to the user preference information and the query formula to obtain the voice processing result.

[0028] In one embodiment, before the step of acquiring historical interaction data based on the memory module, the method further includes:

[0029] Constructing a long short-term memory network model, and training the long short-term memory network model with the historical interaction data and the user preference information to obtain the memory module;

[0030] After the step of performing query processing on the historical interaction data based on the query formula and generating a voice processing result associated with the voice signal based on the query result when the query result is not empty, the method further includes:

[0031] When a preset data update period is reached, the user preference information and the historical interaction data in the memory module are updated.

[0032] In one embodiment, before the step of acquiring historical interaction data based on the memory module, the method further includes:

[0033] Constructing a knowledge graph corresponding to the historical interaction data;

[0034] The step of acquiring historical interaction data based on the memory module includes:

[0035] Based on the entity structure relationship of the knowledge graph, the user preference information and the historical interaction data are obtained in the memory module.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a speech processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speech processing method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the speech processing method described above are implemented.

[0038] One or more technical solutions proposed in this application have at least the following technical effects:

[0039] When performing voice processing, the voice signal is first converted into a query formula and background calculations are performed. During the calculation, repeated calculations are reduced by reusing past solutions. The query processing is performed by utilizing the user historical interaction data stored in the memory module. During the query process, the existing solutions are reused, which can avoid the complex processing of the entire voice data and improve the accuracy of the voice processing results based on the existing solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 A flowchart of the first embodiment of the speech processing method provided by the present application;

[0043] Figure 2 A flowchart of a second embodiment of the speech processing method of the present application is provided;

[0044] Figure 3 A flowchart of a third embodiment of the speech processing method of the present application;

[0045] Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the voice processing method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0048] The main solution of the embodiment of the present application is: when the voice processing process is triggered, the query formula corresponding to the received voice signal is determined;

[0049] Obtain historical interaction data based on the memory module;

[0050] Query processing is performed on the historical interaction data based on the query formula, and when a query result is not empty, a voice processing result associated with the voice signal is generated based on the query result.

[0051] In this embodiment, for ease of description, the following description is made with the voice recognition device as the execution subject.

[0052] In current voice question-answering systems, although automated speech-to-text conversion has been implemented, existing speech recognition systems may not be able to accurately capture speech signals when faced with different accents, speaking speeds, and intonations, resulting in misrecognition or missed recognition. Background noise and echoes can also affect the accuracy of speech recognition, resulting in low accuracy in the results obtained based on voice question-answering.

[0053] The present application provides a solution that, when performing voice processing, first converts the voice signal into a query formula, and performs background operations based on the query formula. During the operation, historical interaction data is obtained through a memory module so that past solutions can be reused based on historical interaction data, reducing repeated calculations. As a result, during the query process, the processing model is more adapted to the user's pronunciation habits, usage habits, and actual needs, thereby improving the accuracy of the voice processing results.

[0054] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device or voice processing device capable of implementing the above functions. The following uses a voice processing device as an example to illustrate this embodiment and the following embodiments.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The present application provides a method for voice processing. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the speech processing method of the present application.

[0057] In this embodiment, the speech processing method includes steps S10 to S30:

[0058] Step S10, when the voice processing process is triggered, determining the query formula corresponding to the received voice signal;

[0059] In this embodiment, after the voice processing device is running and receives a voice signal, it lacks a voice processing process, and thus needs to convert the received voice signal into text information. During this process, the voice signal needs to be adjusted so that the processing model can recognize the signal and query accurately. The query formula can include a SQL (Structured Query Language) query of the database, an API (Application Programming Interface) request, a SPARQL (SPARQL Protocol and RDF Query Language) query, or a custom query format.

[0060] As an optional implementation, taking SQL queries as an example, when determining the query formula corresponding to a voice signal, a deep learning speech recognition model can be used to extract features from the voice signal and convert these features into text information. This process can directly achieve audio-to-text mapping using an end-to-end model. Once the text information is obtained, natural language processing techniques are used to parse the text content and generate a query formula based on the parsed content.

[0061] Optionally, speech signals can also be processed based on convolutional neural networks and recurrent neural networks to improve recognition performance by simultaneously capturing local features and global context information of speech.

[0062] In this embodiment, by converting speech into text and then converting text into a query formula, the speech processing device can perform calculations and processing in the background based on the query formula, thereby improving the convenience and recognition efficiency of speech recognition operations. At the same time, when converting text into a query formula, deep learning, natural language processing (NLP), semantic analysis, and database query processing are integrated to form a highly intelligent processing system, enabling the speech processing device to handle complex speech processing tasks.

[0063] Step S20, acquiring historical interaction data based on the memory module;

[0064] It should be noted that when acquiring historical interaction data, the system of the voice processing device needs to perform data query processing based on the knowledge graph constructed based on the historical interaction data. Among them, the memory module is a storage module in the voice processing device used to store relevant operation information, which assists the system in understanding and predicting user needs. Based on the memory module, the processing system of the voice processing device can remember historical interaction data, and use the knowledge graph maintenance and query mode to select historical query content associated with the query formula, so that past solutions can be reused to reduce repeated calculations. At the same time, more reasonable decisions can be made based on historical data, thereby improving the accuracy of voice processing results. Among them, in addition to storing historical interaction data, the memory module also stores user preference information.

[0065] In this embodiment, the voice processing device also includes a knowledge graph for storing and retrieving historical data. Therefore, when obtaining historical interaction data based on the memory module, it is necessary to maintain and query the historical data through the knowledge graph. That is, before this step, it is also necessary to construct a knowledge graph corresponding to the historical interaction data, and then obtain the historical interaction data in the memory module based on the entity structure relationship in the knowledge graph.

[0066] Optionally, the memory module can be constructed through a long short-term memory network (LSTM), that is, after constructing the long short-term memory network model, the long short-term memory network model is trained based on the historical interaction data and user preference information to obtain the memory module. It should be noted that after the long short-term memory network model is trained, it can be regarded as a memory module because it has learned the representation of historical interaction data and user preference information.

[0067] Optionally, in order to avoid outdated information from interfering with the voice processing equipment, it is also necessary to regularly update and optimize the content stored in the memory module. Based on this, after obtaining the voice processing results, when the preset data update period is reached, the user preference information and historical interaction data of the memory module need to be updated. The updating method can be to optimize the data based on the voice signal received in real time.

[0068] It should be noted that in the process of constructing the knowledge graph, the entity structure relationship of the knowledge graph can be constructed based on historical interaction data. Then, when it is necessary to determine whether the current query formula has a solution in a similar scenario, the entity structure relationship of the knowledge graph can be used to search and process from the corresponding node, and then the relevant historical interaction data and user preference information can be obtained. When there is a target formula associated with the query formula in the queried historical interaction data, the existing solution can be reused to improve the accuracy of the speech processing results.

[0069] Step S30: performing query processing on the historical interaction data based on the query formula, and when the query result is not empty, generating a voice processing result associated with the voice signal based on the query result.

[0070] In this embodiment, query processing is performed on the historical interaction data based on the query formula. If the query result is not empty, it indicates that in the current voice processing process, a similar solution case already exists for the content to be queried corresponding to the voice signal. For example, the generated query formula is an SQL statement containing the query terms "Lease Agreement" and "Default", and the historical interaction data contains a similar SQL statement with the target formula containing the terms "Lease Agreement", "Default", and "Consequence". In this case, the query result is not empty. Based on this, it is necessary to generate voice processing based on the query result, that is, based on the existing solution, search the database for the query result associated with the voice signal.

[0071] For example, consider a legal consulting company with a vast amount of historical case data in its memory module, including client questions, provided answers, and final legal resolutions. This company first uses this historical interaction data to construct a knowledge graph to improve service efficiency and quality. During this construction process, the company first analyzes the historical interaction data to extract key entities such as case type, involved legal clauses, client questions, and resolutions. Relationships between these entities are then defined, such as "case type - involved - legal clauses" or "client question - result - resolution." Based on these entities and relationships, a legal knowledge graph is constructed. Subsequently, when a user voice-generated legal inquiry is received, the voice signal is converted into a corresponding query formula. The query formula is then parsed to identify key entities such as case type and specific question. SQL statements are then used to search the knowledge graph for cases similar to the new case, i.e., the user's inquiry regarding the legal consequences of breaching a lease contract.

[0072] If the system finds a target formula associated with the current query formula (similar cases identified using SQL) in historical interaction data, meaning similar cases previously processed with non-empty query results, it can reuse the solutions for those cases. For example, the system might discover a similar lease contract breach case and successfully apply a specific interpretation of the legal clause and solution.

[0073] In this way, voice processing devices can quickly provide accurate recommendations, improving the accuracy of voice processing results. By leveraging knowledge graph maintenance and query patterns, existing solutions can be reused to reduce repetitive calculations. This also enables more reasonable decisions based on historical data, further improving the accuracy of voice processing results.

[0074] In another optional embodiment, when the query result is empty, the user preference information is obtained based on the memory module, and then query processing is performed in the database according to the user preference information and the query formula to obtain the voice processing result. Specifically, there is no target formula corresponding to the query formula in the historical interaction data, which means that there is no relevant solution recorded in the current historical interaction data. At this time, the corresponding voice processing result can be directly queried and output based on the user preference information and the current query formula, thereby improving the accuracy of the voice processing result obtained by the voice processing device. In the process of query processing, SQL or other data access methods are used to retrieve data from the database, and then mathematical operations, statistical analysis, machine learning model prediction and other operations are performed. It should be noted that during the query process, it is also necessary to connect to the database to be queried. At the same time, if the amount of data query is large, tasks can be processed in parallel or based on an asynchronous execution mechanism to improve processing efficiency.

[0075] This embodiment provides a speech processing method. After the speech processing process is triggered, the speech information is first converted into a corresponding query formula. Then, based on the constructed knowledge graph, historical interaction data is obtained from the memory module. Then, based on the query formula, a search is performed in the historical interaction data to determine whether there is an existing solution. When an existing solution exists, the data calculation process is reduced by directly reusing the existing solution. At the same time, the existing solution is verified by the speech processing device, which can effectively improve the accuracy of the speech processing results.

[0076] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be described in detail later. Figure 2 , step S10 also includes steps S11~S12:

[0077] Step S11, when the voice processing process is triggered, the received voice signal is converted into text information based on the deep learning model and the end-to-end model;

[0078] In this embodiment, in the process of converting the voice signal into text information, the voice signal is firstly subjected to feature extraction processing using a deep learning speech recognition model, and these features are then converted into text information.

[0079] Specifically, when the speech processing process is triggered, the received speech signal is preprocessed and feature extracted to obtain a feature vector. The preprocessing process includes audio normalization, noise reduction, and silence detection, while the feature extraction process involves extracting Mel-Frequency Cepstral Coefficients (MFCCs), Mel-spectrograms, and linear spectrum logarithms. The extracted feature vector is then input into a deep learning model, which is used to obtain the initial text information of the speech signal. Then, based on the end-to-end model, text mapping is performed on the initial text to obtain the text information.

[0080] Among them, during the training process of deep learning models, GPU can be used to accelerate processing when training based on large-scale labeled data.

[0081] Step S12: semantically analyze the keywords and grammatical structure of the text information, and generate the query formula based on the query language or expression after the analysis.

[0082] In this embodiment, natural language processing techniques can be used to extract keywords and grammatical structures from text information. Semantic parsing of the keywords and grammatical structures is then performed on the text information, thereby converting the natural language into a structured query language or expression. Finally, a query formula is generated based on the resulting query language or expression. The natural language processing techniques include lexical analysis, such as word segmentation and part-of-speech tagging, and syntactic analysis, such as data dependency analysis or component structure tree analysis. During the semantic parsing process, the analysis results are converted into a query language, such as SQL, SPARQL, or a custom query format.

[0083] Furthermore, during the semantic processing process, a sequence-to-sequence (Seq2Seq) model or a graph neural network (GNN) can be used for semantic parsing training. When determining the query formula, the rule engine and knowledge graph can be used to assist in understanding the context and relationships of the parsed analysis results, thereby generating an accurate query formula.

[0084] Specifically, semantic parsing is performed on the keywords and grammatical structure of the text information based on a sequence-to-sequence model or a graph neural network to obtain the query language or the expression, and then the graph structure information of the query language or expression is determined based on a preset knowledge graph, and the graph structure information is parsed based on a preset rule engine to obtain the query formula.

[0085] For example, in a financial application, a user uses natural language to inquire about stock information. The processing system of the voice processing device needs to convert these inquiries into SQL query statements to extract the corresponding information from the database. The user inputs the following voice information: "I want to know the performance of Company A's stock in 20xx." Based on this, during lexical analysis, the sentence is broken down into words or symbols, namely, "I" want to "know" the "performance" of "Company A's" stock in "20xx." Part-of-speech tagging is then performed to identify the part of speech of each word, where "know" is a verb and "Company A" is a noun phrase. Dependency analysis is then performed to determine the grammatical structure between words, where "know" depends on "want" (indicating a purpose), and the performance of "stocks" is what the user wants to know.

[0086] During semantic processing, a tree diagram representing the sentence structure is constructed to clarify the grammatical components (subject, predicate, and object) and their relationships. Natural language is then converted into meaningful expressions, transforming the sentence into the intent, or query language: "Query stock performance." The query language is further mapped to a structured query statement, such as SQL: "SELECT performance FROM stocks WHERE name = 'A Inc.' AND year = 20xx;"

[0087] Furthermore, if there are domain-specific rules (such as stock data only includes stock data from recent years), the query statement will be adjusted according to the rules. At the same time, in this process, the relationships in the knowledge graph can be used to help understand complex queries, such as linking "Company A" to its stock code, historical data, etc.

[0088] During model training, a Seq2Seq model is used to receive natural language input and then output the target query language (such as SQL). Training is performed using a large number of annotated natural language and SQL query pairs to learn how to accurately convert. Optionally, a GNN can be used to handle complex dependencies and improve parsing accuracy.

[0089] This embodiment provides a speech processing method that, after triggering the speech processing process, integrates deep learning, natural language processing, semantic parsing, and other processing processes to preprocess, extract features, and map text to the speech signal. Simultaneously, based on a knowledge graph and rule engine, it generates corresponding query formulas, improving the accuracy of the query formulas. Furthermore, the entire process, from speech recognition to result feedback and optimization, is designed as a coherent chain, automatically completing the entire process from user input of voice commands to result feedback. This end-to-end design reduces manual intervention and improves the system's automation and efficiency.

[0090] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be described in detail later. Figure 3 After step S30, steps S40 to S60 are also included:

[0091] Step S40: generating query keywords and query tags based on the query results;

[0092] Step S50: input the query keyword and the query tag into the display interface;

[0093] Step S60: adjusting model parameters in response to feedback received by the display interface, and updating the deep learning model and the end-to-end model based on the feedback-adjusted model parameters.

[0094] In this embodiment, the processing system of the voice processing device can generate tags and keywords based on the query results, thereby facilitating users to quickly understand the current results, while increasing the flexibility and interactivity of the system, allowing users to intuitively filter information.

[0095] Therefore, after generating query keywords and query tags, they can be output to a display interface for user filtering. When adjusting model parameters in response to user feedback input in the display interface, the deep learning model and end-to-end model are updated based on the feedback. This means using supervised or reinforcement learning algorithms to adjust model parameters based on user selection feedback.

[0096] Based on this, by collecting user feedback information and using the feedback information for model retraining, the learning strategy can be effectively strengthened to optimize the long-term performance of the model.

[0097] This embodiment provides a speech processing method that dynamically generates tags and keywords based on query results and outputs the generated tags and keywords, making it easier for users to quickly understand the current results and enhancing the flexibility and interactivity of the system. At the same time, the model is continuously adjusted and improved through user feedback and historical data, enabling the system to gradually understand and adapt to user habits and preferences, providing more personalized and accurate services, and thereby improving the accuracy of speech processing results.

[0098] The present application provides a speech processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the speech processing method in the above-mentioned first embodiment.

[0099] Reference below Figure 4 , which shows a schematic diagram of the structure of a speech processing device suitable for implementing the embodiments of the present application. The speech processing device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The voice processing device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application.

[0100] like Figure 4 As shown, the speech processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the speech processing device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007, such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008, such as a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003, such as a magnetic tape or hard disk; and communication device 1009. The communication device 1009 can allow the voice processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a voice processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0101] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0102] The speech processing device provided in this application, employing the speech processing method of the aforementioned embodiment, can resolve the technical issue of low accuracy in results obtained from voice question-and-answering. Compared to the prior art, the beneficial effects of the speech processing device provided in this application are the same as those of the speech processing method provided in the aforementioned embodiment. Other technical features of the speech processing device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0103] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0104] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0105] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the speech processing method in the above-mentioned embodiment.

[0106] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0107] The computer-readable storage medium may be included in the speech processing device, or may exist independently without being incorporated into the speech processing device.

[0108] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the speech processing device, the speech processing device:

[0109] Determining a query formula corresponding to the received voice signal;

[0110] Obtain historical interaction data based on the memory module;

[0111] Query processing is performed on the historical interaction data based on the query formula, and when a query result is not empty, a voice processing result associated with the voice signal is generated based on the query result.

[0112] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0114] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0115] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned speech processing method. This computer-readable storage medium can address the technical issue of low accuracy in results obtained through speech question-and-answering. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the speech processing method provided in the aforementioned embodiments and are not further elaborated here.

[0116] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned speech processing method when executed by a processor.

[0117] The computer program product provided in this application can solve the technical problem of low accuracy of results obtained based on voice question-answering. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the voice processing method provided in the above embodiment, and will not be repeated here.

[0118] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A speech processing method, characterized in that: The method comprises: When the voice processing process is triggered, the received voice signal is converted into text information based on the deep learning model and end-to-end model; Extracting keywords and grammatical structures of the text information based on natural language processing technology; Perform semantic parsing on the keywords and grammatical structure of the text information based on a sequence-to-sequence model or a graph neural network to obtain a query language or expression; Determine the graph structure information of the query language or the expression according to a preset knowledge graph, and parse the graph structure information based on a preset rule engine to obtain a query formula; Obtain historical interaction data based on the memory module; Query processing is performed on the historical interaction data based on the query formula, and when a query result is not empty, a voice processing result associated with the voice signal is generated based on the query result.

2. The method according to claim 1, wherein After the step of performing query processing on the historical interaction data based on the query formula and generating a voice processing result associated with the voice signal based on the query result when the query result is not empty, the method further includes: generating query keywords and query tags based on the query results; Input the query keyword and the query tag into the display interface; Adjust model parameters in response to feedback received by the display interface, and update the deep learning model and the end-to-end model based on the feedback-adjusted model parameters.

3. The method according to claim 1, wherein When the voice processing process is triggered, the step of converting the received voice signal into text information based on the deep learning model and the end-to-end model includes: When the voice processing process is triggered, the received voice signal is preprocessed and feature extracted to obtain a feature vector; Inputting the feature vector into a deep learning model, and obtaining initial text information of the speech signal based on the deep learning model; The initial text information is subjected to text mapping processing based on an end-to-end model to obtain the text information.

4. The method according to claim 1, wherein After the step of acquiring historical interaction data based on the memory module, the method further includes: When the query result is empty, obtaining user preference information based on the memory module; Query processing is performed in a database according to the user preference information and the query formula to obtain the voice processing result.

5. The method according to claim 4, wherein Before the step of acquiring historical interaction data based on the memory module, the method further includes: Constructing a long short-term memory network model, and training the long short-term memory network model with the historical interaction data and the user preference information to obtain the memory module; After the step of performing query processing on the historical interaction data based on the query formula and generating a voice processing result associated with the voice signal based on the query result when the query result is not empty, the method further includes: When a preset data update period is reached, the user preference information and the historical interaction data in the memory module are updated.

6. The method according to claim 5, wherein Before the step of acquiring historical interaction data based on the memory module, the method further includes: Constructing a knowledge graph corresponding to the historical interaction data; The step of acquiring historical interaction data based on the memory module includes: Based on the entity structure relationship of the knowledge graph, the user preference information and the historical interaction data are obtained in the memory module.

7. A speech processing device, characterized in that: The speech processing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speech processing method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the speech processing method according to any one of claims 1 to 6 are implemented.

Images