Short text similarity matching voice question and answer method, refrigeration equipment and storage medium

Abstract

The application discloses a short text similarity matching voice question and answer method, a refrigeration equipment and a storage medium. The method comprises the following steps: extracting voice features and voice text features according to user voice and corresponding voice text; fusing the voice features and the voice text features to generate fusion features; using a feature similarity scoring model to score the fusion features and the fusion features in a candidate set in sequence to obtain multiple scores; if the score is greater than a first preset threshold, obtaining the candidate text corresponding to the score to obtain a candidate text set; judging whether the semantic similarity or the edit distance of each candidate text in the candidate text set and the voice text is within a preset range, deciding the processing mode based on the judgment result, and outputting the best candidate text; and obtaining the response text matched with the best candidate text and converting it into voice output. The method not only improves the quality of the conversation system, but also improves the user satisfaction.

Description

Technical Field

[0001] This invention relates to the field of refrigerator question-and-answer system technology, and in particular to a short text similarity matching method, refrigeration equipment, and storage medium based on a refrigerator question-and-answer system. Background Technology

[0002] In the current field of smart refrigerator food management, how to obtain the best response based on the user's voice in the refrigerator question-and-answer system is a common problem. The solution to this problem is generally to use text similarity algorithms. There are many methods for calculating text similarity, mainly including two aspects: one is to calculate it solely through semantic similarity, and the other is to directly use methods such as edit distance or Hamming distance to calculate the similarity distance between two texts in order to compare text similarity.

[0003] Both of the above methods have their own drawbacks. When calculating semantic similarity, since the feature extraction of text comes only from the text, if there are typos or other problems in the text, the model cannot correctly capture the text features. Edit distance judges the similarity between two texts by counting the number of operation steps involved in the transformation between the two texts. It may misjudge texts with the same semantics but different characters. For example, "add oil" and "cheer up" differ by only one character, but their semantics are different from the original text. Summary of the Invention

[0004] The purpose of this invention is to provide a voice question-and-answer method, device, and storage medium for short text similarity matching.

[0005] To achieve the above-mentioned objectives, this invention provides a voice question-answering method based on short text similarity matching. The method includes the following steps: extracting voice features and voice text features based on user voice and corresponding voice text; fusing the voice features and voice text features to generate fused features; using a feature similarity scoring model to sequentially score the fused features and the fused features in a candidate set to obtain multiple scores; if the score is greater than a first preset threshold, obtaining the candidate text corresponding to the score to obtain a candidate text set; determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the voice text is within a preset range, and based on the determination result, deciding on a processing method and outputting the best candidate text; and obtaining the matching response text based on the best candidate text and converting it into voice output.

[0006] As a further improvement of the present invention, the step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: if the semantic similarity between each candidate text in the candidate text set and the speech text is within a preset threshold range, then the edit distance between the speech text and the candidate text is calculated to obtain multiple edit distance values; the multiple edit distance values ​​are sorted to obtain the candidate text corresponding to the maximum edit distance value.

[0007] As a further improvement of the present invention, the step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: if the semantic similarity between each candidate text in the candidate text set and the speech text is greater than a second preset threshold, then the multiple semantic similarities are sorted to obtain the candidate text corresponding to the maximum semantic similarity.

[0008] As a further improvement of the present invention, the step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: if the edit distance between each candidate text in the candidate text set and the speech text is within a preset threshold range, then the semantic similarity between the speech text and the candidate text is calculated to obtain multiple similarity values; the multiple similarity values ​​are sorted to obtain the candidate text corresponding to the maximum similarity value.

[0009] As a further improvement of the present invention, the step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: if the edit distance between each candidate text in the candidate text set and the speech text is greater than a second preset threshold, then sorting the multiple edit distance values ​​to obtain the candidate text corresponding to the maximum edit distance value.

[0010] As a further improvement of the present invention, the step of "using a feature similarity scoring model to score the fusion feature and the fusion feature in the candidate set in turn to obtain multiple scores" specifically includes: obtaining a candidate speech set; performing feature extraction and feature fusion on each speech and the corresponding speech text in the candidate speech set to obtain a candidate fusion feature set; and scoring the feature similarity between each candidate fusion feature and the fusion feature in turn to obtain multiple scores.

[0011] As a further improvement of the present invention, the step of "based on user voice information and corresponding voice-text information" is preceded by: acquiring user voice data; performing voice noise reduction, voice segmentation, voice-to-text conversion, and text error correction on the voice-to-text data to obtain the voice-text information.

[0012] As a further improvement of the present invention, the "fusion of the speech features and speech-text features to generate fused features" specifically includes: performing PCA dimensionality reduction and normalization on the speech features and speech-text features; and performing fusion processing on the processed speech features and speech-text features using concat or feature overlay according to the training model.

[0013] The present invention also provides a cooling device, the cooling device including a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the program to implement the short text similarity matching voice question answering method as described above.

[0014] The present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements the short text similarity matching voice question answering method described in any of the above embodiments.

[0015] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0016] This invention selects the best response text by fusing speech and semantic features. Specifically, it scores the similarity between the fused features and candidate fused features, calculates and compares the semantic similarity and edit distance of the user's speech text and the candidate text based on the scores, and decides the best result based on the comparison results to obtain the best candidate text. Then, it searches for the matching response text based on the candidate text to obtain the best response speech. This method not only improves the quality of the conversation system but also increases user satisfaction. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the steps of a short text similarity matching voice question answering method in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the steps of a short text similarity matching voice question answering method in an embodiment of the present invention.

[0019] Figure 3 This is a flowchart of a short text similarity matching voice question answering method in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of an embodiment of a short text similarity matching voice question answering method according to an embodiment of the present invention.

[0021] Figure 5 This is a system schematic diagram of a short text similarity matching voice question answering method in an embodiment of the present invention. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0023] It should be noted that the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0024] An embodiment of the present invention is a voice question-answering method based on short text similarity matching. Although this application provides the following implementation methods or processes... Figure 1 The method described herein may involve steps that are based on conventional or non-creative labor, and in steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application.

[0025] This invention provides a short text similarity matching method for voice question answering, such as... Figure 1 As shown, the method includes the following steps, which are described below:

[0026] Step 101: Extract speech features and speech-text features based on the user's speech and the corresponding speech-text.

[0027] Step 102: Fuse the speech features and speech-text features to generate fused features.

[0028] Step 103: Using a feature similarity scoring model, the fused feature and the fused features in the candidate set are scored sequentially to obtain multiple scores.

[0029] Step 104: If the score is greater than the first preset threshold, then obtain the candidate text corresponding to the score to obtain a candidate text set.

[0030] Step 105: Determine whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range. Based on the determination result, decide on the processing method and output the best candidate text.

[0031] Step 106: Based on the best candidate text, obtain the matching response text and convert it into speech output.

[0032] In current smart refrigerator question-and-answer systems, interaction data related to food management can be used to solve the problem of providing the best response to user voice commands or questions. Specifically, the smart refrigerator first analyzes the user's voice content, and then, based on the analysis results, provides the most appropriate and reasonable response to the command or question, achieving accurate interaction between the user and the refrigerator and improving the user experience.

[0033] In step 101 of this embodiment, the user's real-time voice data is first acquired and subjected to voice denoising processing, specifically including voice noise reduction and voice segmentation processing. Voice noise reduction refers to reducing environmental noise and enhancing the intensity of the user's voice; while voice segmentation refers to segmenting the user's voice data and filtering out voice data that meets the requirements. Denoising the voice data can improve the accuracy of subsequent speech-to-text and feature extraction.

[0034] Secondly, the denoised speech data is converted into speech-to-text, converting real-time speech data into corresponding speech-to-text. During this process, the converted speech-to-text can be corrected to remove typos and generate accurate speech-to-text information.

[0035] Finally, feature extraction is performed on the user's speech and the corresponding speech-text. When extracting speech features, the first step is to analyze and extract feature parameters that represent the essence of the speech. Then, the extracted feature parameters are normalized or standardized to obtain a speech feature vector. Similarly, in this embodiment of the invention, after extracting features from the speech-text using a deep learning model or other text feature extraction tools, PCA dimensionality reduction is used to ensure that the resulting text features have the same dimension as the speech features. Furthermore, the extracted text features are also normalized or standardized to obtain a speech-text feature vector.

[0036] In step 102 of this embodiment, the speech feature vector and the corresponding speech-text feature vector obtained in step 101 are fused to obtain a fused feature vector. Fusing features at different scales is an important means to improve segmentation performance. This invention can use concat or feature superposition methods to achieve feature fusion. The specific fusion method can be adjusted to match the specific training model. Concat is a series feature fusion that directly connects two feature vectors. The dimension of the fused feature vector is the sum of the dimensions of the two feature vectors before fusion. Feature superposition is a parallel strategy that superimposes the two input feature vectors.

[0037] In steps 103 and 104 of this embodiment, a candidate question-and-answer database is queried based on the voice information provided by the user. This database contains many candidate question-and-answer pairs of voice data, where a question-and-answer pair refers to a question corresponding to a response. Specifically, a speech-to-text conversion operation is performed on each piece of voice data to obtain matching speech-to-text data. Speech features and speech-to-text features are extracted and normalized for each candidate piece of voice data and its corresponding candidate speech-to-text data, resulting in multiple candidate speech feature vectors and candidate speech-to-text feature vectors. Then, feature fusion is performed on each candidate speech feature vector and candidate speech-to-text feature vector to generate multiple candidate fused feature vectors, thereby obtaining the candidate fused feature set.

[0038] In this embodiment of the invention, each candidate fusion feature in the candidate fusion feature set is compared with the fusion feature for similarity, and a feature similarity scoring model is used to score each feature, resulting in multiple scoring results. The relationship between each scoring result and a first preset threshold is determined. If the scoring result is greater than the first preset threshold, the candidate speech text corresponding to that scoring result is obtained to obtain a candidate text set; if the scoring result is not greater than the first preset threshold, the candidate text corresponding to that scoring result is filtered out.

[0039] In steps 105 and 106 of this embodiment, the candidate text set obtained in steps 103 and 104 is a further screening of the voice question-and-answer data in the question-and-answer knowledge base, filtering out voice question-and-answer data with low similarity and retaining candidate voice data that is closer to the user's voice data. To obtain more accurate candidate texts, it is necessary to further determine whether the semantic similarity or edit distance between each candidate text in the candidate text set and the voice text is within a preset range. Based on the determination result, a processing method is determined to obtain the corresponding candidate text. Then, relevant question-and-answer pairs are searched based on the candidate text to find the best response text and perform speech conversion to generate the corresponding voice data and output it.

[0040] like Figure 2 As shown, step 105 above is further refined. Based on the relationship between the semantic similarity or edit distance and the preset range, the range of candidate text sets is further narrowed down, specifically including the following steps:

[0041] Step 1051: If the semantic similarity is within a preset threshold range, calculate the corresponding edit distance and obtain the candidate text based on the edit distance.

[0042] Step 1052: If the semantic similarity is greater than the second preset threshold, then obtain the corresponding candidate text based on the semantic similarity.

[0043] In this embodiment of the invention, the candidate text set is traversed, and the semantic similarity between each candidate text and the current user's voice text is calculated sequentially to obtain multiple semantic similarity values. Each semantic similarity value is then determined to be within a preset threshold range. If it is, the edit distance between the candidate text and the user's voice text is further calculated; otherwise, the corresponding candidate text is deleted from the candidate text set, thereby further narrowing down the candidate text set. Then, the multiple edit distance values ​​are sorted, and the corresponding candidate text is obtained based on the maximum edit distance value. The corresponding response text is then found based on this candidate text, which is the most accurate reply to the current user's voice text. This response text is then converted into corresponding voice data and output. In this embodiment of the invention, a preset threshold is preferably selected as 0.6, and the preset threshold range is preferably selected as [0.6, 0.9], while the second preset threshold is preferably 0.9. Furthermore, to match the size and range of the preset threshold, the edit distance in this embodiment of the invention is the reciprocal of the actual edit distance calculated between the two texts, ensuring that its range is between 0 and 1.

[0044] If the semantic similarity is not within the preset threshold range but is greater than the second preset threshold, then the multiple semantic similarity values ​​generated by the candidate text set are sorted, and the candidate text with the largest semantic similarity value is selected as the best candidate text, thereby obtaining the best response text.

[0045] Step 1053: If the edit distance is within the preset threshold range, calculate the corresponding semantic similarity and obtain the candidate response text based on the semantic similarity.

[0046] Step 1054: If the edit distance is greater than the second preset threshold, then obtain the corresponding candidate text based on the edit distance.

[0047] On the other hand, in this embodiment of the invention, if the edit distance value between each candidate text and the current user's speech text is calculated first, further filtering of candidate texts is achieved by calculating their corresponding semantic similarity. Specifically, the candidate text set is traversed, and the edit distance between each candidate response text and the current user's speech text is calculated sequentially to obtain multiple edit distance values. Each edit distance value is then determined to be within a preset threshold range. If it is, the semantic similarity between the candidate response text and the user's speech text is further calculated; otherwise, the corresponding candidate text is deleted from the candidate response text, thereby further narrowing down the candidate text set. Next, the multiple semantic similarity values ​​are sorted, and the corresponding candidate text is obtained based on the maximum semantic similarity value. The corresponding response text is then obtained based on this candidate text, which is the optimal response text to the current user's speech text.

[0048] If the edit distance is not within the preset threshold range but is greater than the second preset threshold, then the multiple edit distance values ​​generated by the candidate text set are sorted, and the candidate text with the largest edit distance value is selected as the best candidate text, thereby obtaining the corresponding best response speech.

[0049] Figure 3 The diagram shown is a flowchart of a short text similarity matching voice question answering method according to the present invention. Figure 3 In the scenario shown, user voice is acquired and converted to text to obtain corresponding text. Features are extracted from the voice and text to obtain voice features and text features, respectively. A feature fusion method is then used to fuse these two features to obtain a fused feature. Furthermore, multiple candidate response voices and their corresponding candidate response texts from a question-answering knowledge base are combined, and their voice features and text features are extracted and fused separately to generate a candidate fused feature set containing multiple different candidate fused features.

[0050] In this embodiment of the invention, a feature similarity scoring model is used to score the feature similarity between each candidate fusion feature in the candidate fusion feature set and the fusion feature, thereby obtaining multiple scoring results. Then, the range of candidate response texts is further narrowed by judging the relationship between the scoring results and the first preset threshold. Specifically, candidate response texts with scoring results less than or equal to the first preset threshold are filtered out, and candidate texts with scoring results greater than the first preset threshold are retained to obtain a candidate text set. The first preset threshold can be adjusted accordingly based on the actual situation.

[0051] The candidate text set is further filtered. Specifically, the semantic similarity between each element in the candidate text set and the user's voice text is calculated to obtain multiple semantic similarity values. Each semantic similarity value is evaluated, and candidate texts below a certain preset threshold are filtered out. Then, it is determined whether the semantic similarity is within the preset threshold range. If so, the edit distance between the corresponding candidate text and the user's voice text is calculated. The multiple edit distance values ​​are then sorted, and the candidate text corresponding to the largest edit distance value is selected as the best candidate text. If the semantic similarity is greater than a second preset threshold, the multiple semantic similarities are directly sorted, and the candidate text corresponding to the largest semantic similarity is selected as the best candidate text. Finally, the best response voice is output based on the best candidate text.

[0052] On the other hand, such as Figure 4As shown, the calculation order of semantic similarity and edit distance has been reversed; the other processes are similar and will not be elaborated on here. Specifically, the edit distance between multiple candidate texts and the user's speech text is calculated sequentially. Candidate texts with edit distances below a certain set threshold are filtered out. Then, it is determined whether the edit distance value is within the preset threshold range. If so, the semantic similarity between the candidate text and the user's speech text is further calculated. The multiple semantic similarity values ​​are then sorted, and the candidate text corresponding to the highest semantic similarity is obtained as the best candidate text. If the edit distance is greater than the second preset threshold, the multiple edit distance values ​​corresponding to the candidate text are directly sorted, and the candidate text corresponding to the highest edit distance value is obtained as the best candidate text. Finally, the best response speech is output based on the best candidate text.

[0053] This invention also provides a short text similarity matching voice question answering system, which includes a voice module, a feature extraction module, a decision module, and a text conversion module, as detailed in the following embodiments. Figure 5 As shown. Figure 5 The speech module shown receives user speech data, performs speech denoising and speech-to-text conversion, and then corrects errors in the converted speech-to-text to obtain accurate speech-to-text. Next, a feature extraction module extracts speech features and speech-to-text features, and simultaneously fuses these features. Then, a decision module uses a feature similarity scoring model to score the similarity between the fused features and candidate fused features, and further determines the best candidate text and the best response text based on the scoring results. Finally, the speech-to-text conversion module converts the response text into the corresponding response speech.

[0054] This invention also provides a cooling device, which includes a memory and a processor. The memory stores instructions, and the processor calls the instructions in the memory to enable the cooling device to implement a short text similarity matching voice question answering method as described above.

[0055] This invention also provides a storage medium storing a computer program that, when executed by a processor, implements a short text similarity matching voice question-answering method as described in any of the above embodiments.

[0056] In summary, this invention provides a short text similarity matching method for voice question answering, a cooling device, and a storage medium. By fusing voice features and text features, it further matches the most similar candidate texts. In addition, it uses a feature similarity scoring mechanism, semantic similarity, and edit distance to determine the optimal candidate text, thereby obtaining the corresponding response text. Then, it performs post-processing, text conversion to generate the corresponding voice, and outputs it. This method not only improves the quality of the conversation system but also enhances the accuracy of conversation question responses, thereby increasing user satisfaction.

[0057] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0058] The detailed descriptions listed above are merely specific descriptions of feasible implementations of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent implementations or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A short text similarity matching-based voice question answering method, characterized in that, Includes the following steps: Extract speech features and speech-text features based on user speech and corresponding speech-text; The speech features and speech-text features are fused to generate fused features; Using a feature similarity scoring model, the fused feature and the fused features in the candidate set are scored sequentially to obtain multiple scores; If the score is greater than the first preset threshold, then the candidate text corresponding to the score is obtained to obtain a candidate text set; Determine whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range. Based on the determination result, decide on the processing method and output the best candidate text. According to the best candidate text, obtain the matching response text and convert it into speech output. Specifically, the phrase "using a feature similarity scoring model to sequentially score the fused feature and the fused features in the candidate set to obtain multiple scores" includes: Obtain a candidate speech set; perform feature extraction and feature fusion on each speech and its corresponding speech text in the candidate speech set to obtain a candidate fused feature set; sequentially score the feature similarity between each candidate fused feature and the fused feature to obtain multiple scores; The phrase "fusing the speech features and speech-text features to generate fused features" specifically includes: The speech features and speech-text features are subjected to PCA dimensionality reduction and normalization; based on the trained model, the processed speech features and speech-text features are fused using concat or feature overlay. The step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: If the semantic similarity between each candidate text in the candidate text set and the speech text is within a preset threshold range, then the edit distance between the speech text and the candidate text is calculated to obtain multiple edit distance values; the multiple edit distance values ​​are then sorted to obtain the candidate text corresponding to the maximum edit distance value; If the edit distance between each candidate text in the candidate text set and the speech text is within a preset threshold range, then the semantic similarity between the speech text and the candidate text is calculated to obtain multiple similarity values; the multiple similarity values ​​are sorted to obtain the candidate text corresponding to the maximum similarity value.

2. The short text similarity matching voice question answering method according to claim 1, characterized in that... The step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: If the semantic similarity between each candidate text in the candidate text set and the speech text is greater than a second preset threshold, then the multiple semantic similarities are sorted to obtain the candidate text corresponding to the maximum semantic similarity.

3. The short text similarity matching voice question answering method according to claim 1, characterized in that... The step of "determining whether the semantic similarity or edit distance between each candidate text in the candidate text set and the speech text is within a preset range" specifically includes: If the edit distance between each candidate text in the candidate text set and the speech text is greater than a second preset threshold, then the multiple edit distance values ​​are sorted to obtain the candidate text corresponding to the maximum edit distance value.

4. The short text similarity matching voice question answering method according to claim 1, characterized in that... The step of "based on user voice information and corresponding voice-text information" includes the following before: Obtain user voice data; The speech data is subjected to speech noise reduction, speech truncation, speech-to-text conversion, and text correction after speech-to-text conversion to obtain the speech-text information.

5. A refrigeration device, characterized in that, The refrigeration device includes a memory and a processor. The memory stores a computer program that runs on the processor, and the processor executes the program to implement claim 1. A short text similarity matching voice question answering method as described in any one of the four.

6. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements claim 1.

4. A short text similarity matching voice question answering method as described in any one of the above.

Images