Dual-layer speech search method, system and device based on semantic fluency and relevance

Abstract

The application belongs to the technical field of artificial intelligence, and particularly relates to a double-layer voice search method, system and device based on semantic fluency and correlation. The method comprises the following steps: obtaining user voice and converting the user voice into an initial text sequence, generating a plurality of query candidate texts based on voice recognition confidence; evaluating the fluency scores of each candidate text by using a deep semantic model, and screening out a candidate set to be searched; for each candidate to be searched, adaptively determining a search depth according to the confidence level thereof, using a dense search model to obtain a candidate document and a first correlation score thereof; fusing the semantic fluency score of the candidate text and the first correlation score of the document, calculating a second correlation score and performing preliminary sorting; finally, combining user historical interaction information to perform context-aware reordering on the preliminary sorting result, and outputting a final recommended document. The application combines deep semantic understanding and adaptive search sorting, and effectively improves the accuracy and individualization degree of voice search.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a two-layer speech search method, system, and device based on semantic fluency and relevance. Background Technology

[0002] Currently, with the deep integration of speech recognition and information retrieval technologies, voice search has become an important method of human-computer interaction. Among existing technologies, a Chinese invention patent (publication number CN113722447B) proposes a voice search method based on multi-strategy matching. This method first converts the user's speech into text using a speech recognition module. If homophones exist, all possible text candidates are generated, and an n-gram-based language fluency model is used to score the candidate texts, selecting the top K highest-scoring texts as query candidates. Subsequently, each query candidate text is retrieved separately, and the top N documents for each query are retrieved. A text matching model is used to score the relevance between the query and the documents, resulting in K×N documents and their relevance scores. Finally, the fluency scores of the query candidate texts and the relevance scores of the documents are combined to calculate the comprehensive score of the documents and rank them, recommending the highest-scoring document to the user. This method, through multi-candidate generation and multi-strategy matching, improves the accuracy of voice search to a certain extent.

[0003] However, the aforementioned existing technologies still have significant limitations. Their semantic understanding depth is limited, and the n-gram language models they rely on are essentially statistical tools based on word frequency. They can only assess the local co-occurrence probability of words, but cannot deeply understand the semantic logic and contextual rationality at the sentence level. For example, for homophone sequences like "I go to the bank" and "I go to the navigation," the model struggles to accurately determine which one is semantically more fluent and better matches the user's true intent without considering the specific context. Furthermore, it lacks the ability to differentiate between seemingly awkward but context-appropriate queries. Summary of the Invention

[0004] The purpose of this invention is to provide a two-layer voice search method, system, and device based on semantic fluency and relevance, so as to solve the technical problems of inaccurate search results and low intent matching degree caused by the shallow semantic understanding and insufficient use of context in existing voice search methods.

[0005] The present invention achieves the above objectives through the following technical solutions: Firstly, this invention proposes a two-layer speech search method based on semantic fluency and relevance, the method comprising the following steps: S1. Obtain the user's voice input and convert it into an initial text sequence. Based on the confidence information of the initial text sequence, generate query candidate text. S2. Based on the semantic fluency score of each query candidate text, sort the K query candidate texts and select the top M texts as the candidate texts to be retrieved, where M≤K, and K and M are both preset positive integers; S3. For each candidate text to be retrieved, use the pre-built retrieval model to retrieve the top P candidate documents with the highest relevance from the document library; obtain M×P candidate documents, and determine the first relevance score of each candidate document and its corresponding candidate text to be retrieved. S4. For the M×P candidate documents, fuse the semantic fluency score and the first relevance score of the corresponding candidate text to be retrieved to determine the second relevance score of each candidate document. S5. Sort the M×P candidate documents according to the second relevance score to determine N first sorting results, where N is a preset positive integer, and N <M×P; S6. Determine if the user has historical interaction information. If so, reorder the N first sort results, generate a second sort result, and output the recommended document; otherwise, output the recommended document based on the first sort result.

[0006] Furthermore, in step S1, generating query candidate text based on the confidence information of the initial text sequence includes: S11. Based on the preset high confidence threshold CT and low confidence threshold PT, the words in the initial text sequence are distinguished as follows: High-confidence words: Their speech recognition confidence level is no lower than that of CT. Ambiguous candidate words: their speech recognition confidence is between PT and CT; Fuzzy words: their speech recognition confidence level is lower than PT; S12. For the ambiguous candidate words, generate a near-sound candidate set based on a predefined near-sound confusion rule set; for the fuzzy words, generate an enumerated candidate set based on homophones / word database. S13. Combine all high-confidence words and the candidates selected from the near-phonetic candidate set and the enumerated candidate set of ambiguous candidate words respectively, and generate the query candidate text by traversal.

[0007] Furthermore, step S3 includes: S31. Based on the speech recognition confidence level of each query candidate text source in the candidate text set to be retrieved, assign a retrieval depth parameter Pi to it; wherein, the determination rule of the retrieval depth parameter Pi is as follows: If the query candidate text consists of high-confidence words, then Pi = P1, where P1 is the first preset depth value; If the query candidate text contains ambiguous candidate words, then Pi = P2, where P2 is the second preset depth value; If the candidate text contains ambiguous words, then Pi = P3, where P3 is the third preset depth value; Where P1, P2, and P3 are preset positive integers, and satisfy P1 > P2 > P3; S32. Using a pre-built dense retrieval model, perform a retrieval for each query candidate text; the dense retrieval model encodes the query candidate text into a query vector and the documents in the document library into document vectors, and calculates the cosine similarity between the query vector and each document vector as the first relevance score; for each query candidate text, only the top Pi candidate documents with the highest first relevance score and their scores are retained.

[0008] Furthermore, the pre-built dense retrieval model is a dual encoder model based on the BERT architecture.

[0009] Furthermore, step S4 includes: S41. Standardize the first relevance score to obtain the standardized first relevance score Ri; normalize the semantic fluency score of each candidate text to be retrieved to obtain the normalized semantic fluency score St. S42. For each candidate text to be retrieved, dynamically determine the combination of weight coefficients used for fusion based on its corresponding speech recognition confidence classification. ; Wherein: when the candidate text comes from high-confidence words, the first weight coefficient is set. The first preset value Second weighting coefficient The second preset value 1, and satisfy > 1; When candidate text originates from ambiguous candidate words, set The third preset value , The fourth preset value 2, and satisfy ≈ 2; When candidate text originates from fuzzy words, set The fifth preset value 3, The sixth preset value 3, and satisfy 3 < 3; among which, ; S43. For candidate document i belonging to the candidate text t to be retrieved, the formula for calculating its second relevance score Ti is: .

[0010] Furthermore, in step S6, reordering the N first sorting results to generate a second sorting result includes: S61. Obtain the user's historical interaction document collection H; S62. For each candidate document Dj in the N first ranking results, calculate its set similarity Sim(H, Dj) with the set of historical interaction documents H; S63. Calculate the reordering bonus based on the set similarity. and according to the formula Calculate the final score for each candidate document. Where Ti is the second relevance score. The preset historical influence coefficient; S64. Based on the final score Reorder the N first sort results to generate a second sort result.

[0011] Furthermore, the set similarity Sim(H, Dj) mentioned in step S62 is calculated as follows: S621. Calculate the semantic similarity sim(Dj, hk) between the candidate document Dj and each document hk in the historical interaction document set H; S622. Select the maximum value from all sim(Dj, hk) as the set similarity Sim(H, Dj).

[0012] Furthermore, the reordering of bonus values ​​described in step S63 The calculation method is as follows: ; in, The preset similarity threshold, The preset gain coefficient; and when hour, .

[0013] Secondly, the present invention proposes a two-layer speech search system based on semantic fluency and relevance for implementing the two-layer speech search method described above, the system comprising: The first generation module is used to acquire the user's voice input, convert it into an initial text sequence, and generate multiple semantically relevant query candidate texts based on the confidence information of the initial text sequence. The second generation module is used to sort the K query candidate texts according to the semantic fluency score of each query candidate text, and select the top M texts as the candidate texts to be retrieved, where M≤K, and K and M are both preset positive integers; The first relevance analysis module is used to retrieve the top P candidate documents with the highest relevance from the document library for each candidate text to be retrieved using a pre-built retrieval model; to obtain M×P candidate documents, and to determine the first relevance score of each candidate document and its corresponding candidate text to be retrieved. The second relevance analysis module is used to determine the second relevance score of each candidate document by fusing the semantic fluency score and the first relevance score of the corresponding candidate text to be retrieved for the M×P candidate documents. The first sorting module is used to sort the M×P candidate documents according to the second relevance score, and determine N first sorting results, where N is a preset positive integer, and N <M×P; The re-sorting module is used to determine whether the user has historical interaction information. If so, it re-sorts the N first sort results, generates a second sort result, and outputs recommended documents; otherwise, it outputs recommended documents based on the first sort result.

[0014] Thirdly, the present invention proposes a computer device including a processor and a memory, wherein the memory is used to store a computer program, and the computer program, when executed by the processor, implements the steps of the two-layer voice search method described above.

[0015] The beneficial effects of this invention are as follows: This invention addresses the uncertainty of speech recognition by integrating speech recognition confidence information into the entire process of query generation, retrieval ranking, and result fusion. Specifically, the scheme classifies recognized words based on confidence thresholds to generate diverse query candidates and evaluates their fluency using a deep semantic model. In the retrieval stage, the retrieval depth is dynamically allocated based on the overall confidence level of the query, and a dense retrieval model is used to obtain semantic relevance. In the fusion ranking stage, the fusion weights of semantic fluency and retrieval relevance are dynamically adjusted based on query confidence to achieve differentiated processing of queries of different quality. Finally, historical interaction information is introduced for context-aware re-ranking. This invention solves the problem of insufficient search accuracy caused by speech recognition ambiguity and vague user expressions. Attached Figure Description

[0016] Figure 1 This is a flowchart of a two-layer speech search method based on semantic fluency and relevance in this invention; Figure 2 This is a flowchart illustrating the generation of candidate text in this invention. Figure 3 This is a flowchart of semantic fluency evaluation and screening in this invention; Figure 4 This is a flowchart of adaptive retrieval and first relevance score calculation in this invention; Figure 5 This is a flowchart of a two-layer fusion calculation and preliminary sorting process in this invention; Figure 6 This is a flowchart of a context-aware reordering method in this invention; Figure 7 This is a system block diagram of a two-layer speech search system based on semantic fluency and relevance in this invention. Detailed Implementation

[0017] The present application will now be described in further detail with reference to the accompanying drawings. It should be noted that the following specific embodiments are only used to further illustrate the present application and should not be construed as limiting the scope of protection of the present application. Those skilled in the art can make some non-essential improvements and adjustments to the present application based on the above application content.

[0018] Voice interaction is one of the core methods of human-computer interaction today, and the widespread adoption of voice search has greatly improved the user experience. However, challenges exist in the process of converting speech recognition into text for retrieval, including ambiguity in homophone recognition, non-standard colloquial expressions, and unclear user intent. Existing methods typically treat speech recognition and text retrieval as two independent stages, failing to effectively transfer the uncertainty of the recognition stage to the retrieval system, thus limiting the accuracy of search results. Especially in multi-turn dialogues or complex query scenarios, the system struggles to understand the user's continuously changing intent at the semantic level, resulting in insufficient accuracy in personalized recommendations. Therefore, there is an urgent need for a voice search solution that can span the entire process of speech recognition and semantic retrieval, achieving deep semantic fusion and context awareness.

[0019] Example 1 To address the aforementioned problems, this invention provides a two-layer speech search method based on semantic fluency and relevance, which will be discussed below. Figure 1 The method flow is described in detail. This two-layer speech search method based on semantic fluency and relevance may include the following steps: S1, Combination Figure 2 The system acquires user-input speech and converts it into an initial text sequence using a speech recognition module. During this process, the speech recognition module simultaneously outputs confidence information for each recognized word, which characterizes the reliability of the recognition result. Based on the initial text sequence and its confidence information, multiple semantically different query candidate texts are generated to cover potential ambiguities arising from speech recognition.

[0020] In this embodiment, the user-input speech signal is processed by a speech recognition module. This speech recognition module is built on a deep neural network and can convert speech into an initial text sequence, simultaneously outputting the recognition confidence score of each word in the sequence. The confidence score is a value between 0 and 1, reflecting the degree of certainty with which the model recognizes the word.

[0021] In a preferred embodiment, step S1 involves generating query candidate text based on the confidence information of the initial text sequence, including: S11. Based on the preset high confidence threshold CT and low confidence threshold PT, classify each word in the initial text sequence according to its confidence level. CT and PT are preset values ​​within the interval (0,1), and CT > PT. The specific classification rules are as follows: High-confidence words: Their speech recognition confidence Ci is not lower than CT (Ci≥CT). The recognition results of such words are reliable and can be directly used as candidates. Ambiguous candidate words: Their speech recognition confidence Ci is between PT and CT (PT < Ci < CT). These words have certain recognition uncertainty and need further processing. Fuzzy words: Their speech recognition confidence Ci is lower than PT (Ci < PT). The recognition reliability of such words is low, and extensive candidate enumeration is required.

[0022] S12. Employ differentiated candidate generation strategies for different types of vocabulary: For ambiguous candidate words, a near-phonetic candidate set is generated based on a predefined set of near-phonetic confusion rules. This rule set is constructed based on linguistic knowledge and historical data, recording common easily confused phoneme combinations (such as "n" and "l", "zh" and "z") and their corresponding candidate words. A near-phonetic candidate set containing a finite number of candidates is generated for each ambiguous candidate word according to this rule set.

[0023] For fuzzy words, an enumerated candidate set is generated based on homophones / word databases; a complete homophone / word database is queried to enumerate all possible homophones or near-homophones of the word, forming an enumerated candidate set.

[0024] S13. Generate the final query candidate text by combining and traversing.

[0025] Specifically, keeping all high-confidence words unchanged, for the position of each ambiguous candidate word, a candidate is selected from its near-phonetic candidate set for replacement; for the position of each vague word, a candidate is selected from its enumerated candidate set for replacement. By traversing all possible combinations, multiple complete query candidate texts are generated.

[0026] Optionally, to control the combinatorial explosion problem, a maximum number of generated combinations L can be set in actual implementation, or pruning can be performed based on the weighted sum of the confidence of the combinations, retaining only the top L most likely candidates.

[0027] S2, Combination Figure 3 Based on the semantic fluency score of each query candidate text, the K query candidate texts are sorted, and the top M texts are selected as the candidate texts to be retrieved, where M≤K, and K and M are preset positive integers set according to the system performance and response time requirements.

[0028] Specifically, all the query candidate texts generated in step S1 are input into a pre-trained semantic fluency evaluation model. This model is preferably based on a Transformer architecture (such as BERT, RoBERTa, etc.) and outputs a quantified semantic fluency score. Based on this score, all K query candidate texts are sorted, and the top M texts with the highest semantic fluency scores are selected as the candidate texts to be retrieved. This step, through deep semantic filtering, effectively selects the query expressions that best conform to language habits and common sense.

[0029] S3, Combination Figure 4 For each candidate text to be retrieved, the top P candidate documents with the highest relevance are retrieved from the document library using a pre-built retrieval model; M×P candidate documents are obtained, and the first relevance score of each candidate document and its corresponding candidate text to be retrieved is determined.

[0030] In a preferred embodiment, step S3 includes: S31. Assign a retrieval depth parameter Pi to each candidate text source in the candidate text set to be retrieved, based on the speech recognition confidence level of that source.

[0031] Understandably, identifying query candidate texts with high confidence is semantically clear and requires a broader search to find the optimal match; while query candidate texts with low confidence are inherently uncertain, and a broad search may introduce a lot of noise, so their search scope should be limited.

[0032] The rule for determining the retrieval depth parameter Pi is as follows: If the query candidate text consists of high-confidence words, then Pi = P1, where P1 is the first preset depth value; If the query candidate text contains ambiguous candidate words, then Pi = P2, where P2 is the second preset depth value; If the candidate text contains ambiguous words, then Pi = P3, where P3 is the third preset depth value; Where P1, P2, and P3 are preset positive integers, and satisfy P1 > P2 > P3.

[0033] S32. Using a pre-built dense retrieval model, perform a retrieval for each query candidate text. The dense retrieval model overcomes the semantic limitations of traditional keyword matching by mapping queries and documents to the same high-dimensional semantic vector space for similarity comparison. Specifically, the dense retrieval model encodes query candidate texts into query vectors and documents in the document library into document vectors. It calculates the cosine similarity between the query vector and each document vector as the first relevance score. For each query candidate text, only the top Pi candidate documents with the highest first relevance scores and their scores are retained.

[0034] In this embodiment, by adaptively setting the retrieval depth parameter Pi for query candidate texts with different confidence levels, and retaining only the top Pi candidate documents, both retrieval efficiency and retrieval result quality are positively impacted. For example, for high-confidence queries, expanding the retrieval scope (larger P1) helps improve recall and avoids missing potential high-quality answers; for low-confidence queries, limiting the retrieval scope (smaller P2 and P3) effectively filters out a large number of low-relevance noise documents caused by ambiguity in the query itself. This setting ensures that retrieval resources are tilted towards high-quality queries, while avoiding over-computation of low-quality queries.

[0035] In a preferred approach, the pre-built dense retrieval model is a dual-encoder model based on the BERT architecture. This model contains two independent BERT encoders, one for semantic encoding of the query and the other for encoding the document. During the training phase, a contrastive learning strategy is employed, which maximizes the similarity of positive sample pairs (relevant query-document) and minimizes the similarity of negative sample pairs (irrelevant query-document), enabling the model to learn high-quality semantic representations.

[0036] S4, Combination Figure 5 For M×P candidate documents, the semantic fluency score and the first relevance score of the corresponding candidate text to be retrieved are combined to determine the second relevance score of each candidate document.

[0037] In a preferred embodiment, step S4 includes: S41. Standardize the first relevance score to obtain the standardized first relevance score Ri; normalize the semantic fluency score of each candidate text to be retrieved to obtain the normalized semantic fluency score St.

[0038] The purpose of standardization and normalization is to eliminate dimensional differences and distribution shifts between different scores, ensuring the fairness and effectiveness of subsequent fusion calculations. Specifically, the Z-score standardization method can be used to process the first relevance score, and the min-maximum normalization method can be used to process the semantic fluency score.

[0039] S42. For each candidate text to be retrieved, dynamically determine the combination of weight coefficients used for fusion based on its corresponding speech recognition confidence classification. Speech recognition confidence refers to the overall confidence level of the query candidate text formed after classifying the words of the initial text sequence based on speech recognition confidence in step S1.

[0040] Wherein: when the candidate text comes from high-confidence words, the first weight coefficient is set. The first preset value Second weighting coefficient The second preset value 1, and satisfy > 1; When candidate text originates from ambiguous candidate words, set The third preset value , The fourth preset value 2, and satisfy ≈ 2; When candidate text originates from fuzzy words, set The fifth preset value 3, The sixth preset value 3, and satisfy 3 < 3; among which, .

[0041] Understandably, speech recognition confidence directly reflects the reliability of the query text. For queries consisting of high-confidence words, the expression is clear and the intent is explicit; therefore, the semantic fluency score has higher reference value and should be given greater weight. > 1) For queries containing vague terms, whose semantics are ambiguous, the relevance score returned by the retrieval model becomes a more reliable criterion, and therefore relevance should be given higher weight. 3 < 3) For ambiguous candidate words, which fall between the two, a balanced strategy is adopted. ≈ 2) The above-mentioned dynamic weighting method allows the system to adaptively evaluate the credibility of different information sources. Through this fusion strategy that matches the level of uncertainty, the semantic information of high-quality queries can be utilized to the greatest extent while suppressing the noise that low-quality queries may bring, thereby improving the overall accuracy of the ranking results.

[0042] S43. For candidate document i belonging to the candidate text t to be retrieved, the formula for calculating its second relevance score Ti is: .

[0043] S5. Sort the M×P candidate documents according to the second relevance score, and determine N first ranking results, where N is a preset positive integer, and N <M×P。

[0044] Step S5 completed the initial screening based on semantic fusion, and obtained a set of high-quality candidates with a controllable number.

[0045] S6, Combination Figure 6 Determine if the user has historical interaction information. If so, reorder the N first sort results, generate a second sort result, and output the recommended document; otherwise, output the recommended document based on the first sort result.

[0046] Specifically, determining whether a user has historical interaction information can be done by querying the user behavior log based on the current user identifier. If there are historical query and feedback records within a preset valid time window, then historical interaction information is determined to exist.

[0047] If no historical interaction information is found, the top N first-ranked results are directly used as the final output. Specifically, the highest-ranked document is selected from this ranking list as the recommended document, or the entire list is returned to the user in score order.

[0048] When historical interaction information is determined to exist, in a preferred embodiment, step S6 involves reordering the N first sorting results to generate a second sorting result, including: S61. Obtain the user's historical interaction document set H; the historical interaction document set H can consist of the result documents that the user ultimately adopted or explicitly fed back as satisfactory in past search sessions.

[0049] S62. For each candidate document Dj in the N first ranking results, calculate its set similarity Sim(H, Dj) with the set of historical interaction documents H; this step can measure the degree of association between the current candidate document and the user's historical interests from a semantic level.

[0050] S63. Calculate reordering bonus based on set similarity. and according to the formula Calculate the final score for each candidate document. Where Ti is the second relevance score. This is the preset historical influence coefficient.

[0051] Among them, the historical influence coefficient It is a preset adjustable parameter used to control the weight of user history interaction information in the final sorting. The larger the value, the stronger the influence of historical interaction preferences on the ranking; The smaller the value, the more the system relies on the semantic and content relevance of the current query. In practical applications, it can be adaptively adjusted based on user activity or query type.

[0052] For example, the size can be appropriately increased for highly active users or exploratory queries. To enhance personalization, the size of queries for new users or navigational queries needs to be reduced. To ensure the certainty and accuracy of the results.

[0053] S64. Based on the final score Reorder the N first sort results to generate a second sort result.

[0054] In a preferred embodiment, the set similarity Sim(H, Dj) in step S62 is calculated as follows: S621. Calculate the semantic similarity sim(Dj, hk) between the candidate document Dj and each document hk in the historical interaction document set H; the semantic similarity can be obtained by calculating the cosine similarity between document vectors, where the document vectors are generated by the document encoder in the aforementioned dense retrieval model.

[0055] S622. Select the maximum value from all sim(Dj, hk) as the set similarity Sim(H, Dj). The strategy of taking the maximum value focuses on identifying the most relevant part between the current document and the user's historical interests, rather than the average degree of correlation, which can better highlight the user's potential intent focus.

[0056] In a preferred embodiment, step S63 involves reordering the bonus values. The calculation method is as follows: ; in, The preset similarity threshold, The preset gain coefficient; and when hour, This design introduces a similarity threshold. Only documents with a relevance exceeding this threshold are considered to have significant personalized guidance value, thus avoiding unnecessary score perturbation of weakly related documents.

[0057] According to the above embodiments, the method proposed in this invention first performs hierarchical processing on the recognized text based on speech recognition confidence and generates multiple candidate queries, and then uses a deep semantic model to evaluate the query fluency for filtering; in the retrieval stage, the retrieval scope is adaptively adjusted according to the query confidence, and a dense retrieval model is used to obtain a semantic relevance score; then, a dynamic weight fusion mechanism is used to weight the query fluency and retrieval relevance to obtain a comprehensive relevance score; finally, context-aware reordering is performed in combination with user historical interaction information to achieve personalized result optimization.

[0058] Example 2 Combination Figure 7 Based on the same inventive concept, another preferred embodiment of the present invention proposes a two-layer speech search system based on semantic fluency and relevance for implementing the two-layer speech search method as described in Embodiment 1. The system includes: The first generation module is used to acquire the user's voice input, convert it into an initial text sequence, and generate multiple semantically relevant query candidate texts based on the confidence information of the initial text sequence.

[0059] Specifically, the module integrates a speech recognition component, which can process audio input in real time and output text with word-level confidence information. Its internal logic executes the confidence classification and candidate generation process as described in steps S11 to S13 of Embodiment 1.

[0060] The second generation module is used to sort the K query candidate texts according to the semantic fluency scores of each query candidate text, and select the top M texts as the candidate texts to be retrieved, where M≤K, and K and M are preset positive integers; this module has built-in or calls a pre-trained semantic fluency evaluation model (such as a Transformer-based model) to perform deep semantic evaluation and filtering of the query candidate texts.

[0061] The first relevance analysis module is used to retrieve the top P candidate documents with the highest relevance from the document library for each candidate text to be retrieved using a pre-built retrieval model; resulting in M×P candidate documents, and determining the first relevance score of each candidate document with its corresponding candidate text to be retrieved; the pre-built dense retrieval model in this module can encode text into high-dimensional semantic vectors and calculate similarity. This module also integrates an adaptive strategy, which can dynamically adjust its retrieval depth parameter Pi according to the confidence classification (high confidence, ambiguous, fuzzy) of the query candidate text, and the specific execution logic is the same as steps S31 to S32 in Example 1.

[0062] The second relevance analysis module is used to determine the second relevance score for each of the M×P candidate documents by fusing the semantic fluency score and the first relevance score of the corresponding candidate text to be retrieved. This module is responsible for score standardization / normalization preprocessing and dynamically determines the combination of fusion weight coefficients based on the confidence classification of the query candidate text. Finally, the weighted summation formula is used ( The second correlation score is calculated, which is specifically implemented in steps S41 to S43 of Embodiment 1.

[0063] The first sorting module is used to sort the M×P candidate documents according to the second relevance score, and determine N first sorting results, where N is a preset positive integer, and N <M×P。

[0064] The re-ranking module determines whether the user has historical interaction information. If so, it re-ranks the N first ranking results, generates a second ranking result, and outputs recommended documents; otherwise, it outputs recommended documents based on the first ranking result. The judgment logic of this module is based on the user's query behavior log. When valid historical information exists, this module adjusts the preliminary ranking result by calculating the semantic relevance between the current candidate document and the set of historical interaction documents, combined with a preset historical influence coefficient λ and a threshold θ. The specific execution flow corresponds to steps S61 to S64 in Embodiment 1.

[0065] It should be noted that each module in the above-mentioned two-layer voice search system corresponds to steps S1 to S6 in implementing the above-mentioned two-layer voice search method. The instances and application scenarios implemented by multiple modules and their corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned embodiment 1.

[0066] Example 3 Another preferred embodiment of the present invention provides a computer device including a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the steps of the two-layer voice search method as described in Embodiment 1.

[0067] It is understood that the computer equipment can take the form of various hardware devices, such as smart terminal devices and cloud servers; smart terminal devices may include smartphones, smart speakers, smart vehicle systems and tablets, and in practical applications, the devices may adopt corresponding configurations depending on the deployment scenario.

[0068] At the application level, devices equipped with the aforementioned computer program can be integrated into various voice interaction systems, such as intelligent customer service platforms, voice assistants (e.g., mobile phone voice assistants, smart home control centers), in-vehicle infotainment systems, and mobile applications with voice search capabilities. Users send voice queries to the device via input devices such as microphones, and the device, by executing the method described in Example 1, ultimately returns search results in text or structured data format to the user.

[0069] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0070] In addition, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0071] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A two-layer speech search method based on semantic fluency and relevance, characterized in that, The method includes the following steps: S1. Obtain the user's voice input and convert it into an initial text sequence. Based on the confidence information of the initial text sequence, generate query candidate text. S2. Based on the semantic fluency score of each query candidate text, sort the K query candidate texts and select the top M texts as the candidate texts to be retrieved, where M≤K, and K and M are both preset positive integers; S3. For each candidate text to be retrieved, use the pre-built retrieval model to retrieve the top P candidate documents with the highest relevance from the document library; We obtain M×P candidate documents and determine the first relevance score of each candidate document to its corresponding candidate text to be retrieved. S4. For the M×P candidate documents, fuse the semantic fluency score and the first relevance score of the corresponding candidate text to be retrieved to determine the second relevance score of each candidate document. S5. Sort the M×P candidate documents according to the second relevance score to determine N first sorting results, where N is a preset positive integer, and N <M×P; S6. Determine if the user has historical interaction information. If so, reorder the N first sort results, generate the second sort result and output the recommended document. Otherwise, output recommended documents based on the first ranking result; Step S3 includes: S31. Based on the speech recognition confidence level of each query candidate text source in the candidate text set to be retrieved, assign a retrieval depth parameter Pi to it; wherein, the determination rule of the retrieval depth parameter Pi is as follows: If the query candidate text consists of high-confidence words, then Pi = P1, where P1 is the first preset depth value; If the query candidate text contains ambiguous candidate words, then Pi = P2, where P2 is the second preset depth value; If the candidate text contains ambiguous words, then Pi = P3, where P3 is the third preset depth value; Where P1, P2, and P3 are preset positive integers, and satisfy P1 > P2 > P3; S32. Using a pre-built dense retrieval model, perform a retrieval for each query candidate text; the dense retrieval model encodes the query candidate text into a query vector and the documents in the document library into document vectors, and calculates the cosine similarity between the query vector and each document vector as the first relevance score; for each query candidate text, only the top Pi candidate documents with the highest first relevance score and their scores are retained.

2. The two-layer speech search method based on semantic fluency and relevance according to claim 1, characterized in that, In step S1, generating query candidate text based on the confidence information of the initial text sequence includes: S11. Based on the preset high confidence threshold CT and low confidence threshold PT, the words in the initial text sequence are distinguished as follows: High-confidence words: Their speech recognition confidence level is no lower than that of CT. Ambiguous candidate words: their speech recognition confidence is between PT and CT; Fuzzy words: their speech recognition confidence level is lower than PT; S12. For the ambiguous candidate words, generate a near-sound candidate set based on a predefined near-sound confusion rule set; for the fuzzy words, generate an enumerated candidate set based on homophones / word database. S13. Combine all high-confidence words and the candidates selected from the near-phonetic candidate set and the enumerated candidate set of ambiguous candidate words respectively, and generate the query candidate text by traversal.

3. The two-layer speech search method based on semantic fluency and relevance according to claim 1, characterized in that, The pre-built dense retrieval model is a dual encoder model based on the BERT architecture.

4. The two-layer speech search method based on semantic fluency and relevance according to claim 1, characterized in that, Step S4 includes: S41. Standardize the first relevance score to obtain the standardized first relevance score Ri; normalize the semantic fluency score of each candidate text to be retrieved to obtain the normalized semantic fluency score St. S42. For each candidate text to be retrieved, dynamically determine the combination of weight coefficients used for fusion based on its corresponding speech recognition confidence classification. ; Wherein: when the candidate text comes from high-confidence words, the first weight coefficient is set. The first preset value Second weighting coefficient The second preset value 1, and satisfy > 1; When candidate text originates from ambiguous candidate words, set The third preset value , The fourth preset value 2, and satisfy ≈ 2; When candidate text originates from fuzzy words, set The fifth preset value 3, The sixth preset value 3, and satisfy 3< 3; among which, ; S43. For candidate document i belonging to the candidate text t to be retrieved, the formula for calculating its second relevance score Ti is: 。 5. The two-layer speech search method based on semantic fluency and relevance according to claim 1, characterized in that, In step S6, reordering the N first sorting results to generate a second sorting result includes: S61. Obtain the user's historical interaction document collection H; S62. For each candidate document Dj in the N first ranking results, calculate its set similarity Sim(H, Dj) with the set of historical interaction documents H; S63. Calculate the reordering bonus based on the set similarity. and according to the formula Calculate the final score for each candidate document. Where Ti is the second relevance score. The preset historical influence coefficient; S64. Based on the final score Reorder the N first sort results to generate a second sort result.

6. The two-layer speech search method based on semantic fluency and relevance according to claim 5, characterized in that, The calculation method for the set similarity Sim(H, Dj) in step S62 is as follows: S621. Calculate the semantic similarity sim(Dj, hk) between the candidate document Dj and each document hk in the historical interaction document set H; S622. Select the maximum value from all sim(Dj, hk) as the set similarity Sim(H, Dj).

7. The two-layer speech search method based on semantic fluency and relevance according to claim 6, characterized in that, The reordering of bonus values ​​described in step S63 The calculation method is as follows: ； in, The preset similarity threshold, The preset gain coefficient; and when hour, .

8. A two-layer speech search system based on semantic fluency and relevance for implementing the two-layer speech search method as described in any one of claims 1-7, characterized in that, The system includes: The first generation module is used to acquire the user's voice input, convert it into an initial text sequence, and generate multiple semantically relevant query candidate texts based on the confidence information of the initial text sequence. The second generation module is used to sort the K query candidate texts according to the semantic fluency score of each query candidate text, and select the top M texts as the candidate texts to be retrieved, where M≤K, and K and M are both preset positive integers; The first relevance analysis module is used to retrieve the top P candidate documents with the highest relevance from the document library for each candidate text to be retrieved using a pre-built retrieval model; to obtain M×P candidate documents, and to determine the first relevance score of each candidate document and its corresponding candidate text to be retrieved. The second relevance analysis module is used to determine the second relevance score of each candidate document by fusing the semantic fluency score and the first relevance score of the corresponding candidate text to be retrieved for the M×P candidate documents. The first sorting module is used to sort the M×P candidate documents according to the second relevance score, and determine N first sorting results, where N is a preset positive integer, and N <M×P; The re-sorting module is used to determine whether the user has historical interaction information. If so, it re-sorts the N first sort results, generates a second sort result, and outputs recommended documents; otherwise, it outputs recommended documents based on the first sort result.

9. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program, which, when executed by the processor, implements the steps of the two-layer voice search method as described in any one of claims 1-7.

Images