Knowledge retrieval feedback method, electronic device and computer readable storage medium

Abstract

The present application discloses a knowledge retrieval feedback method, an electronic device, and a computer-readable storage medium, including: obtaining an initial query statement and at least one corresponding transcribed query statement, determining the vector feature corresponding to each query statement and the keyword set corresponding to all query statements; obtaining a search sublist matched by each vector feature, deduplicating and reordering the knowledge fragments in all search sublists based on the ranking of the knowledge fragments in each search sublist and the number of recurrences of the knowledge fragments in all search sublists, and obtaining a vector search feedback list; obtaining a keyword search feedback list matched by a keyword set, deduplicating and reordering the knowledge fragments in the vector search feedback list and the keyword search feedback list based on the ranking and number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, and obtaining a target search feedback list. The above scheme can improve the accuracy of knowledge retrieval feedback.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge retrieval, and in particular to a knowledge retrieval feedback method, an electronic device and a computer-readable storage medium. Background Art

[0002] With the application of artificial intelligence in knowledge retrieval, the efficiency of retrieval feedback has been greatly improved. Multiple knowledge fragments can be efficiently fed back for the input query statement. However, when feeding back multiple knowledge fragments, it is still difficult to ensure that the knowledge fragments with higher matching degree can be fed back first, resulting in low accuracy of knowledge retrieval feedback. In view of this, how to improve the accuracy of knowledge retrieval feedback has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a knowledge retrieval feedback method, an electronic device and a computer-readable storage medium, which can improve the accuracy of knowledge retrieval feedback.

[0004] To solve the above technical problems, the first aspect of the present application provides a knowledge retrieval feedback method, including: obtaining an initial query statement and at least one corresponding transcribed query statement, determining the vector features corresponding to each query statement and the keyword set corresponding to all query statements; obtaining a retrieval sub-list matched by each vector feature, deduplicating and reordering the knowledge fragments in all the retrieval sub-lists based on the ranking of the knowledge fragments in each retrieval sub-list and the number of recurrences of the knowledge fragments in all the retrieval sub-lists, to obtain a vector retrieval feedback list; obtaining a keyword retrieval feedback list matched by the keyword set, deduplicating and reordering the knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list based on the ranking and number of recurrences of the knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list, to obtain a target retrieval feedback list.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] The above scheme obtains the initial query statement and at least one transcribed query statement obtained after transcribing the query statement, determines the vector features corresponding to each query statement and the keyword set composed of the keywords extracted from all query statements, thereby expanding other similar query statements in addition to the initial query statement during knowledge retrieval, enriching the expression of the query statement to obtain more vector features and a richer keyword set. Obtain the search sublist that matches each vector feature after retrieval, deduplicate all knowledge fragments in all search sublists based on the ranking of knowledge fragments included in each search sublist and the number of recurrences of knowledge fragments in all search sublists, and determine the ranking corresponding to the knowledge fragments after deduplication, and obtain a vector search feedback list, thereby combining the ranking of knowledge fragments in the search sublist and the number of recurrences of knowledge fragments during vector retrieval, merging and deduplicating the knowledge fragments and reordering them, and obtaining a vector search feedback list with higher sorting accuracy. A keyword search feedback list that matches after a keyword set search is obtained. Based on the ranking of knowledge fragments in the vector search feedback list, the ranking of knowledge fragments in the keyword search feedback list, and the number of recurrences of knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated, and the corresponding ranking of the deduplicated knowledge fragments is determined to obtain a target search feedback list. The ranking and number of recurrences of knowledge fragments in the search feedback list obtained by combining the two different search methods are combined, and the knowledge fragments are deduplicated and reordered to obtain a target search feedback list for feeding back the final search results, thereby improving the accuracy of knowledge retrieval feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a flowchart of an implementation method of the knowledge retrieval feedback method of the present application;

[0010] Figure 2 It is a flowchart of another implementation method of the knowledge retrieval feedback method of the present application;

[0011] Figure 3 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0012] Figure 4 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0015] The knowledge retrieval feedback method provided in the present application is used to perform knowledge retrieval based on a query statement and to sort the retrieved knowledge fragments and then provide feedback, and its corresponding execution subject is a processing unit capable of performing data processing.

[0016] See also Figure 1 , Figure 1 : is a flow chart of an implementation method of the knowledge retrieval feedback method of the present application, the method comprising:

[0017] S101: Obtain an initial query statement and at least one corresponding transcribed query statement, and determine a vector feature corresponding to each query statement and a keyword set corresponding to all query statements.

[0018] Specifically, an initial query statement and at least one transcribed query statement obtained by transcribing the query statement are obtained, and a vector feature corresponding to each query statement and a keyword set consisting of keywords extracted from all query statements are determined.

[0019] It is understandable that in addition to the initial query statement, other similar query statements are expanded during knowledge retrieval to enrich the expression of the query statement in order to obtain more vector features and a richer keyword set.

[0020] In some implementation scenarios, an initial query statement is obtained, a prompt statement is constructed using a prompt template and the initial query statement, the prompt statement is input into a large model, and at least one query statement is obtained after the large model transcribes the query statement. Vector feature extraction is performed on each query statement to obtain the vector feature corresponding to each query statement, and keyword extraction is performed on all query statements to obtain a keyword set composed of keywords.

[0021] In some implementation scenarios, an initial query statement is obtained, an expression corresponding to the initial query statement is determined, at least one other expression is obtained from an expression template, other expressions and the initial query statement are used to construct at least one query statement obtained after transcription, vector features are extracted for each query statement to obtain vector features corresponding to each query statement, keywords are extracted for all query statements to obtain a keyword set consisting of keywords.

[0022] Optionally, the number of transcribed query statements is usually multiple, so as to enrich query statements with different expressions. The large model may include but is not limited to deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTM), and generative pre-trained Transformer models, etc. No specific restrictions are made on the specific construction and deployment of the large model.

[0023] S102: Obtain a search sublist for each vector feature match, deduplicate and reorder the knowledge fragments in all search sublists based on the order of the knowledge fragments in each search sublist and the number of times the knowledge fragments recur in all search sublists, and obtain a vector search feedback list.

[0024] Specifically, a search sub-list is obtained for each vector feature that is matched after retrieval. Based on the ranking of the knowledge fragments included in each search sub-list and the number of times the knowledge fragments recur in all search sub-lists, all knowledge fragments in all search sub-lists are deduplicated, and the ranking corresponding to the deduplicated knowledge fragments is determined to obtain a vector search feedback list.

[0025] It can be understood that, by combining the order of knowledge fragments in the search sublist and the number of times the knowledge fragments recur during vector search, the knowledge fragments are merged, deduplicated and reordered to obtain a vector search feedback list with higher ordering accuracy.

[0026] It should be noted that the search sublist includes at least one knowledge fragment, the search sublist corresponds to the ranking of the knowledge fragment, and the knowledge fragment corresponds to the ranking weight associated with the ranking in the search sublist. When searching based on each vector feature, the same knowledge fragment can be located in multiple search sublists. Therefore, the same knowledge fragment can correspond to different rankings and their matching ranking weights in different search sublists.

[0027] It can be understood that when the knowledge fragments in the retrieval sub-list are arranged in reverse order, the size of the sorting weight is negatively correlated with the sorting position.

[0028] In some implementation scenarios, based on the vector features of each query statement, a corresponding sublist weight is set for each search sublist, and each search sublist is traversed in turn, and the knowledge fragments in the search sublist are tried to be added to the vector search feedback list to determine whether the vector search feedback list includes the knowledge fragment to be added. If not included, the ranking weight of the knowledge fragment when added to the vector search feedback list is determined based on the ranking weight matched by the ranking of the knowledge fragment in the search sublist and the sublist weight of the search sublist, and the knowledge fragment is added to the vector search feedback list according to the ranking weight; if it is included, the ranking weight of the knowledge fragment in the vector search feedback list is updated based on the ranking of the knowledge fragment in the search sublist and the sublist weight of the search sublist, as well as the number of recurrences of the knowledge fragment, until all search sublists are traversed to obtain the vector search feedback list.

[0029] In some implementation scenarios, the number of recurrences of knowledge fragments in all search sub-lists is obtained, and the ranking weights corresponding to the same knowledge fragments in the vector search feedback list are determined based on the ranking of the knowledge fragments in each search sub-list, their matching ranking weights, and the number of recurrences of the knowledge fragments. Based on the reverse order of the ranking weights, the knowledge fragments that are different from each other in all search sub-lists are added to the vector search feedback list.

[0030] It can be understood that the knowledge fragments in the retrieval sublist obtained based on vector feature retrieval have a strong correlation with the semantics of the query statement. The number of recurrences of the knowledge fragments in all retrieval sublists is used to improve the ranking of the corresponding knowledge fragments in the vector retrieval feedback list, thereby setting a higher ranking weight for the knowledge fragments that can be retrieved under the semantics of multiple expressions.

[0031] S103: Obtain a keyword search feedback list that matches the keyword set, and based on the order and number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, deduplicate and reorder the knowledge fragments in the vector search feedback list and the keyword search feedback list to obtain a target search feedback list.

[0032] Specifically, a keyword search feedback list that matches after the keyword set search is obtained, and based on the ranking of knowledge fragments in the vector search feedback list, the ranking of knowledge fragments in the keyword search feedback list, and the number of recurrences of knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated, and the corresponding ranking of the knowledge fragments after deduplication is determined to obtain a target search feedback list.

[0033] It can be understood that by combining the ranking and recurrence times of the knowledge fragments in the retrieval feedback list obtained by two different retrieval methods, the knowledge fragments are merged, deduplicated and re-sorted to obtain a target retrieval feedback list for feedback of the final retrieval results, thereby improving the accuracy of knowledge retrieval feedback.

[0034] Furthermore, the keyword search feedback list retrieved based on the keyword set has a strong correlation with the keyword, and the keyword search feedback list corresponds to the ranking of the knowledge fragments and the ranking weight matching the ranking.

[0035] In some implementation scenarios, all the knowledge fragments in the vector search feedback list are added to the target search feedback list according to the ranking weights, and the knowledge fragments in the keyword search feedback list are traversed in turn to determine whether the target search feedback list includes the knowledge fragment to be added. If not included, the knowledge fragment is added to the target search feedback list according to the ranking weights matched by the ranking of the knowledge fragment in the keyword search feedback list; if included, the ranking weights of the knowledge fragment in the target search feedback list are updated based on the ranking weights matched by the ranking of the knowledge fragment in the keyword search feedback list and the number of recurrences of the knowledge fragment in the vector search feedback list and the keyword search feedback list, until the knowledge fragments in the keyword search feedback list are traversed to obtain the vector search feedback list.

[0036] In some implementation scenarios, the number of recurrences of knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list is obtained, and the ranking weights corresponding to the same knowledge fragments in the target retrieval feedback list are determined based on the ranking weights matched by the rankings of the knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list, and the number of recurrences of the knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list, and the different knowledge fragments are added to the target retrieval feedback list in reverse order of the ranking weights.

[0037] Optionally, the knowledge fragments finally retained in the target retrieval feedback list may only include knowledge fragments with a weight higher than a threshold.

[0038] The above scheme obtains the initial query statement and at least one transcribed query statement obtained after transcribing the query statement, determines the vector features corresponding to each query statement and the keyword set composed of the keywords extracted from all query statements, thereby expanding other similar query statements in addition to the initial query statement during knowledge retrieval, enriching the expression of the query statement to obtain more vector features and a richer keyword set. Obtain the search sublist that matches each vector feature after retrieval, deduplicate all knowledge fragments in all search sublists based on the ranking of knowledge fragments included in each search sublist and the number of recurrences of knowledge fragments in all search sublists, and determine the ranking corresponding to the knowledge fragments after deduplication, and obtain a vector search feedback list, thereby combining the ranking of knowledge fragments in the search sublist and the number of recurrences of knowledge fragments during vector retrieval, merging and deduplicating the knowledge fragments and reordering them, and obtaining a vector search feedback list with higher sorting accuracy. A keyword search feedback list that matches after a keyword set search is obtained. Based on the ranking of knowledge fragments in the vector search feedback list, the ranking of knowledge fragments in the keyword search feedback list, and the number of recurrences of knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated, and the corresponding ranking of the deduplicated knowledge fragments is determined to obtain a target search feedback list. The ranking and number of recurrences of knowledge fragments in the search feedback list obtained by combining the two different search methods are combined, and the knowledge fragments are deduplicated and reordered to obtain a target search feedback list for feeding back the final search results, thereby improving the accuracy of knowledge retrieval feedback.

[0039] See also Figure 2 , Figure 2 : is a flow chart of another implementation of the knowledge retrieval feedback method of the present application, the method comprising:

[0040] S201: Obtain an initial query statement and its corresponding original expression, and transcribe the initial query statement using other expressions different from the original expressions to obtain at least one transcribed query statement.

[0041] Specifically, an initial query statement is obtained to determine an original expression corresponding to the initial query statement, and the initial query statement is transcribed using other expressions different from the original expressions to obtain a transcribed query statement.

[0042] Optionally, the query statements are converted and rewritten using a large model to obtain multiple similar query statements in different expressions, thereby enriching the expression of the query statements and enhancing the matching degree between all query statements and knowledge fragments in the knowledge base.

[0043] S202: Perform vector feature extraction on each query statement to obtain a vector feature corresponding to each query statement, perform keyword extraction on all query statements to obtain a keyword set corresponding to all query statements.

[0044] Specifically, vector feature extraction is performed on each query statement to obtain the vector feature corresponding to each query statement, and keyword extraction is performed on all query statements to obtain a keyword set consisting of all extracted keywords.

[0045] Optionally, vector feature extraction can be implemented using a pre-trained vector model, and keyword extraction can be implemented using a pre-trained keyword model.

[0046] S203: Obtain the search sublists that match each vector feature, and based on the ranking of the knowledge fragments in each search sublist and the number of times the knowledge fragments recur in all search sublists, deduplicate and reorder the knowledge fragments in all search sublists to obtain a vector search feedback list.

[0047] Specifically, a search sub-list is obtained for each vector feature that is matched after retrieval. Based on the ranking of the knowledge fragments included in each search sub-list and the number of times the knowledge fragments recur in all search sub-lists, all knowledge fragments in all search sub-lists are deduplicated, and the ranking corresponding to the deduplicated knowledge fragments is determined to obtain a vector search feedback list.

[0048] In some implementation scenarios, a search sublist matching each vector feature is obtained, and based on the ranking of the knowledge fragments in each search sublist and the number of times the knowledge fragments recur in all search sublists, the knowledge fragments in all search sublists are deduplicated and reordered to obtain a vector search feedback list, including: searching based on each vector feature to obtain a search sublist matching each vector feature; wherein, the search sublist has an initial ranking weight corresponding to the ranking of the knowledge fragments; based on the initial ranking weight of the knowledge fragments in each search sublist and the number of times the knowledge fragments recur in all search sublists, the knowledge fragments in all search sublists are deduplicated, and the final ranking weight of the knowledge fragment in the vector search feedback list is determined to obtain a vector search feedback list.

[0049] Specifically, based on each vector feature, a search is performed in the vector index knowledge base to obtain a search sub-list matching each vector feature, wherein an initial sorting weight is set in the search sub-list according to the sorting of the knowledge fragments, and the initial sorting weight is only associated with the sorting of the knowledge fragments in the search sub-list.

[0050] Optionally, the ranking weights of the knowledge fragments and ranking associations can be set using a Reciprocal Rank Fusion (RRF) ranker, which can be expressed using the following formula:

[0051] (1)

[0052] Among them, d is the current knowledge fragment, k is a constant, R is the set of rankers, and r(d) is the ranking of the knowledge fragment in the ranker. In addition to the RRF ranker, the ranking weight can also be set using a preset ranking formula. When the knowledge fragments in the retrieval sublist are arranged in reverse order, the size of the ranking weight is negatively correlated with the ranking position.

[0053] Optionally, a first query prompt statement is constructed based on each vector feature, the corresponding first query prompt statement is input into the big model, and the big model is used to retrieve a search sublist matching each vector feature from the vector index knowledge base.

[0054] Furthermore, based on the initial ranking weights of the knowledge fragments in each retrieval sub-list and the number of times the knowledge fragments recur in all retrieval sub-lists, the repeated knowledge fragments are deduplicated, and the final ranking weights of the knowledge fragments in the vector retrieval feedback list are determined, so that the ranking weights of the knowledge fragments in different retrieval sub-lists are fused, and the fused weights are adjusted using the number of recurrences to determine the final more accurate ranking weights of the knowledge fragments in the vector retrieval feedback list, so that the more relevant the knowledge fragments are to the query statement in the vector retrieval feedback list, the higher they are ranked, thereby improving the recall rate of results that are strongly associated with the query statement.

[0055] In a specific implementation scenario, there are corresponding similarity scores between the knowledge fragments in the search sub-list and the corresponding vector features.

[0056] Specifically, based on the features of the knowledge fragment and the vector features corresponding to the search sublist where the knowledge fragment is located, a similarity score can be obtained. When the same knowledge fragment is in different search sublists, since the vector features corresponding to the search sublists are different, the similarity scores of the same knowledge fragment compared with different vector features can represent the similarity between the knowledge fragment and different semantics. The calculation method of the similarity score can be expressed by the following formula:

[0057] (2)

[0058] Among them, d is the vector feature of the current knowledge fragment, q is the vector feature of the query statement, and cos is the cosine similarity.

[0059] Furthermore, based on the initial ranking weight of the knowledge fragments in each retrieval sublist and the number of recurrences of the knowledge fragments in all retrieval sublists, the knowledge fragments in all retrieval sublists are deduplicated, and the final ranking weight of the knowledge fragments in the vector retrieval feedback list is determined to obtain the vector retrieval feedback list, including: trying to add the knowledge fragments in the retrieval sublists to the vector retrieval feedback list in turn; in response to the knowledge fragment to be added not being included in the vector retrieval feedback list, determining the ranking weight of the knowledge fragment in the vector retrieval feedback list based on the ranking weight of the knowledge fragment in the retrieval sublist and the similarity score corresponding to the knowledge fragment; in response to the knowledge fragment to be added being included in the vector retrieval feedback list, determining the latest number of recurrences of the knowledge fragment, obtaining a recurrence factor score matching the number of recurrences, and updating the ranking weight of the knowledge fragment in the vector retrieval feedback list based on the ranking weight of the knowledge fragment in the vector retrieval feedback list, the ranking weight of the knowledge fragment in the retrieval sublist, and the similarity score and recurrence factor score currently corresponding to the knowledge fragment; in response to traversing all retrieval sublists, determining the final ranking weight of the knowledge fragment in the vector retrieval feedback list to obtain the vector retrieval feedback list.

[0060] Specifically, the vector retrieval feedback list is taken as an empty list, and each retrieval sub-list is traversed in turn, and the knowledge fragments in the retrieval sub-list are tried to be added to the vector retrieval feedback list in turn, and it is determined whether the current vector retrieval feedback list includes the knowledge fragment to be added. If the knowledge fragment to be added is not included in the vector retrieval feedback list, the knowledge fragment and its corresponding ranking weight in the retrieval sub-list are added to the vector retrieval feedback list, and then the similarity score is superimposed on the ranking weight to determine the current ranking weight of the knowledge fragment in the vector retrieval feedback list.

[0061] Furthermore, if the knowledge fragment to be added is already included in the vector retrieval feedback list, the latest number of recurrences of the knowledge fragment is determined, and a recurrence factor score matching the number of recurrences is obtained. The ranking weight of the knowledge fragment in the currently traversed search sublist is merged with the current ranking weight of the knowledge fragment in the vector retrieval feedback list, and then the similarity score is superimposed on the ranking weight, and then the superimposed ranking weight is adjusted using the recurrence factor score, wherein the recurrence factor score is positively correlated with the number of recurrences, thereby being able to appropriately increase the ranking weight and update the ranking weight of the knowledge fragment in the vector retrieval feedback list.

[0062] Optionally, the fusion between the sorting weights can generally be an addition or weighted superposition operation, the similarity score superimposed on the sorting weight is generally an addition or multiplication operation, and the adjustment of the sorting weight by the reproduction factor score can be a multiplication or addition operation. The specific operation method can be adaptively selected, and the present application does not impose any specific restrictions on this.

[0063] It can be understood that after traversing all search sub-lists, the final ranking weight of the knowledge fragment in the vector search feedback list can be determined to obtain the vector search feedback list.

[0064] It should be noted that the recurrence factor score is positively correlated with the number of recurrences corresponding to the knowledge fragment and the total number of indexes, and the total number of indexes represents the total number of knowledge fragments from the same index source.

[0065] Alternatively, the reproducibility factor score can be expressed using the following formula:

[0066] (3)

[0067] Where n is the total number of knowledge fragments retrieved from the index source where the knowledge fragment is located, and m is the number of recurrences of the knowledge fragment. The index source usually corresponds to the same chapter, and the same chapter includes knowledge points of the same subject.

[0068] Furthermore, based on the ranking weights of the knowledge fragments in the vector retrieval feedback list, the ranking weights of the knowledge fragments in the retrieval sub-list, and the similarity scores and re-factor scores currently corresponding to the knowledge fragments, the ranking weights of the knowledge fragments in the vector retrieval feedback list are updated, including: weighted superposition of the ranking weights of the knowledge fragments in the vector retrieval feedback list and the ranking weights of the knowledge fragments in the retrieval sub-list to obtain a weighted ranking weight; adjusting the weighted ranking weight using the similarity scores currently corresponding to the knowledge fragments to obtain an adjusted ranking weight; correcting the adjusted ranking weight using the re-occurrence factor scores currently corresponding to the knowledge fragments, and updating the ranking weights of the knowledge fragments in the vector retrieval feedback list.

[0069] Specifically, when a knowledge fragment recurs in multiple search sublists, the ranking weight of the knowledge fragment in the vector search feedback list is weighted and superimposed with the ranking weight of the knowledge fragment in the search sublist to obtain a weighted ranking weight, thereby achieving weight fusion.

[0070] Furthermore, the similarity score currently corresponding to the knowledge fragment is superimposed with the weighted sorting weight to obtain an adjusted sorting weight, and the adjusted sorting weight is corrected using the recurrence factor score currently corresponding to the knowledge fragment, so that the corrected sorting weight is greater than the corrected sorting weight before correction, thereby updating the sorting weight of the knowledge fragment in the vector retrieval feedback list, so that when the sorting weight is updated, the similarity score between the knowledge fragment and different vector features and the number of recurrences can be comprehensively determined, thereby improving the accuracy of the sorting weight corresponding to the knowledge fragment in the vector retrieval feedback list and improving the recall rate of results that are strongly associated with the query statement.

[0071] S204: Obtain a keyword search feedback list that matches the keyword set, and based on the order and number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, deduplicate and reorder the knowledge fragments in the vector search feedback list and the keyword search feedback list to obtain a target search feedback list.

[0072] Specifically, a keyword search feedback list that matches after the keyword set search is obtained, and based on the ranking of knowledge fragments in the vector search feedback list, the ranking of knowledge fragments in the keyword search feedback list, and the number of recurrences of knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated, and the corresponding ranking of the knowledge fragments after deduplication is determined to obtain a target search feedback list.

[0073] In some implementation scenarios, a keyword search feedback list matching a keyword set is obtained, and based on the ranking and number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated and reordered to obtain a target search feedback list, including: searching based on the keyword set to obtain a keyword search feedback list matching the keyword set; wherein, in the keyword search feedback list, there are initial ranking weights corresponding to the ranking of the knowledge fragments; based on the ranking weights of the knowledge fragments in the vector search feedback list, the ranking weights of the knowledge fragments in the keyword search feedback list, and the number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated, and the final ranking weights of the knowledge fragments in the target search feedback list are determined to obtain the target search feedback list.

[0074] Specifically, a keyword search feedback list corresponding to the keyword set is obtained by searching the keyword knowledge base based on the keyword set, wherein the keyword search feedback list is provided with an initial ranking weight corresponding to the ranking of the knowledge fragments, and the initial ranking weight is only associated with the ranking of the knowledge fragments in the keyword search feedback list.

[0075] Optionally, a second query prompt statement is constructed based on the keyword set, the corresponding second query prompt statement is input into the big model, and the big model is used to retrieve a keyword search feedback list corresponding to the keyword set from the keyword knowledge base.

[0076] Furthermore, based on the ranking weights of knowledge fragments in the vector retrieval feedback list, the ranking weights of knowledge fragments in the keyword retrieval feedback list, and the number of recurrences of knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list, the repeated knowledge fragments are deduplicated, and the final ranking weights of the knowledge fragments in the target retrieval feedback list are determined, thereby merging the ranking weights of the knowledge fragments in different retrieval feedback lists obtained by multiple different retrieval methods, and adjusting the fused weights using the number of recurrences to determine a final, more accurate ranking weight of the knowledge fragment in the target retrieval feedback list, so that the more relevant the knowledge fragments are to the query statement in the target retrieval feedback list, the higher they are ranked, thereby improving the recall rate of results that are strongly associated with the query statement.

[0077] It should be noted that the initial query statement and the transcribed query statement have their own query weights, and the keywords in each query statement have their own matching weights. Searching based on the keyword set to obtain a keyword search feedback list matching the keyword set includes: determining the search weights corresponding to the keywords in the keyword set based on the query weights of the query statements to which the keywords in the keyword set belong and the matching weights corresponding to the keywords; searching based on the keywords in the keyword set and their corresponding search weights to obtain a keyword search feedback list matching the keyword set.

[0078] Specifically, each query statement corresponds to a respective query weight, and all query statements correspond to a keyword set consisting of multiple keywords after word segmentation, and each keyword corresponds to a respective matching weight.

[0079] Furthermore, the query weight of the query statement to which the keyword belongs and the matching weight corresponding to the keyword are fused to obtain the retrieval weight corresponding to the keyword in the keyword set, thereby determining the importance of different keywords in the retrieval process. Based on the keywords in the keyword set and the retrieval weights corresponding to the keywords, a keyword retrieval feedback list matching the keyword set is retrieved from the keyword knowledge base to improve the accuracy of keyword retrieval.

[0080] It should be noted that the knowledge fragments in the keyword search feedback list are configured with a preset similarity score; based on the ranking weights of the knowledge fragments in the vector search feedback list, the ranking weights of the knowledge fragments in the keyword search feedback list, and the number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, the knowledge fragments in the vector search feedback list and the keyword search feedback list are deduplicated, and the final ranking weights of the knowledge fragments in the target search feedback list are determined to obtain the target search feedback list, including: merging the knowledge fragments in the vector search feedback list and the keyword search feedback list into a fusion list according to the ranking weights, and combining the knowledge fragments in the fusion list Try to add the target retrieval feedback list in sequence; in response to the knowledge fragment to be added being not included in the target retrieval feedback list, determine the ranking weight of the knowledge fragment in the target retrieval feedback list based on the ranking weight of the knowledge fragment in the fusion list, the similarity score corresponding to the knowledge fragment and the current number of recurrences of the knowledge fragment; in response to the knowledge fragment to be added being included in the target retrieval feedback list, update the number of recurrences of the knowledge fragment, and determine the ranking weight of the knowledge fragment in the target retrieval feedback list based on the ranking weight of the knowledge fragment in the target retrieval feedback list, the ranking weight of the knowledge fragment in the fusion list, the similarity score corresponding to the knowledge fragment and the current number of recurrences of the knowledge fragment.

[0081] Specifically, the knowledge fragments in the keyword search feedback list are configured with a preset similarity score, wherein the preset similarity score can be obtained based on the average value of the similarity scores corresponding to the positive samples obtained by the vector search.

[0082] It can be understood that the vector search results and keyword search results are counted, and the knowledge fragments in the vector search feedback list and the keyword search feedback list are arranged in reverse order according to the sorting weight, so as to merge the multiple search results into the fusion list, and try to add the knowledge fragments in the fusion list to the target search feedback list in turn.

[0083] Furthermore, it is determined whether the knowledge fragment to be added is included in the target search feedback list. If not, the ranking weight of the knowledge fragment in the target search feedback list is determined based on the ranking weight of the knowledge fragment in the fusion list, the similarity score corresponding to the knowledge fragment, and the current number of recurrences of the knowledge fragment. The ranking weight of the knowledge fragment in the fusion list is fused with the similarity score corresponding to the knowledge fragment, and the fused weight is adjusted using the current number of recurrences of the knowledge fragment to obtain the ranking weight of the knowledge fragment in the target search feedback list.

[0084] In a specific implementation scenario, the number of recurrences corresponds to a recurrence factor score, and the above process is expressed by the formula as follows:

[0085] (4)

[0086] Wherein, d is the current knowledge fragment, n is the total number of knowledge fragments retrieved from the index source where the knowledge fragment is located, m is the number of recurrences of the knowledge fragment, v(d) is the similarity score of the current knowledge fragment, and s is the adjustment constant. In addition, in other implementation scenarios, the ranking weight of the knowledge fragment in the fusion list can be superimposed with the similarity score corresponding to the knowledge fragment, and then multiplied by the recurrence factor score, or other specific methods for determining the final ranking weight, which are not specifically limited in this application.

[0087] It can be understood that if the knowledge fragment to be added is already included in the target retrieval feedback list, the number of recurrences of the knowledge fragment is updated, the ranking weight of the knowledge fragment in the target retrieval feedback list and the ranking weight of the knowledge fragment in the fusion list are merged, and the similarity score corresponding to the knowledge fragment and the current number of recurrences of the knowledge fragment are used to adjust the fused weight to determine the ranking weight of the knowledge fragment in the vector retrieval feedback list, thereby achieving knowledge fragment deduplication and obtaining a more accurate ranking weight at the same time, so that the more relevant the knowledge fragment is to the query statement in the target retrieval feedback list, the higher the ranking will be, thereby improving the recall rate of results that are strongly associated with the query statement.

[0088] In this embodiment, the initial query statement is obtained to determine the original expression corresponding to the initial query statement, and the initial query statement is transcribed using other expressions different from the original expression to obtain the transcribed query statement, and vector features are extracted for each query statement to obtain the vector features corresponding to each query statement, and keywords are extracted for all query statements to obtain a keyword set composed of all extracted keywords. Based on each vector feature, a search is performed in the vector index knowledge base to obtain a search sublist matching each vector feature, and based on the initial ranking weight of the knowledge fragment in each search sublist and the number of recurrences of the knowledge fragment in all search sublists, the knowledge fragments that have been repeated are deduplicated, and the final ranking weight of the knowledge fragment in the vector search feedback list is determined, so that the ranking weights of the knowledge fragments in different search sublists are merged, and the fused weights are adjusted using the number of recurrences to determine the final more accurate ranking weight of the knowledge fragment in the vector search feedback list. Based on the keyword set, search is performed in the keyword knowledge base to obtain a keyword search feedback list corresponding to the keyword set, and the ranking weights of knowledge fragments in different search feedback lists obtained by multiple different search methods are fused. The fused weights are adjusted using the number of recurrences to determine the final more accurate ranking weight of the knowledge fragment in the target search feedback list, so that the more relevant the knowledge fragments are to the query statement in the target search feedback list, the higher the ranking will be, thereby improving the recall rate of results that are strongly associated with the query statement.

[0089] See also Figure 3 , Figure 3 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 30 includes a memory 301 and a processor 302 coupled to each other, wherein the memory 301 stores program data (not shown), and the processor 302 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0090] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0091] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0092] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0094] The above description is only an implementation method of the present application, and does not limit the protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A knowledge retrieval feedback method, characterized in that: The method comprises: Obtaining an initial query statement and at least one corresponding transcribed query statement, and determining a vector feature corresponding to each query statement and a keyword set corresponding to all query statements; Obtaining a search sublist for each vector feature match, and based on the order of the knowledge fragments in each search sublist and the number of times the knowledge fragments recur in all the search sublists, de-duplicate and re-order the knowledge fragments in all the search sublists to obtain a vector search feedback list; Obtain a keyword search feedback list that matches the keyword set, and based on the order and recurrence times of knowledge fragments in the vector search feedback list and the keyword search feedback list, deduplicate and reorder the knowledge fragments in the vector search feedback list and the keyword search feedback list to obtain a target search feedback list; Among them, the recurrence number corresponds to a recurrence factor score, the recurrence factor score is positively correlated with the recurrence number corresponding to the knowledge fragment and the total number of indexes, the total number of indexes represents the total number of knowledge fragments from the same index source, and the recurrence factor score is used to increase the ranking weight of the corresponding knowledge fragment in the vector retrieval feedback list.

2. The knowledge retrieval feedback method according to claim 1, characterized in that: The step of obtaining a search sublist for each vector feature match and removing duplicates and reordering the knowledge fragments in all the search sublists based on the order of the knowledge fragments in each search sublist and the number of times the knowledge fragments recur in all the search sublists to obtain a vector search feedback list includes: Perform a search based on each vector feature to obtain a search sublist matching each vector feature; wherein the search sublist has an initial ranking weight corresponding to the ranking of the knowledge fragments; Based on the initial ranking weight of the knowledge fragment in each retrieval sublist and the number of recurrences of the knowledge fragment in all retrieval sublists, the knowledge fragments in all the retrieval sublists are deduplicated, and the final ranking weight of the knowledge fragment in the vector retrieval feedback list is determined to obtain the vector retrieval feedback list.

3. The knowledge retrieval feedback method according to claim 2, characterized in that: There is a corresponding similarity score between the knowledge fragments in the search sublist and the corresponding vector features; The method of removing duplicate knowledge fragments from all the search sublists based on the initial ranking weight of the knowledge fragments in each search sublist and the number of recurrences of the knowledge fragments in all the search sublists, and determining the final ranking weight of the knowledge fragments in the vector search feedback list to obtain the vector search feedback list includes: Try to add the knowledge fragments in the search sub-list to the vector search feedback list in sequence; In response to the knowledge fragment to be added not being included in the vector search feedback list, determining the ranking weight of the knowledge fragment in the vector search feedback list based on the ranking weight of the knowledge fragment in the search sublist and the similarity score corresponding to the knowledge fragment; In response to the knowledge fragment to be added being included in the vector search feedback list, determining the latest number of recurrences of the knowledge fragment, obtaining a recurrence factor score matching the number of recurrences, and updating the ranking weight of the knowledge fragment in the vector search feedback list based on the ranking weight of the knowledge fragment in the vector search feedback list, the ranking weight of the knowledge fragment in the search sublist, and the similarity score and the recurrence factor score currently corresponding to the knowledge fragment; In response to traversing all the search sub-lists, the final ranking weights of the knowledge fragments in the vector search feedback list are determined to obtain the vector search feedback list.

4. The knowledge retrieval feedback method according to claim 3, characterized in that: The updating of the ranking weight of the knowledge fragment in the vector search feedback list based on the ranking weight of the knowledge fragment in the vector search feedback list, the ranking weight of the knowledge fragment in the search sublist, and the similarity score and the recurrence factor score currently corresponding to the knowledge fragment includes: Performing weighted superposition of the ranking weight of the knowledge fragment in the vector search feedback list and the ranking weight of the knowledge fragment in the search sublist to obtain a weighted ranking weight; The weighted sorting weight is adjusted using the similarity score currently corresponding to the knowledge fragment to obtain the adjusted sorting weight, the adjusted sorting weight is corrected using the recurrence factor score currently corresponding to the knowledge fragment, and the sorting weight of the knowledge fragment in the vector retrieval feedback list is updated.

5. The knowledge retrieval feedback method according to claim 1, characterized in that: The step of obtaining a keyword search feedback list matching the keyword set, deduplicating and reordering the knowledge fragments in the vector search feedback list and the keyword search feedback list based on the order and recurrence times of the knowledge fragments in the vector search feedback list and the keyword search feedback list, to obtain a target search feedback list includes: Performing a search based on the keyword set to obtain a keyword search feedback list matching the keyword set; wherein the keyword search feedback list has an initial ranking weight corresponding to the ranking of the knowledge fragments; Based on the ranking weights of the knowledge fragments in the vector retrieval feedback list, the ranking weights of the knowledge fragments in the keyword retrieval feedback list, and the number of recurrences of the knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list, the knowledge fragments in the vector retrieval feedback list and the keyword retrieval feedback list are deduplicated, and the final ranking weights of the knowledge fragments in the target retrieval feedback list are determined to obtain the target retrieval feedback list.

6. The knowledge retrieval feedback method according to claim 5, characterized in that: The original query statement and the transcribed query statement have their own query weights, and the keywords in each query statement have their own matching weights. The step of performing a search based on the keyword set to obtain a keyword search feedback list matching the keyword set includes: Determine the search weight corresponding to the keyword in the keyword set based on the query weight of the query statement to which the keyword in the keyword set belongs and the matching weight corresponding to the keyword; A search is performed based on the keywords in the keyword set and their corresponding search weights to obtain a keyword search feedback list matching the keyword set.

7. The knowledge retrieval feedback method according to claim 5, characterized in that: The knowledge fragments in the keyword search feedback list are configured with preset similarity scores; The method removes duplicates of the knowledge fragments in the vector search feedback list and the keyword search feedback list based on the ranking weights of the knowledge fragments in the vector search feedback list, the ranking weights of the knowledge fragments in the keyword search feedback list, and the number of recurrences of the knowledge fragments in the vector search feedback list and the keyword search feedback list, and determines the final ranking weights of the knowledge fragments in the target search feedback list to obtain the target search feedback list, including: Merge the knowledge fragments in the vector search feedback list and the keyword search feedback list into a fusion list according to the ranking weights, and try to add the knowledge fragments in the fusion list to the target search feedback list in sequence; In response to the knowledge fragment to be added not being included in the target retrieval feedback list, determining the ranking weight of the knowledge fragment in the target retrieval feedback list based on the ranking weight of the knowledge fragment in the fusion list, the similarity score corresponding to the knowledge fragment and the current number of recurrences of the knowledge fragment; In response to the fact that the knowledge fragment to be added is included in the target retrieval feedback list, the number of recurrences of the knowledge fragment is updated, and the ranking weight of the knowledge fragment in the target retrieval feedback list is determined based on the ranking weight of the knowledge fragment in the target retrieval feedback list, the ranking weight of the knowledge fragment in the fusion list, the similarity score corresponding to the knowledge fragment and the current number of recurrences of the knowledge fragment.

8. The knowledge retrieval feedback method according to claim 1, characterized in that: The obtaining of the initial query statement and at least one corresponding transcribed query statement, and determining the vector feature corresponding to each query statement and the keyword set corresponding to all query statements, includes: Obtaining an initial query statement and its corresponding original expression, transcribing the initial query statement using another expression different from the original expression to obtain at least one transcribed query statement; Vector feature extraction is performed on each query statement to obtain the vector feature corresponding to each query statement, and keyword extraction is performed on all query statements to obtain the keyword set corresponding to all query statements.

9. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images