Knowledge retrieval result sorting method and device, equipment, medium and program product

Abstract

The invention provides a knowledge retrieval result sorting method and device, equipment, a medium and a program product. The method comprises the steps of obtaining a first knowledge document list and a second knowledge document list; calculating a plurality of dynamic target ranking control factors based on the real-time interaction amount of each knowledge document; based on each dynamic target ranking control factor, performing comprehensive scoring on a first ranking of each knowledge document in the first knowledge document list and a second ranking of each knowledge document in the second knowledge document list to obtain a comprehensive ranking score value of each knowledge document; and sorting the knowledge documents again to obtain a target knowledge document list. According to the knowledge retrieval result sorting method provided by the invention, through intelligent fusion of the static sorting list and the dynamic sorting list, retrieval results more meeting requirements are provided for the user, and in a scene that the knowledge document updating frequency is not high and behaviors of the user change quickly in a short time, the time hysteresis of sorting is effectively solved, so that the user experience is improved. And the accuracy and the real-time performance of the retrieval result are integrally improved.

Description

Technical Field

[0001] The present invention relates to the technical field of information retrieval, and in particular, to a method, apparatus, device, medium, and program product for sorting knowledge retrieval results. Background Art

[0002] A knowledge base is a knowledge cluster within an enterprise and an important supporting tool for carrying the knowledge of the entire network's business services. By collecting, editing, and storing various types of business knowledge, unified collection, unified storage, and unified management of knowledge can be achieved. Knowledge search refers to performing similarity search and sorting in the knowledge base using keywords, presenting a list of relevant knowledge documents to the user, and the user selects and views the documents of interest in the list to obtain relevant knowledge information.

[0003] For the method of sorting knowledge retrieval results according to relevance, when the content of the knowledge document, the search keyword, and its weight remain unchanged, the sorting result order remains constant, and it is impossible to achieve real-time update of the sorted list, with a large time lag. For a knowledge base search system, especially in scenarios where the update frequency of knowledge documents is not high and the user's behavior changes rapidly in a short period of time, the accuracy and real-time performance of the existing knowledge retrieval sorting method are poor. Summary of the Invention

[0004] The present invention provides a method, apparatus, device, medium, and program product for sorting knowledge retrieval results to solve the problem of poor accuracy and real-time performance of the existing method for sorting knowledge retrieval results.

[0005] In a first aspect, the present invention provides a method for sorting knowledge retrieval results, including: Sorting multiple knowledge documents matched under the current retrieval condition to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting knowledge documents based on the similarity score between the knowledge document and the current retrieval condition; the second knowledge document list is obtained by sorting knowledge documents based on the real-time interaction volume between the knowledge document and the user; Calculating a dynamic target ranking control factor for each knowledge document based on the real-time interaction volume of each knowledge document; Based on each dynamic target ranking control factor, comprehensively scoring the first ranking of each knowledge document in the first knowledge document list and the second ranking of each knowledge document in the second knowledge document list to obtain a comprehensive ranking score value for each knowledge document; the dynamic target ranking control factor is used to adjust the influence degree of the ranking of the knowledge document in different sorted lists on the comprehensive score; Sorting each knowledge document based on each comprehensive ranking score value to obtain a target knowledge document list.

[0006] In one embodiment, calculating a dynamic target ranking control factor for each of the knowledge documents based on the real-time interaction volume of each of the knowledge documents includes: Calculating an average value of the dynamic interaction volume and a standard deviation of the dynamic interaction volume based on the real-time interaction volume of each of the knowledge documents; Calculating a Z-score for each of the real-time interaction volumes based on the average value of the dynamic interaction volume and the standard deviation of the dynamic interaction volume; Quantifying each of the Z-scores to obtain a quantization result for each of the Z-scores; Performing a hyperbolic tangent mapping on each of the quantization results to obtain a dynamic initial ranking control factor for each of the knowledge documents; Calculating a dynamic target ranking control factor for each of the knowledge documents based on each of the dynamic initial ranking control factors.

[0007] In one embodiment, calculating a dynamic target ranking control factor for each of the knowledge documents based on each of the dynamic initial ranking control factors includes: Calculating a difference between a preset ranking control factor and a dynamic initial ranking control factor multiplied by each preset multiple to obtain a dynamic target ranking control factor for each of the knowledge documents.

[0008] In one embodiment, comprehensively scoring the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list based on each of the dynamic target ranking control factors, and performing the following steps for each knowledge document: Adjusting the first ranking of the knowledge document in the first knowledge document list and the second ranking of the knowledge document in the second knowledge document list based on the dynamic target ranking control factor of the knowledge document; Calculating a reciprocal of the adjusted first ranking to obtain a first ranking score value, and calculating a reciprocal of the adjusted second ranking to obtain a second ranking score value; Summing the first ranking score value and the second ranking score value to obtain a comprehensive ranking score value of the knowledge document.

[0009] In one embodiment, sorting each of the knowledge documents based on each of the comprehensive ranking score values to obtain a target knowledge document list includes: Arranging each of the knowledge documents in descending order according to the magnitude of each of the comprehensive ranking score values to obtain a target knowledge document list; If there are knowledge documents with the same ranking in the target knowledge document list, arranging the knowledge documents with the same ranking in descending order according to the magnitude of the similarity scores, and updating the target knowledge document list.

[0010] In one embodiment, after sorting each of the knowledge documents based on each of the comprehensive ranking score values to obtain a target knowledge document list, the method further includes: When it is monitored that the user triggers a click target knowledge document event according to the target knowledge document list, capture click event information and encapsulate the click event information into a message; Store the message in a message queue cluster; Process the messages stored in the message queue cluster in real time and update the real-time interaction volume between the target knowledge document and the user according to the processing results; Store the updated real-time interaction volume between the target knowledge document and the user in a database.

[0011] In a second aspect, the present invention further provides a knowledge retrieval result sorting device, including: A first sorting module, configured to sort multiple knowledge documents matched under a current retrieval condition to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting knowledge documents based on the similarity score between the knowledge documents and the current retrieval condition; the second knowledge document list is obtained by sorting knowledge documents based on the real-time interaction volume between the knowledge documents and the user; A calculation module, configured to calculate a dynamic target ranking control factor for each of the knowledge documents based on the real-time interaction volume of each of the knowledge documents; A comprehensive scoring module, configured to comprehensively score the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list based on each of the dynamic target ranking control factors to obtain a comprehensive ranking score value for each of the knowledge documents; the dynamic target ranking control factor is used to adjust the influence degree of the ranking of knowledge documents in different sorted lists on the comprehensive score; A second sorting module, configured to sort each of the knowledge documents based on each of the comprehensive ranking score values to obtain a target knowledge document list.

[0012] In a third aspect, the present invention provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the steps of the knowledge retrieval result sorting method as described in any one of the above are implemented.

[0013] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the knowledge retrieval result sorting method as described in any one of the above are implemented.

[0014] Fifth aspect, the present invention further provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by the processor, it implements the steps of any one of the above knowledge retrieval result sorting methods.

[0015] The knowledge retrieval result sorting method, device, equipment, medium and program product provided by the present invention match multiple knowledge documents under the current retrieval conditions, comprehensively consider the static sorting list based on semantic retrieval and the dynamic sorting list based on real-time interaction volume, and adjust the sensitivity of the ranking of knowledge documents in the static sorting list and the dynamic sorting list through the dynamic target ranking control factor calculated by the real-time interaction volume, realizing the intelligent fusion of the static sorting list and the dynamic sorting list. It not only reflects the matching degree between the knowledge document and the retrieval conditions, but also reflects the user's preference and interest in the knowledge document. By comprehensively considering various sorting information, it provides retrieval results that better meet the needs of users. Moreover, this hybrid retrieval sorting method has dynamics and real-time performance, and effectively solves the time lag of the knowledge retrieval result sorting in the scenario where the update frequency of knowledge documents is not high and the user's behavior changes rapidly in a short time, thereby improving the accuracy and real-time performance of the knowledge retrieval results as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 is a flowchart of the knowledge retrieval result sorting method provided by the present invention.

[0018] Figure 2 is a flowchart of the real-time storage of interaction volume provided by the present invention.

[0019] Figure 3 is a structural diagram of the knowledge retrieval result sorting device provided by the present invention.

[0020] Figure 4 is a structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein.

[0023] The following will be combined with Figures 1-4 to describe the knowledge retrieval result ranking method, device, equipment, medium, and program product provided by the present invention.

[0024] It should be noted that the knowledge retrieval result ranking method provided in the embodiments of the present invention is implemented based on a knowledge retrieval result ranking device. For the knowledge retrieval result ranking method provided by the present invention, first, N most relevant knowledge documents are obtained using a traditional search and matching method. According to the recent user interaction volume of these N knowledge documents, through an innovative ranking optimization algorithm, the comprehensive score of the ranking of each knowledge document in terms of semantic relevance and the ranking in terms of recent user interaction volume is calculated. Based on the comprehensive scores of each knowledge document, these N knowledge documents are re-ranked, and finally, a document list that best meets the user's search intent is presented to the user.

[0025] In the embodiments of the present invention, the knowledge retrieval result ranking method is described by taking the knowledge retrieval result ranking device as the execution subject.

[0026] Combined with Figure 1 , Figure 1 is a schematic flowchart of the knowledge retrieval result ranking method provided by the present invention.

[0027] As Figure 1 shown, the method includes the following: Step 101: Rank multiple knowledge documents matched under the current retrieval condition to obtain a first knowledge document list and a second knowledge document list; Step 102: Calculate the dynamic target ranking control factor for each knowledge document based on the real-time interaction volume of each knowledge document; Step 103: Based on each dynamic target ranking control factor, comprehensively score the first ranking of each knowledge document in the first knowledge document list and the second ranking in the second knowledge document list to obtain the comprehensive ranking score value of each knowledge document; Step 104: Sort each of the knowledge documents based on the comprehensive ranking score values to obtain a target knowledge document list.

[0028] Specifically, when a user needs to perform knowledge retrieval, the user will enter search keywords related to their needs in the search box on the Web front end, which can be one or more keywords. The system retrieves the knowledge documents in the knowledge base according to the search keywords entered by the user through the traditional retrieval matching method.

[0029] The method to implement this knowledge retrieval function is usually a solution based on distributed Elasticsearch (ES) search. ES is a distributed search and analysis engine with the characteristics of high scalability and high real-time, which is beneficial to search and analyze a large amount of data. Its basic search steps are to first store a large amount of data in the database. When the user searches for data, relevant algorithms are used to score and rank each retrieved knowledge document, and then the ranked return results are presented to the user.

[0030] It can be understood that when the user enters search keywords for query, ES will first perform word segmentation on the search keywords, and the word segments after word segmentation will be used to find matching knowledge documents in the index. In the query analysis process, the fuzzy query and matching query methods of ES are mainly used to meet the search needs of the user. After query analysis, ES will use the inverted index to traverse the list of knowledge documents corresponding to the query word segments.

[0031] Then, ES will calculate the similarity score between each matching knowledge document and the word segment. Currently, the mainstream algorithm of the ES scoring mechanism is based on TF / IDF for scoring, that is, the word segment will match the content of the knowledge document multiple times. The more frequently the word segment appears in the knowledge document, the higher the similarity score between the knowledge document and the word segment will be. Therefore, as long as one field of the knowledge document can be completely matched, it can be retrieved. If multiple knowledge documents are all matched, the one with the highest similarity score among the multiple knowledge documents will be used as the best result of the current retrieval. ES can sort and page the retrieval results according to the similarity scores of the knowledge documents.

[0032] Finally, ES will sort the multiple retrieved and matched knowledge documents in descending order of similarity scores to form a first knowledge document list and output and return it.

[0033] The use of an efficient retrieval mechanism based on an inverted index has greatly improved the query speed, reduced the latency of data processing, and its rich Application Programming Interface (API) and distributed architecture have made the development and maintenance costs relatively low. At the same time, the compatibility with multiple databases and applications ensures the convenience of system integration. In addition, ES supports complex query requirements, has good scalability, and can easily handle the growth of data volume. These characteristics together promote the rapid implementation and efficient operation of technology realization. More importantly, it provides users with fast and accurate retrieval results, significantly improving the user experience, and is an ideal tool for processing large-scale data and achieving efficient search.

[0034] However, although the ES search engine sorts knowledge documents according to similarity scores, this sorting result does not always fully conform to the user's personalized interests. Although the initial sorting of the retrieval results is based on similarity scores, users may be more inclined to click on knowledge documents that highly match their own needs, and these knowledge documents may not always be ranked at the top of the retrieval results. Therefore, in order to improve the user experience, the interaction volume between the knowledge document and the user is introduced, specifically the real-time click volume of the user on the knowledge document, which intuitively reflects the user's preference degree for the knowledge document under the current retrieval conditions. That is to say, a knowledge document with a large interaction volume indicates that this knowledge document better meets the needs of the majority of users under the current retrieval conditions, while a knowledge document with a small interaction volume is the opposite. In addition, in order to maintain the dynamics and real-time nature of the system and at the same time reduce the processing pressure on the system, a mechanism for dynamically updating the interaction volume data is implemented. Through the real-time interaction volume of the knowledge document, it reflects the preference degree in the scenario where the user's behavior changes rapidly in a short period of time, and the pressure on the real-time requirement of the system is not too large, making the system more applicable.

[0035] It can be understood that while sorting knowledge documents based on the similarity score between the knowledge document and the current retrieval conditions, the real-time interaction volume between the knowledge document and the user, that is, the real-time click volume, will also be queried from the database. The multiple knowledge documents matched under the current retrieval conditions are sorted in descending order according to the real-time interaction volume to form a second knowledge document list and output and return.

[0036] Each knowledge document in the first knowledge document list and the second knowledge document list contains a unique document identifier and the corresponding ranking.

[0037] It can be seen from this that the embodiment of the present invention integrates the sorting methods of knowledge retrieval results in multiple dimensions. In order to provide users with retrieval results that better meet their needs, it is necessary to intelligently fuse multiple sorting information, and a reordering mechanism for the following hybrid retrieval dynamic scoring and sorting algorithm is introduced to present results that more conform to the user's interests.

[0038] Based on the real-time interaction volume of each matched knowledge document, calculate the dynamic target ranking control factors of each knowledge document, and these dynamic target ranking control factors can be used to adjust the influence degree of ranking. Since the interaction volume of knowledge documents is real-time, the calculated target ranking control factors are dynamically changing rather than fixed, and can effectively adapt to the preference degree in the scenario where users' behaviors change rapidly in a short period of time.

[0039] Based on the dynamic target ranking control factors of each knowledge document, comprehensively score the first ranking of the knowledge document in the first knowledge document list and the second ranking in the second knowledge document list to obtain the comprehensive ranking score value of the knowledge document.

[0040] Based on the magnitudes of the comprehensive ranking score values of each knowledge document, reorder each knowledge document to form a target knowledge document list, and output and return it to the Web front end for users to select and click to view.

[0041] The knowledge retrieval result sorting method provided by the present invention matches multiple knowledge documents under the current retrieval condition, comprehensively considers the static sorting list based on semantic retrieval and the dynamic sorting list based on real-time interaction volume, and through the dynamic target ranking control factors calculated by the real-time interaction volume, adjusts the sensitivity of the ranking of knowledge documents in the static sorting list and the dynamic sorting list, realizes the intelligent fusion of the static sorting list and the dynamic sorting list, not only reflects the matching degree between the knowledge document and the retrieval condition, but also reflects the user's preference and interest in the knowledge document. By comprehensively considering various sorting information, it provides a retrieval result that better meets the needs of users. Moreover, this hybrid retrieval sorting method has dynamics and real-time performance, and effectively solves the time lag of the knowledge retrieval result sorting in the scenario where the update frequency of knowledge documents is not high while users' behaviors change rapidly in a short period of time, thereby overall improving the accuracy and real-time performance of the knowledge retrieval result.

[0042] In some embodiments, based on step 102, the calculating the dynamic target ranking control factors of each of the knowledge documents based on the real-time interaction volume of each of the knowledge documents includes: Based on the real-time interaction volume of each of the knowledge documents, calculate the average value of the dynamic interaction volume and the standard deviation of the dynamic interaction volume; Based on the average value of the dynamic interaction volume and the standard deviation of the dynamic interaction volume, calculate the Z-score of each of the real-time interaction volumes; Quantify each of the Z-scores to obtain the quantization result of each of the Z-scores; Perform hyperbolic tangent mapping on each of the quantization results to obtain the dynamic initial ranking control factors of each of the knowledge documents; Based on the dynamic initial ranking control factors of each of the knowledge documents, calculate the dynamic target ranking control factors of each of the knowledge documents.

[0043] Specifically, according to the real-time interaction volume of each knowledge document queried in step 101, the average value of dynamic interaction volume and the standard value of dynamic interaction volume under the current search condition are calculated, that is, the average value of dynamic click volume and the standard value of dynamic click volume, and the average value of dynamic interaction volume and the standard value of dynamic interaction volume under the current search condition are stored in the database as the data basis for implementing the hybrid search dynamic scoring algorithm. Similarly, since the interaction volume of knowledge documents is real-time, the calculated average value of interaction volume and the standard value of interaction volume are dynamically changing rather than fixed.

[0044] Based on the characteristics of the sorted list of real-time interaction volume, knowledge documents with high click volume are ranked higher, and the probability of user browsing is higher; conversely, knowledge documents with low click volume are ranked lower, and the probability of user browsing is lower. Therefore, the knowledge base click volume data is more likely to have extreme data samples, that is, data samples far higher or lower than the mean, which can also be called outlier samples. In knowledge base search, such outlier sample documents can often highlight the user's search usage preferences. In order to highlight the scores of these outlier sample documents in the hybrid retrieval dynamic scoring, the z score is introduced to separate such outlier samples.

[0045] The z-score, also called the standard score, is calculated as follows: Among them, X is the real-time interaction volume of a certain knowledge document; is the average value of dynamic interaction; It is the standard value of dynamic interaction volume.

[0046] The Z value represents the distance between the real-time interaction volume and the average value of the dynamic interaction volume, which can indicate the position of the original data in its distribution and can well separate outlier samples. Specifically, taking the normal distribution as an example, about 68% of the data points will fall within the range of plus or minus one standard deviation of the mean value (that is, the z score is -1 to +1), and about 95% of the data points will fall within the range of plus or minus two standard deviations of the mean value (that is, the z score is -2 to +2). In other words, when the z score of the sample data exceeds this range (-2 to +2), it can be regarded as an outlier sample data.

[0047] To quantify the distance between real-time interactions and the average dynamic interactions, the following formula is used to measure this distance: in, is an adjustment factor used to control The sensitivity of the value to changes in the z-score is usually small.

[0048] The theoretical range of z-score values is from negative infinity to positive infinity (i.e., −∞ to +∞). Directly using will result in some samples having values that are too large or too small, which does not conform to the actual situation of dynamic control. Through the tanh mapping transformation based on the z-score, the value range is limited between (-1, 1), that is, finally the hyperbolic tangent function is used to control the smooth adjustment of the value range to obtain the dynamic initial ranking control factor .

[0049] Through the above method, the dynamic initial ranking control factor of each knowledge document can be calculated. Further, based on the dynamic initial ranking control factor of each knowledge document, the dynamic target ranking control factor of each knowledge document is calculated.

[0050] In the embodiment of the present invention, by calculating the Z-score of the real-time interaction volume of the knowledge document, the Z-score is used to quantify and measure the degree of deviation of the interaction volume from the mean. Further, based on the tanh mapping transformation of the Z-score, the dynamic initial ranking control factor is calculated, successfully using its characteristics to quantify the gap between individual extreme data samples and the mean, and dynamically adjusting the hybrid retrieval according to this gap. The hybrid retrieval not only ensures the hybrid retrieval effect of the algorithm, but also enables the hybrid retrieval algorithm to dynamically increase the sorting weight of extreme interaction volume data, and has better applicability to the data characteristics of the system.

[0051] According to the above content, calculating the dynamic target ranking control factor of each knowledge document based on each of the dynamic initial ranking control factors includes: Calculating the difference between the preset ranking control factor and each preset multiple of the dynamic initial ranking control factor to obtain the dynamic target ranking control factor of each knowledge document.

[0052] Specifically, calculating the difference between the preset ranking control factor and each preset multiple of the dynamic initial ranking control factor to obtain the dynamic target ranking control factor of each knowledge document. Among them, the preset ranking control factor is a pre-set standard ranking control factor, which can perform well in various data sets and retrieval tasks, and can provide good balance and robustness in the fusion ranking algorithm; the preset multiple is also pre-set.

[0053] Therefore, the dynamic target ranking control factor of each knowledge document can be calculated by the following formula: where k is the dynamic target ranking control factor; is the preset ranking control factor; the preset multiple takes the value of 10; is the dynamic initial ranking control factor.

[0054] Since the tanh function in the above steps can linearly map the z-score to the range of (-1, 1), the final range of the k value can be smoothly scaled to (50, 70). Since the coefficient value is small and the tanh transition is relatively smooth, the calculation results of most knowledge document data will take values around 60 in the end. Only about 5% of the outlier sample data will take values close to the boundary values of the range (50, 70), that is, the more the interaction volume is higher than the mean, the smaller the k value and the closer it is to 50, and vice versa, the more the interaction volume is lower than the mean, the larger the k value and the closer it is to 70. According to the result of the user interaction volume analysis, the proportion of outlier sample data in the hybrid retrieval ranking score is highlighted according to the above dynamic target ranking control factor. For example, if it is found that in a keyword search, the click volume of a knowledge document is significantly higher than that of other knowledge documents, indicating that users are more inclined to click on the results with higher rankings, then the k value is appropriately reduced through the above algorithm to increase the weight of the documents with higher rankings; otherwise, the k value is increased to reduce the weight of the documents with lower rankings, so that the documents with high click volumes can be more prominent in the hybrid retrieval algorithm.

[0055] The embodiment of the present invention calculates the dynamic initial ranking control factor based on the tanh mapping transformation of the Z-score, solves the problem that the fixed k value in the original RRF algorithm has poor adaptability to extreme samples in the distribution, successfully quantifies the gap between individual extreme data samples and the mean, and dynamically adjusts the hybrid retrieval according to this gap, highlighting the characteristics of high-interaction-volume data more. It not only ensures the hybrid retrieval effect of the algorithm, but also enables the hybrid retrieval algorithm to dynamically increase the sorting weight of extreme interaction-volume data, and has better applicability to the data characteristics of the system.

[0056] In some embodiments, based on step 103, based on each of the dynamic target ranking control factors, the first ranking of each knowledge document in the first knowledge document list and the second ranking in the second knowledge document list are comprehensively scored to obtain the comprehensive ranking score value of each knowledge document. For each knowledge document, the following steps are performed:

[0057] Based on the dynamic target ranking control factor of the knowledge document, adjust the first ranking of the knowledge document in the first knowledge document list and the second ranking in the second knowledge document list; Calculate the reciprocal of the adjusted first ranking to obtain the first ranking score value, and calculate the reciprocal of the adjusted second ranking to obtain the second ranking score value; Sum the first ranking score value and the second ranking score value to obtain the comprehensive ranking score value of the knowledge document. Sum the first ranking score value and the second ranking score value to obtain the comprehensive ranking score value of the knowledge document.

[0058] Specifically, in order to integrate the ranking methods of knowledge retrieval results in multiple dimensions, the Reciprocal Rank Fusion (RRF) algorithm is adopted to dynamically fuse the sorted list based on semantic retrieval and the sorted list based on real-time interaction volume. RRF is based on the concept of reciprocal rank, that is, the reciprocal of the rank of a knowledge document in the retrieval result list, to merge and homogenize the ranking results from different retrievals, and finally generate a comprehensive ranking score value, and then re-rank the knowledge documents according to the comprehensive ranking score value.

[0059] First, based on the dynamic target ranking control factor of the knowledge document, adjust the first ranking of the knowledge document in the first knowledge document list and the second ranking in the second knowledge document list.

[0060] Then, calculate the reciprocal of the adjusted first ranking to obtain the first ranking score value, and calculate the reciprocal of the adjusted second ranking to obtain the second ranking score value.

[0061] Finally, sum the first ranking score value and the second ranking score value to obtain the comprehensive ranking score value of the knowledge document.

[0062] Therefore, the comprehensive ranking score value of the knowledge document is calculated using the reciprocal sorting fusion algorithm, and the algorithm formula is as follows: Among them, d is a certain knowledge document; q represents querying the first knowledge document list or the second knowledge document list; rank(q, d) represents the ranking of the knowledge document d in the result list of query q (counting from 1); k is the dynamic target ranking control factor of the knowledge document d, which is used to adjust the influence degree of the ranking, that is, to adjust the contribution size of the ranking of the knowledge document d in different sorted lists to the final comprehensive score.

[0063] In the embodiments of the present invention, by comprehensively considering the sorting information of the static sorted list and the dynamic sorted list, a dynamic comprehensive score is calculated for each knowledge document, and the knowledge documents are re-ranked according to the dynamic comprehensive score, which can balance the sorting based on interaction volume and the sorting based on relevance, so that the order of the returned document list can not only conform to the user's search habits but also maintain good semantic relevance. Finally, it not only improves the accuracy and relevance of the retrieval results, but also reduces the dependence on a single sorting algorithm, and improves the robustness and stability of the system.

[0064] In some embodiments, based on step 104, the sorting the knowledge documents according to the comprehensive ranking score values to obtain a target knowledge document list includes: Arrange the knowledge documents in descending order according to the magnitudes of the comprehensive ranking score values to obtain a target knowledge document list; If there are knowledge documents with the same ranking in the target knowledge document list, then arrange the knowledge documents with the same ranking in descending order according to the similarity scores, and update the target knowledge document list.

[0065] Specifically, re-arrange the multiple knowledge documents matched under the current retrieval condition in descending order according to the size of their comprehensive ranking score values to obtain the target knowledge document list.

[0066] If there are no knowledge documents with the same ranking in the target knowledge document list, there is no need to update the target knowledge document list, and it can be output and returned to the front-end Web for the user to select and click to view.

[0067] If there are knowledge documents with the same ranking in the target knowledge document list, then arrange the knowledge documents with the same ranking in descending order according to the similarity scores, thereby updating the target knowledge document list, and then return the updated target knowledge document list to the front-end Web for the user to select and click to view.

[0068] In the embodiment of the present invention during the re-ranking process, first arrange the knowledge documents in descending order according to the size of the comprehensive ranking score values. If there are documents with the same ranking, then further arrange these documents in descending order according to the similarity scores to ensure that the ranking is more accurate and reasonable, thereby improving the accuracy of the knowledge retrieval results.

[0069] In some embodiments, after sorting each of the knowledge documents based on each of the comprehensive ranking score values to obtain the target knowledge document list, it further includes: When it is monitored that the user triggers a click on a target knowledge document event according to the target knowledge document list, capture the click event information and encapsulate the click event information into a message; Store the message in the message queue cluster; Process the messages stored in the message queue cluster in real time and update the real-time interaction volume between the target knowledge document and the user according to the processing results; Store the updated real-time interaction volume between the target knowledge document and the user in the database.

[0070] It should be noted that, in order to accurately capture the user's interest preferences for search results, the embodiments of the present invention have constructed a set of interaction volume data collection systems and adopted a mechanism for dynamically updating interaction volume data. The interaction volume data collection system mainly consists of four parts: the Web front end, the message queue cluster, the stream processing application, and the data storage. The Web front end is responsible for capturing the user's click events and generating messages; the message queue cluster, as a message middleware, is responsible for the transmission and storage of messages; the stream processing application is responsible for real-time processing of messages and updating the interaction volume data; and the data storage is used for persistent storage of the interaction volume data.

[0071] Specifically, in the Web front end, by adding click event listeners at key positions on the web page, when it is detected that the user triggers a click event on the target knowledge document according to the target knowledge document list, the front-end code will capture this event and extract the click event information, such as the search keyword, click time, click knowledge topic, and knowledge atomic content, etc. Subsequently, the Web front end encapsulates the click event information into a message and sends the encapsulated message to the specified topic in the message queue cluster through the Producer API of the message queue cluster.

[0072] The message queue cluster usually adopts a Kafka cluster. As a message middleware, the Kafka cluster is responsible for receiving, storing, and forwarding messages from the Web front end. To ensure the high availability and scalability of the system, the Kafka cluster adopts a distributed structure, including multiple Brokers and topics. When configuring the Kafka cluster, appropriate partition numbers and replica numbers are set for each topic, which can be used to store messages from the Web front end and store backups to ensure data reliability and throughput.

[0073] The stream processing application is the core part of this mechanism. It is responsible for real-time processing of messages in the Kafka cluster and updating the interaction volume data. Specifically, the stream processing application subscribes to the specified topic through the Consumer API within the Kafka cluster and continuously listens for new messages in this topic. Once new messages are generated, the stream processing application will immediately read and process these messages. The processing logic includes operations such as data cleaning, aggregation, and calculation of the interaction volume, and updates the interaction volume data of the target knowledge document according to the processing results. After the processing is completed, the stream processing application will call the API interface of the data storage according to the processing results and write the updated interaction volume data into the data storage.

[0074] The data storage is used for persistent storage of the interaction volume data for subsequent analysis and query. According to actual requirements, a relational database can be selected as the data storage.

[0075] In the embodiments of the present invention, through the high-performance message transmission and processing capabilities of the message queue cluster, the system can capture and process the click event information of the Web front-end in real time, and realize the real-time update of the interaction volume. At the same time, the stream processing application can quickly process a large amount of data and update the interaction volume data, improving the processing efficiency of the system. Such an interaction volume dynamic update mechanism based on the message queue cluster ensures advantages such as the timeliness and accuracy of data, and can provide accurate and timely interaction volume data support for subsequent analysis and query.

[0076] Based on the above content, it can be combined with Figure 2 , Figure 2 is provided by the present invention Figure 2 is a schematic flowchart of the real-time storage of the interaction volume provided by the present invention. First, initialize the real-time interaction volume between the knowledge document and the user. Whenever a user conducts a keyword search, through the re-ranking mechanism of the hybrid retrieval dynamic scoring and sorting algorithm in the above steps, the optimal retrieval result arrangement order is finally returned. The user selects and clicks to view the knowledge document according to the finally returned knowledge document list based on their own needs and preferences. At this time, the interaction volume data of the knowledge document can be recorded through data embedding, and the interaction volume data under the current retrieval condition can be dynamically updated. Specifically, it can be the binding association relationship of the search keyword + knowledge document identification number + interaction volume data, and the interaction volume data is dynamically updated. Through this process, real-time interaction volume data support can be provided for subsequent analysis and query.

[0077] In view of the characteristics of the knowledge base system, the balance between the system update frequency and the real-time requirement is carefully evaluated. To reduce the system pressure and ensure the search experience, it is possible to preferentially synchronize the interaction volume update data to the ES database at a frequency of once per minute. This means that when the user searches for the same keyword again later, the retrieval results will be dynamically adjusted based on the latest interaction volume data, presenting the sorting results that best meet the user's expectations to the user.

[0078] Relying on such a full-process closed-loop dynamic real-time search control system that integrates collection, execution, calculation, and adjustment, through the innovative sorting optimization strategy introduced in the embodiments of the present invention and combined with a large amount of user usage habit data, the dynamic update of the retrieval sort list is realized. Compared with the traditional retrieval relevance algorithm, this solution not only ensures the accuracy and relevance of the retrieval results, but also significantly improves the efficiency of the user querying the target knowledge through real-time update and dynamic sorting, bringing a more convenient and efficient retrieval experience to the user.

[0079] It should be noted that the hybrid retrieval dynamic scoring and sorting method introduced in the embodiments of the present invention is not limited to the hybrid ES search sorting results itself, and is also applicable to the sort lists of other similar systems.

[0080] The knowledge retrieval result sorting device provided by the present invention will be described below. The knowledge retrieval result sorting device described below can be correspondingly referred to the knowledge retrieval result sorting method described above.

[0081] Referring to Figure 3 , Figure 3 is a schematic structural diagram of the knowledge retrieval result sorting device provided by the present invention.

[0082] The knowledge retrieval result sorting device includes: A first sorting module 310, configured to sort multiple knowledge documents matched under the current retrieval condition to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting knowledge documents based on the similarity score between the knowledge documents and the current retrieval condition; the second knowledge document list is obtained by sorting knowledge documents based on the real-time interaction volume between the knowledge documents and the user.

[0083] A calculation module 320, configured to calculate a dynamic target ranking control factor for each of the knowledge documents based on the real-time interaction volume of each of the knowledge documents.

[0084] A comprehensive scoring module 330, configured to comprehensively score the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list based on each of the dynamic target ranking control factors to obtain a comprehensive ranking score value for each of the knowledge documents; the dynamic target ranking control factor is used to adjust the influence degree of the ranking of the knowledge document in different sorted lists on the comprehensive score.

[0085] A second sorting module 340, configured to sort each of the knowledge documents based on each of the comprehensive ranking score values to obtain a target knowledge document list.

[0086] The knowledge retrieval result sorting device provided by the present invention matches multiple knowledge documents under the current retrieval condition, comprehensively considers the static sorted list based on semantic retrieval and the dynamic sorted list based on real-time interaction volume, and adjusts the sensitivity of the ranking of the knowledge document in the static sorted list and the dynamic sorted list through the dynamic target ranking control factor calculated by the real-time interaction volume, realizing the intelligent fusion of the static sorted list and the dynamic sorted list, not only reflecting the matching degree between the knowledge document and the retrieval condition, but also reflecting the user's preference and interest in the knowledge document. By comprehensively considering various sorting information, more accurate retrieval results that meet the user's needs are provided for the user. Moreover, this hybrid retrieval sorting method has dynamics and real-time performance, effectively solving the time lag of the knowledge retrieval result sorting in the scenario where the update frequency of knowledge documents is not high while the user's behavior changes rapidly in a short time, thereby overall improving the accuracy and real-time performance of the knowledge retrieval result.

[0087] Further, the calculation module 320 is further configured to: Calculate the average value of the dynamic interaction volume and the standard deviation of the dynamic interaction volume based on the real-time interaction volume of each of the knowledge documents; Calculate the Z-score of each of the real-time interaction volumes based on the average value of the dynamic interaction volume and the standard deviation of the dynamic interaction volume; Quantify each of the Z-scores to obtain the quantization result of each of the Z-scores; Perform hyperbolic tangent mapping on each of the quantization results to obtain the dynamic initial ranking control factor of each of the knowledge documents; Calculate the dynamic target ranking control factor of each of the knowledge documents based on each of the dynamic initial ranking control factors.

[0088] Further, the calculation module 320 is further configured to: Calculate the difference between the preset ranking control factor and each preset multiple of the dynamic initial ranking control factor to obtain the dynamic target ranking control factor of each of the knowledge documents.

[0089] Further, the comprehensive scoring module 330 is further configured to: Adjust the first ranking of the knowledge document in the first knowledge document list and the second ranking in the second knowledge document list based on the dynamic target ranking control factor of the knowledge document; Calculate the reciprocal of the adjusted first ranking to obtain the first ranking score value, and calculate the reciprocal of the adjusted second ranking to obtain the second ranking score value; Sum the first ranking score value and the second ranking score value to obtain the comprehensive ranking score value of the knowledge document.

[0090] Further, the comprehensive scoring module 330 is further configured to: Arrange the knowledge documents in descending order according to the magnitudes of the comprehensive ranking score values to obtain a target knowledge document list; If there are knowledge documents with the same ranking in the target knowledge document list, arrange the knowledge documents with the same ranking in descending order according to the magnitudes of the similarity scores, and update the target knowledge document list.

[0091] Further, the knowledge retrieval result sorting device is further configured to: When it is monitored that the user triggers a click target knowledge document event according to the target knowledge document list, capture the click event information and encapsulate the click event information into a message; Store the message in the message queue cluster; Process the messages stored in the message queue cluster in real time and update the real-time interaction volume between the target knowledge document and the user according to the processing result. Store the real-time interaction volume between the updated target knowledge document and the user in the database.

[0092] It should be noted that the knowledge retrieval result sorting device provided by the present invention can execute the knowledge retrieval result sorting method described in any of the above embodiments during specific operation, and this embodiment will not be elaborated herein.

[0093] Figure 4 is a schematic structural diagram of the electronic device provided by the present invention, as Figure 4 shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 complete mutual communication through the communication bus 440. The processor 410 can call the logical instructions in the memory 430 to execute the knowledge retrieval result sorting method, and the method includes: sorting a plurality of knowledge documents matched under the current retrieval condition to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting knowledge documents based on the similarity score between the knowledge document and the current retrieval condition; the second knowledge document list is obtained by sorting knowledge documents based on the real-time interaction volume between the knowledge document and the user; calculating a dynamic target ranking control factor for each knowledge document based on the real-time interaction volume of each knowledge document; based on each dynamic target ranking control factor, comprehensively scoring the first ranking of each knowledge document in the first knowledge document list and the second ranking in the second knowledge document list to obtain a comprehensive ranking score value for each knowledge document; the dynamic target ranking control factor is used to adjust the influence degree of the ranking of the knowledge document in different sorted lists on the comprehensive score; sorting each knowledge document based on each comprehensive ranking score value to obtain a target knowledge document list.

[0094] In addition, when the logical instructions in the above-mentioned memory 430 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0095] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the knowledge retrieval result sorting method provided in each of the above embodiments. The method includes: sorting a plurality of knowledge documents matched under the current retrieval condition to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting knowledge documents based on the similarity score between the knowledge documents and the current retrieval condition; the second knowledge document list is obtained by sorting knowledge documents based on the real-time interaction volume between the knowledge documents and the user; calculating a dynamic target ranking control factor for each of the knowledge documents based on the real-time interaction volume of each of the knowledge documents; based on each of the dynamic target ranking control factors, comprehensively scoring the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list to obtain a comprehensive ranking score value for each of the knowledge documents; the dynamic target ranking control factor is used to adjust the influence degree of the ranking of knowledge documents in different sorted lists on the comprehensive score; sorting each of the knowledge documents based on each of the comprehensive ranking score values to obtain a target knowledge document list.

[0096] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the knowledge retrieval result sorting method provided in the above embodiments. The method includes: sorting a plurality of knowledge documents matched under the current retrieval condition to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting knowledge documents based on the similarity score between the knowledge documents and the current retrieval condition; the second knowledge document list is obtained by sorting knowledge documents based on the real-time interaction volume between the knowledge documents and the user; calculating a dynamic target ranking control factor for each knowledge document based on the real-time interaction volume of each knowledge document; based on each dynamic target ranking control factor, comprehensively scoring the first ranking of each knowledge document in the first knowledge document list and the second ranking of each knowledge document in the second knowledge document list to obtain a comprehensive ranking score value for each knowledge document; the dynamic target ranking control factor is used to adjust the influence degree of the ranking of knowledge documents in different sorted lists on the comprehensive score; sorting each knowledge document based on each comprehensive ranking score value to obtain a target knowledge document list.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0098] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for ranking knowledge retrieval results, characterized in that: include: Sorting multiple knowledge documents matched under the current search condition to obtain a first knowledge document list and a second knowledge document list; The first knowledge document list is obtained by sorting the knowledge documents based on the similarity scores between the knowledge documents and the current search conditions; the second knowledge document list is obtained by sorting the knowledge documents based on the real-time interaction between the knowledge documents and the user; Calculating a dynamic target ranking control factor of each of the knowledge documents based on the real-time interaction volume of each of the knowledge documents; Based on each of the dynamic target ranking control factors, comprehensively score the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list to obtain a comprehensive ranking score value of each of the knowledge documents; The dynamic target ranking control factor is used to adjust the degree of influence of the ranking of knowledge documents in different ranking lists on the comprehensive score; Based on the comprehensive ranking score values, the knowledge documents are sorted to obtain a target knowledge document list.

2. The knowledge retrieval result ranking method according to claim 1, characterized in that: The step of calculating the dynamic target ranking control factor of each of the knowledge documents based on the real-time interaction amount of each of the knowledge documents includes: Based on the real-time interaction volume of each of the knowledge documents, calculate the average dynamic interaction volume and the standard deviation of the dynamic interaction volume; Calculate the Z score of each real-time interaction amount based on the average dynamic interaction amount and the standard deviation of the dynamic interaction amount; quantifying each of the Z scores to obtain a quantified result of each of the Z scores; Performing hyperbolic tangent mapping on each of the quantification results to obtain a dynamic initial ranking control factor of each of the knowledge documents; Based on each of the dynamic initial ranking control factors, a dynamic target ranking control factor of each of the knowledge documents is calculated.

3. The knowledge retrieval result ranking method according to claim 2, characterized in that: The step of calculating the dynamic target ranking control factor of each of the knowledge documents based on each of the dynamic initial ranking control factors includes: The preset ranking control factor is calculated by difference with each preset multiple of the dynamic initial ranking control factor to obtain the dynamic target ranking control factor of each of the knowledge documents.

4. The knowledge retrieval result ranking method according to claim 3, characterized in that: Based on each of the dynamic target ranking control factors, a comprehensive score is given to the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list to obtain a comprehensive ranking score value of each of the knowledge documents. The following steps are performed for each knowledge document: adjusting a first ranking of the knowledge document in the first knowledge document list and a second ranking of the knowledge document in the second knowledge document list based on a dynamic target ranking control factor of the knowledge document; Perform a countdown calculation on the adjusted first ranking to obtain a first ranking score, and perform a countdown calculation on the adjusted second ranking to obtain a second ranking score; The first ranking score value and the second ranking score value are summed to obtain a comprehensive ranking score value of the knowledge document.

5. The method for ranking knowledge retrieval results according to claim 1, characterized in that: The knowledge documents are sorted based on the comprehensive ranking scores to obtain a target knowledge document list, including: Arrange the knowledge documents in descending order according to the comprehensive ranking scores to obtain a target knowledge document list; If there are knowledge documents with the same ranking in the target knowledge document list, the knowledge documents with the same ranking are arranged in descending order according to the size of the similarity scores, and the target knowledge document list is updated.

6. The method for ranking knowledge retrieval results according to any one of claims 1 to 5, characterized in that: After sorting the knowledge documents based on the comprehensive ranking scores to obtain a target knowledge document list, the method further includes: When monitoring that a user triggers a click event on a target knowledge document according to the target knowledge document list, capturing the click event information, and encapsulating the click event information into a message; Storing the message in a message queue cluster; Processing the messages stored in the message queue cluster in real time, and updating the real-time interaction volume between the target knowledge document and the user according to the processing result; The updated real-time interaction amount between the target knowledge document and the user is stored in the database.

7. A knowledge retrieval result ranking device, characterized in that: include: A first sorting module is used to sort multiple knowledge documents matched under the current search conditions to obtain a first knowledge document list and a second knowledge document list; the first knowledge document list is obtained by sorting the knowledge documents based on the similarity scores between the knowledge documents and the current search conditions; the second knowledge document list is obtained by sorting the knowledge documents based on the real-time interaction between the knowledge documents and the users; A calculation module, used for calculating a dynamic target ranking control factor of each of the knowledge documents based on the real-time interaction amount of each of the knowledge documents; A comprehensive scoring module, used for comprehensively scoring the first ranking of each of the knowledge documents in the first knowledge document list and the second ranking of each of the knowledge documents in the second knowledge document list based on each of the dynamic target ranking control factors, to obtain a comprehensive ranking score value of each of the knowledge documents; The dynamic target ranking control factor is used to adjust the degree of influence of the ranking of knowledge documents in different ranking lists on the comprehensive score; The second sorting module is used to sort the knowledge documents based on the comprehensive ranking score to obtain a target knowledge document list.

8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the knowledge retrieval result ranking method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for ranking knowledge retrieval results as claimed in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for ranking knowledge retrieval results as claimed in any one of claims 1 to 6 are implemented.

Images