Information recall method and device, electronic equipment and storage medium

Abstract

The invention provides an information recall method and device, electronic equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical field of artificial intelligence and large models. According to the specific implementation scheme, query information is obtained; obtaining semantic related information between the query information and candidate contents in the knowledge base; quality evaluation information of the candidate content is determined, wherein the quality evaluation information is related to metadata, used for quality evaluation, of the candidate content; and determining the target content corresponding to the query information from the candidate content according to the semantic related information and the quality evaluation information corresponding to the candidate content, thereby ensuring the recall quality of the target content while ensuring the recall accuracy of the target content.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to the fields of artificial intelligence and large model technology, and particularly to an information retrieval method, apparatus, electronic device, and storage medium. Background Technology

[0002] Knowledge base retrieval and question answering recall is a core component of intelligent question answering systems. It refers to the process by which the system retrieves and returns the most relevant answer candidates to the user's question from the knowledge base. The core objective is to cover all potential correct answers as comprehensively as possible, thus laying the foundation for accurate matching and generation. Currently, most knowledge base retrieval and question answering recall technologies focus on semantic or keyword relevance, resulting in low-quality recalled content. Summary of the Invention

[0003] This disclosure provides an information recall method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this disclosure, an information recall method is provided, comprising: Retrieve query information; Obtain semantically relevant information between the query information and the candidate content in the knowledge base; Determine the quality assessment information of the candidate content, which is related to the metadata of the candidate content used for quality assessment; Based on the semantic information and quality assessment information corresponding to the candidate content, the target content corresponding to the query information is determined from the candidate content.

[0005] According to another aspect of this disclosure, an information recall device is provided, comprising: The first acquisition module is used to acquire query information; The second acquisition module is used to acquire semantically relevant information between the query information and candidate content in the knowledge base; The third acquisition module is used to determine the quality assessment information of the candidate content, wherein the quality assessment information is related to the metadata of the candidate content used for quality assessment. The recall module is used to determine the target content corresponding to the query information from the candidate content based on the semantic information and quality assessment information corresponding to the candidate content.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a flowchart illustrating an information retrieval method provided in an embodiment of this disclosure; Figure 2 This is a schematic flowchart of another information retrieval method provided in this embodiment of the disclosure; Figure 3 This is a schematic flowchart of another information retrieval method provided in this embodiment of the disclosure; Figure 4 This is a schematic flowchart of another information retrieval method provided in this embodiment of the disclosure; Figure 5 This is a schematic diagram of the structure of an information recall device provided in an embodiment of this disclosure; Figure 6 A schematic block diagram of an electronic device used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0011] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0012] Data processing is the collection, storage, retrieval, processing, transformation, and transmission of data. Its basic purpose is to extract and derive valuable and meaningful data from large amounts of potentially messy and difficult-to-understand data.

[0013] Artificial intelligence (AI) is a new technological science that studies and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. It attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence.

[0014] A large model is a machine learning model with a large number of parameters and a complex structure. Its purpose is to improve the model's expressive power and predictive performance, enabling it to process massive amounts of data and complete various complex tasks, such as natural language processing, computer vision, and speech recognition.

[0015] Figure 1 This is a schematic flowchart of an information retrieval method provided in an embodiment of this disclosure. Figure 1 As shown, the method includes: S101, retrieve query information.

[0016] Optionally, the query information can be the query content entered by the user on the client.

[0017] S102, obtain semantic information related to the query information and the candidate content in the knowledge base.

[0018] Understandably, a knowledge base is a database that stores facts, concepts, entities, and their relationships in a structured format, which may include structured data, semi-structured data, unstructured text, and other forms of content.

[0019] Optionally, the query information can be converted into a corresponding query vector, and semantically relevant information can be determined based on the query vector and the feature vector corresponding to each candidate content in the knowledge base. For example, the semantic similarity between the query vector and the feature vector corresponding to each candidate content can be calculated as the semantically relevant information between the query information and the candidate content in the knowledge base.

[0020] S103, determine the quality assessment information of the candidate content.

[0021] Quality assessment information is related to the metadata of candidate content used for quality assessment.

[0022] In some embodiments, metadata is extracted for each candidate content. Metadata is data about data and is used to describe, manage, and retrieve the core attributes of the content. In this embodiment, metadata includes, but is not limited to, document creation / last update time, author / source department, document type (such as "official policy", "technical sharing" or "meeting minutes"), number of times it has been cited / linked, etc.

[0023] Optionally, metadata of candidate content can be extracted using a pre-trained model or configured rules, and a quality score of the candidate content can be determined based on metadata analysis, with the quality score serving as the quality assessment information for the candidate content.

[0024] In some embodiments, the metadata can be inferred and analyzed based on a pre-trained large model and pre-configured prompts to obtain a quality score for the candidate content. This quality score is used to reflect the richness of the candidate content, whether the content has flaws, and whether there are quality issues such as large areas of duplicate content. The lower the quality score, the worse the content quality of the corresponding candidate content. The calculated quality score is then stored.

[0025] S104. Based on the semantic information and quality assessment information corresponding to the candidate content, determine the target content corresponding to the query information from the candidate content.

[0026] Optionally, multiple candidate contents with high semantic similarity can be determined from the candidate contents based on semantically related information, and the candidate contents with the highest quality score can be determined from the multiple candidate contents as the target content corresponding to the query information.

[0027] Optionally, the semantic similarity in semantically related information and the evaluation score in quality assessment information can be weighted and summed, and the summed result can be used as a comprehensive score. The candidate content with the highest comprehensive score can be determined as the target content corresponding to the query information.

[0028] In this embodiment, semantic information related to the query information and candidate content in the knowledge base is obtained, and quality assessment information is determined based on the metadata corresponding to the candidate content. The semantic information and the quality assessment information of the candidate content are comprehensively considered, rather than only considering semantic relevance to determine the target content. This ensures that the target content has a high semantic relevance to the query information while also guaranteeing the quality of the target content itself, thereby improving the quality of information retrieval.

[0029] Figure 2 This is a schematic flowchart of another information retrieval method provided in this embodiment of the disclosure. Figure 2 As shown, the method includes: S201, retrieve query information.

[0030] In this application embodiment, the implementation method of step S201 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0031] S202, obtain semantically relevant information between the query information and the candidate content in the knowledge base.

[0032] Optionally, a query vector for the query information can be obtained; in this embodiment, the query information is transformed based on the Large Language Model (LLM) to obtain the query vector for the query information.

[0033] Obtain the semantic vector of the candidate content, and accordingly, the candidate content can be converted into a semantic vector based on LLM.

[0034] Furthermore, based on the query vector and semantic vector, the semantically relevant information between the query information and the candidate content is determined. For example, the cosine similarity between the query vector and the semantic vector can be calculated, and this cosine similarity can be used as the semantically relevant information between the query information and the candidate content in the knowledge base. The semantically relevant information can accurately reflect the similarity between the query information and the candidate content, thereby improving the accuracy of information retrieval.

[0035] S203, determine the quality assessment information of the candidate content.

[0036] In this application embodiment, the implementation method of step S203 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0037] S204. Generate selection reference information for candidate content based on semantic information and quality assessment information corresponding to the candidate content.

[0038] In some embodiments, the query type to which the query information belongs can be determined, and the weights corresponding to semantically related information and quality assessment information can be determined according to the query type. For example, if the query type of the query information is a policy query, the weight of the quality assessment information can be determined to be greater than the weight of the semantically related information, thereby ensuring the authority of the recalled information. If the query type of the query information is a technical problem investigation, the weight of the semantically related information can be greater than the weight of the quality assessment information, thereby ensuring that the recalled information is related to the technical problem in the query information.

[0039] After determining the respective weights of semantically relevant information and quality assessment information, the semantically relevant information and quality assessment information corresponding to the candidate content are weighted and fused based on the weights to obtain the candidate content screening reference information. The screening reference information reasonably fuses semantically relevant information and the candidate content's own quality assessment information based on the query type. The quality assessment information can be directly read from the storage area. In this embodiment, the screening reference information can be a fusion score, and the calculation method of the fusion score can be expressed as: S final = α*normalize(S relevance ) + (1-α)*normalize(S quality ) Among them, S final For the fusion score; S relevance S is semantically related information; quality α represents quality assessment information; α and 1-α are the weights of semantically relevant information and quality assessment information, respectively.

[0040] S205, based on the screening reference information, reorder the candidate content to obtain the reordering result.

[0041] In this embodiment, the candidate content is reordered in descending order to obtain the reordering result. The higher the fusion score in the screening reference information, the higher the sorting order of the candidate content.

[0042] S206, Based on the re-ranking results, determine the target content corresponding to the query information from the candidate content.

[0043] Optionally, the top K candidate contents in the reordering result can be determined as the target contents corresponding to the query information, where K is an integer greater than or equal to 1.

[0044] In some embodiments, the first candidate content can be determined from the candidate content based on the semantic information between the query information and the candidate content in the knowledge base; alternatively, the semantic information can be sorted, and the candidate content with higher similarity among the semantic information can be used as the first candidate content, for example, N candidate contents with higher semantic information can be determined as the first candidate content, where N is a positive integer.

[0045] Based on the semantic information and quality assessment information corresponding to the first candidate content, the target content corresponding to the query information is determined from the first candidate content. That is, the semantic information and quality assessment information corresponding to the first candidate content are weighted and summed to obtain the filtering reference information. Then, based on the filtering reference information, K candidate contents are determined from N first candidate contents as the target content corresponding to the query information. The filtering is carried out from a finite number of first candidate contents, saving the computational resources of the filtering reference information and improving the efficiency of target content acquisition.

[0046] In this embodiment, the query vector of the query information and the semantic vector of the candidate content are obtained. Based on the cosine similarity between the query vector and each semantic vector, the semantic related information between the query information and each candidate content is determined. After obtaining the quality assessment information, the weights corresponding to the semantic related information and the quality assessment information are determined according to the query type of the query information. Then, the semantic related information and the quality assessment information are weighted and summed according to the weights. The resulting filtering reference information is more consistent with the current query information. Therefore, the target content is re-ranked according to the filtering reference information to obtain more accurate target content, thereby improving the accuracy and quality of information retrieval.

[0047] Figure 3 This is a schematic flowchart of another information retrieval method provided in this embodiment of the disclosure. Figure 3 As shown, the method includes: S301, retrieve query information.

[0048] In this application embodiment, the implementation method of step S301 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0049] S302, obtain semantically relevant information between the query information and the candidate content in the knowledge base.

[0050] In this application embodiment, the implementation method of step S302 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0051] S303, Obtain metadata for candidate content for quality assessment.

[0052] Alternatively, metadata of candidate content can be extracted using a pre-trained model or configured rules.

[0053] S304, Based on at least one of the metadata and candidate content, determine the evaluation parameters of the candidate content on different quality evaluation dimensions.

[0054] Optionally, quality assessment dimensions may include quantifiable and non-quantifiable dimensions, such as quantifiable dimensions like timeliness and professionalism, and non-quantifiable dimensions like richness and factual errors.

[0055] In some embodiments, the metadata includes the update time of the candidate content, and the time interval between the update time of the candidate content and the current time can be determined; based on the time interval and a set decay coefficient, a first evaluation parameter of the candidate content in terms of timeliness is determined, and the probability of recalling outdated information is reduced by using the first evaluation parameter as a reference.

[0056] Optionally, the calculation of the first evaluation parameter can be expressed as: F1 = e (-λ*Δt) Where F1 is the first evaluation parameter, Δt is the time interval between the update time of the candidate content and the current time, and λ is the set decay coefficient, for example, a value of 0.01, which represents the rate at which the information value decays over time.

[0057] In some embodiments, metadata includes source information of candidate content, which can be used to determine the object to which the candidate content belongs. In this embodiment, the source information can be the author of the candidate content, etc., and the object to which the candidate content belongs can be the corresponding author and the department or institution to which the author belongs, etc. Based on the object to which the candidate content belongs, a second evaluation parameter of the candidate content in the professional dimension is determined.

[0058] Optionally, the candidate content can be searched from a preset professional dimension table based on its target audience to obtain a second evaluation parameter for the candidate content in the professional dimension. The professional dimension can correspond to the professionalism dimension of the candidate content, thus avoiding the probability of recalling unprofessional or low-quality information. For example, the second evaluation parameter is: { "Legal Department": 0.95, "CEO Office": 1.0, "Zhang San (Expert)": 0.8, "Li Si (New Employee)": 0.3}.

[0059] In some embodiments, for dimensions for which evaluation parameters cannot be accurately quantified based on metadata, evaluation reference information and evaluation execution information for quality evaluation dimensions can be determined. The evaluation reference information and evaluation execution information are used to correspond to quality scoring standards and scoring requirements. For example, the quality scoring standards define standards for content quality integrity and accuracy, and the scoring requirements include scoring requirements under different quality integrity or accuracy. They may also include information such as the score being in the form of a grade system or a percentage system.

[0060] Based on the evaluation reference information and evaluation execution information, prompts are generated for the large model. These prompts are used to clearly describe user needs, determine the output range of the model, and improve the practicality and reliability of the output results.

[0061] The large model performs quality assessment on candidate content based on the prompts and outputs structured assessment results. These results can include scores for candidate content on auxiliary quality assessment dimensions, such as professionalism, richness, and factual errors. The results are returned in a structured format, such as JSON. Therefore, the assessment parameters for candidate content on auxiliary quality assessment dimensions can be determined based on the structured assessment results, improving the accuracy of parameter acquisition. In this embodiment, the assessment parameters can be represented as: {"Content Value": 0.9, "Richness": 0.7, "Factual Errors": 0, "Severe Watering": 0,...}.

[0062] It is understood that the evaluation parameters of candidate content on different quality evaluation dimensions in this embodiment include at least the first evaluation parameter of candidate content on the timeliness dimension, the second evaluation parameter on the professional dimension, and the evaluation parameter on the auxiliary quality evaluation dimension.

[0063] S305. Determine the quality assessment information of the candidate content based on the weights and assessment parameters corresponding to the quality assessment dimensions.

[0064] In some embodiments, the quality assessment dimensions can be type-identified to determine a first type of quality assessment dimension and a second type of quality assessment dimension. In this embodiment, the first type of quality assessment dimension is a positive dimension, and the second type of quality assessment dimension is a negative dimension. Positive dimensions may include dimensions such as content value, professionalism, timeliness, and content richness, while negative dimensions may include dimensions such as factual errors and large-scale duplication.

[0065] Based on the evaluation parameters corresponding to the second type of quality evaluation dimension, the trigger indication for the second type of quality evaluation dimension is determined. When the evaluation parameter corresponding to the second type of quality evaluation dimension is zero, the trigger indication for the second type of quality evaluation dimension is determined to be 0. When the evaluation parameter corresponding to the second type of quality evaluation dimension is non-zero, the trigger indication for the second type of quality evaluation dimension is determined to be 1.

[0066] The weights and corresponding evaluation parameters of the first quality evaluation dimension, as well as the weights and corresponding trigger indicators of the second quality evaluation dimension, are weighted and calculated to determine the quality evaluation information of the candidate content. This provides a more concise and clear assessment of the candidate content's quality. In this embodiment, the quality evaluation information is the quality score of the candidate content. For example, the expression for calculating the quality score can be: S quality (D) = ( Σ(w i * F i (D)) + Σ(p j * G j (D)) ) / T Among them, S quality (D) represents the quality score of candidate content D; w i F represents the weight of the i-th first-class quality assessment dimension; i (D) represents the evaluation score of candidate content D in the i-th first-class quality dimension; p j G represents the weight of the j-th second-class quality assessment dimension. j (D) is the evaluation score of candidate content D in the j-th second-class quality dimension; T is the normalization factor used to map the score to a specific range, such as 0-100.

[0067] In some embodiments, the weight of each quality assessment dimension can be pre-configured, and the corresponding weight can be obtained based on the quality assessment dimensions in the quality assessment system by establishing a hierarchical quality assessment system.

[0068] For example, the quality assessment system may include: a first-level dimension, a second-level dimension, and a third-level dimension. The quality assessment dimensions corresponding to the first-level dimension are the first type of quality assessment dimensions, specifically including professionalism, timeliness, richness, and content value. The quality assessment dimensions corresponding to the second-level dimension are also the first type of quality assessment dimensions, specifically including demand fulfillment, content defects, and relevance. In this embodiment, the weight of the first dimension is greater than the weight of the second dimension. The quality assessment dimensions corresponding to the third dimension are the second type of quality assessment dimensions, specifically including red-line issues, serious embellishment, factual errors, large-scale duplication, missing core content, and advertising, etc. The weight of the third dimension is a negative dimension. For example, the weight of the first dimension is 3, the weight of the second dimension is 2, and the weight of the third dimension is -5, thereby determining the weight of each quality assessment dimension according to the quality assessment system.

[0069] S306. Based on the semantic information and quality assessment information corresponding to the candidate content, determine the target content corresponding to the query information from the candidate content.

[0070] In this application embodiment, the implementation method of step S306 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0071] In this embodiment, after obtaining the metadata of the candidate content, the first and second evaluation parameters of the candidate content in the timeliness and professionality dimensions are obtained based on the metadata. Prompt information is constructed based on evaluation reference information and evaluation execution information. Evaluation parameters of the candidate content in different auxiliary quality evaluation dimensions are obtained based on the prompt information and the pre-trained large model, thus obtaining the evaluation parameters of the candidate content in quantifiable and non-quantifiable quality evaluation dimensions. Based on the rich evaluation parameters of the candidate content, quality evaluation information is obtained. This quality evaluation information fully reflects the completeness and correctness of the candidate content quality. Target content is determined from the candidate content based on the quality evaluation information and semantically relevant information, ensuring that the target content is strongly correlated with the query information while guaranteeing the authority, professionalism, and effectiveness of the target content.

[0072] Figure 4 This is a schematic flowchart of another information retrieval method provided in this embodiment of the disclosure. Figure 4 As shown, the method includes: S401, retrieve query information.

[0073] In this application embodiment, the implementation method of step S401 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0074] S402, obtain semantically relevant information between the query information and the candidate content in the knowledge base.

[0075] In this application embodiment, the implementation method of step S402 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0076] S403, Obtain metadata for candidate content for quality assessment.

[0077] In this application embodiment, the implementation method of step S403 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0078] S404, based on at least one of the metadata and candidate content, determine the evaluation parameters of the candidate content on different quality evaluation dimensions.

[0079] In this application embodiment, the implementation method of step S404 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0080] S405, determine the quality assessment information of candidate content based on the weights and assessment parameters corresponding to the quality assessment dimensions.

[0081] In this application embodiment, the implementation method of step S405 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0082] S406. Based on the semantic information and quality assessment information corresponding to the candidate content, generate screening reference information for the candidate content.

[0083] In some embodiments, the presence of setting quality issues can be determined based on the evaluation parameters of the candidate content on different quality evaluation dimensions. For example, setting quality issues may include serious problems such as factual errors.

[0084] In response to the existence of quality issues in the candidate content, a corresponding penalty function is determined based on the quality issues. In this embodiment, the penalty function can be a very small value, such as 0.01, which serves as a "veto".

[0085] Furthermore, based on the semantically relevant information and quality assessment information corresponding to the candidate content, as well as the penalty function, screening reference information for candidate content is determined. This screening reference information can effectively avoid selecting candidate content with serious quality problems for recall and avoid interference from low-quality candidate content. The calculation of the fusion score of semantically relevant information and quality assessment information in the screening reference information can be expressed as: S final = S relevance *(1+β*S quality )*P(D) Among them, S final For the fusion score; S relevance S is semantically related information; quality β represents quality assessment information; β is the quality bonus coefficient; P(D) is the penalty function.

[0086] In this application embodiment, the implementation method of step S406 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0087] S407, Based on the filtering reference information, the candidate content is reordered to obtain the reordering result.

[0088] In this application embodiment, the implementation method of step S407 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0089] S408, Based on the re-ranking results, determine the target content corresponding to the query information from the candidate content.

[0090] In this application embodiment, the implementation method of step S408 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0091] In this embodiment, semantically relevant information between the query information and each candidate content is determined based on the query vector of the query information and the semantic vector of the candidate content. Metadata of the candidate content is obtained, and first and second evaluation parameters of the candidate content in terms of timeliness and professionalism are obtained based on the metadata. Evaluation parameters of the candidate content in different auxiliary quality evaluation dimensions are obtained based on the large model. Thus, the quality evaluation information of the candidate content is obtained based on the rich evaluation parameters of the candidate content, which fully reflects the completeness and correctness of the candidate content quality. The semantically relevant information and quality evaluation information are weighted and summed to obtain screening reference information. Based on the screening reference information, more accurate target content is obtained. This ensures that the target content has a high semantic relevance to the query information, while also guaranteeing the authority, professionalism, and effectiveness of the target content itself. It avoids being misled by low-quality or outdated content and significantly improves decision-making and work efficiency.

[0092] Figure 5 This is a schematic diagram of the structure of an information retrieval device provided in an embodiment of this disclosure. Figure 5 As shown, the information recall device 500 includes: The first acquisition module 501 is used to acquire query information; The second acquisition module 502 is used to acquire semantically relevant information between the query information and the candidate content in the knowledge base; The third acquisition module 503 is used to determine the quality assessment information of the candidate content, which is related to the metadata of the candidate content used for quality assessment. The recall module 504 is used to determine the target content corresponding to the query information from the candidate content based on the semantic information and quality assessment information corresponding to the candidate content.

[0093] In some embodiments, the recall module 504 is configured to: Based on the semantic information and quality assessment information corresponding to the candidate content, generate screening reference information for the candidate content; Based on the screening reference information, the candidate content is reordered to obtain the reordering result; Based on the re-ranking results, the target content corresponding to the query information is determined from the candidate content.

[0094] In some embodiments, the recall module 504 is configured to: Determine the query type to which the query information belongs; Based on the query type, determine the weights corresponding to semantically relevant information and quality assessment information; Based on the weights, the semantic information and quality assessment information corresponding to the candidate content are weighted and fused to obtain the selection reference information for the candidate content.

[0095] In some embodiments, the recall module 504 is configured to: Based on the evaluation parameters of the candidate content across different quality assessment dimensions, determine whether the candidate content has any defined quality issues; In response to the existence of quality issues in the candidate content, a corresponding penalty function is determined based on the quality issues. Based on the semantic information and quality assessment information corresponding to the candidate content, as well as the penalty function, the selection reference information for the candidate content is determined.

[0096] In some embodiments, the recall module 504 includes: Based on the semantic information between the query information and the candidate content in the knowledge base, the first candidate content is determined from the candidate content; Based on the semantic information and quality assessment information corresponding to the first candidate content, the target content corresponding to the query information is determined from the first candidate content.

[0097] In some embodiments, the third acquisition module 503 is configured to: Obtain metadata for quality assessment of candidate content; Based on at least one of the metadata and candidate content, determine the evaluation parameters of the candidate content on different quality evaluation dimensions; Based on the weights and evaluation parameters corresponding to the quality assessment dimensions, the quality assessment information of the candidate content is determined.

[0098] In some embodiments, the third acquisition module 503 is configured to: Determine the time interval between the update time of the candidate content and the current time; Based on the time interval and the set attenuation coefficient, the first evaluation parameter for the candidate content in terms of timeliness is determined.

[0099] In some embodiments, the third acquisition module 503 is configured to: Based on the source information, determine the object to which the candidate content belongs; Based on the target audience of the candidate content, determine the second evaluation parameter for the candidate content in the professional dimension.

[0100] In some embodiments, the third acquisition module 503 is configured to: Determine the assessment reference information and assessment implementation information for the quality assessment dimensions; Based on the assessment reference information and assessment execution information, generate prompts for the large model; The large model performs quality assessment on candidate content based on the prompts and outputs structured assessment results. Based on the structured evaluation results, the evaluation parameters for the candidate content in the auxiliary quality evaluation dimensions are determined.

[0101] In some embodiments, the third acquisition module 503 is configured to: Type identification is performed on the quality assessment dimensions to determine the first type of quality assessment dimension and the second type of quality assessment dimension; Based on the assessment parameters corresponding to the second type of quality assessment dimension, determine the trigger indication for the second type of quality assessment dimension; The weights and corresponding evaluation parameters of the first quality assessment dimension, as well as the weights and corresponding triggering indicators of the second quality assessment dimension, are weighted and calculated to determine the quality assessment information of the candidate content.

[0102] In some embodiments, the second acquisition module 502 is configured to: Retrieve the query vector for query information; Obtain the semantic vector of the candidate content; Based on the query vector and semantic vector, determine the semantically relevant information between the query information and the candidate content.

[0103] In this embodiment, semantically relevant information between the query information and each candidate content is determined based on the query vector of the query information and the semantic vector of the candidate content. Metadata of the candidate content is obtained, and first and second evaluation parameters of the candidate content in terms of timeliness and professionalism are obtained based on the metadata. Evaluation parameters of the candidate content in different auxiliary quality evaluation dimensions are obtained based on the large model. Thus, the quality evaluation information of the candidate content is obtained based on the rich evaluation parameters of the candidate content, which fully reflects the completeness and correctness of the candidate content quality. The semantically relevant information and quality evaluation information are weighted and summed to obtain screening reference information. Based on the screening reference information, more accurate target content is obtained. This ensures that the target content has a high semantic relevance to the query information, while also guaranteeing the authority, professionalism, and effectiveness of the target content itself. It avoids being misled by low-quality or outdated content and significantly improves decision-making and work efficiency.

[0104] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0105] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0106] Figure 6 A schematic block diagram of an electronic device for implementing embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0107] like Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0108] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0109] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the information retrieval method. For example, in some embodiments, the information retrieval method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the information retrieval method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the information retrieval method by any other suitable means (e.g., by means of firmware).

[0110] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0112] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0114] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0115] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0116] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0117] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information retrieval method, wherein, The method includes: Retrieve query information; Obtain semantically relevant information between the query information and the candidate content in the knowledge base; Determine the quality assessment information of the candidate content, which is related to the metadata of the candidate content used for quality assessment; Based on the semantic information and quality assessment information corresponding to the candidate content, the target content corresponding to the query information is determined from the candidate content.

2. The method according to claim 1, wherein, The step of determining the target content corresponding to the query information from the candidate content based on the semantic information and quality assessment information corresponding to the candidate content includes: Based on the semantic information and quality assessment information corresponding to the candidate content, filter reference information for the candidate content is generated; Based on the filtering reference information, the candidate content is reordered to obtain the reordering result; Based on the re-sorting results, the target content corresponding to the query information is determined from the candidate content.

3. The method according to claim 2, wherein, The step of generating screening reference information for the candidate content based on the semantically relevant information and quality assessment information corresponding to the candidate content includes: Determine the query type to which the query information belongs; Based on the query type, determine the weights corresponding to the semantic information and quality assessment information; Based on the weights, the semantic information and quality assessment information corresponding to the candidate content are weighted and fused to obtain the screening reference information for the candidate content.

4. The method according to claim 2, wherein, The step of generating screening reference information for the candidate content based on the semantically relevant information and quality assessment information corresponding to the candidate content includes: Based on the evaluation parameters of the candidate content on different quality assessment dimensions, determine whether the candidate content has any set quality issues; In response to the existence of a defined quality problem in the candidate content, a corresponding penalty function is determined based on the defined quality problem; Based on the semantic information and quality assessment information corresponding to the candidate content, as well as the penalty function, the screening reference information for the candidate content is determined.

5. The method according to claim 1, wherein, The step of determining the target content corresponding to the query information from the candidate content based on the semantic information and quality assessment information corresponding to the candidate content includes: Based on the semantic information between the query information and the candidate content in the knowledge base, a first candidate content is determined from the candidate content; Based on the semantic information and quality assessment information corresponding to the first candidate content, the target content corresponding to the query information is determined from the first candidate content.

6. The method according to any one of claims 1-5, wherein, The quality assessment information for determining the candidate content includes: Obtain metadata for quality assessment of the candidate content; Based on the metadata and at least one of the candidate contents, determine the evaluation parameters of the candidate contents on different quality evaluation dimensions; The quality assessment information of the candidate content is determined based on the weights corresponding to the quality assessment dimensions and the assessment parameters corresponding to the quality assessment dimensions.

7. The method according to claim 6, wherein, The metadata includes the update time of the candidate content. Based on the metadata, the evaluation parameters of the candidate content on different quality evaluation dimensions are determined, including: Determine the time interval between the update time of the candidate content and the current time; Based on the time interval and the set attenuation coefficient, the first evaluation parameter of the candidate content in the timeliness dimension is determined.

8. The method according to claim 6, wherein, The metadata includes source information of the candidate content. Based on the metadata, evaluation parameters for the candidate content across different quality assessment dimensions are determined, including: Based on the source information, determine the object to which the candidate content belongs; Based on the object to which the candidate content belongs, determine the second evaluation parameter of the candidate content in the professional dimension.

9. The method according to claim 6, wherein, Based on the candidate content, determine the evaluation parameters of the candidate content on different quality assessment dimensions, including: Determine the assessment reference information and assessment execution information for the aforementioned quality assessment dimensions; Based on the evaluation reference information and evaluation execution information, generate prompt information for the large model; The large model performs a quality assessment on the candidate content based on the prompt information and outputs a structured assessment result. Based on the structured evaluation results, the evaluation parameters of the candidate content in the auxiliary quality evaluation dimensions are determined.

10. The method according to claim 6, wherein, The step of determining the quality assessment information of the candidate content based on the weights corresponding to the quality assessment dimensions and the assessment parameters corresponding to the quality assessment dimensions includes: Type identification is performed on the quality assessment dimensions to determine the first type of quality assessment dimension and the second type of quality assessment dimension; Based on the evaluation parameters corresponding to the second type of quality evaluation dimension, determine the trigger indication for the second type of quality evaluation dimension; The quality assessment information of the candidate content is determined by performing a weighted calculation on the weights and corresponding assessment parameters of the first type of quality assessment dimension, and the weights and corresponding trigger indicators of the second type of quality assessment dimension.

11. The method according to any one of claims 1-5, wherein, The step of obtaining semantically relevant information between the query information and candidate content in the knowledge base includes: Obtain the query vector of the query information; Obtain the semantic vector of the candidate content; Based on the query vector and semantic vector, determine the semantically relevant information between the query information and the candidate content.

12. An information recall device, comprising: The first acquisition module is used to acquire query information; The second acquisition module is used to acquire semantically relevant information between the query information and candidate content in the knowledge base; The third acquisition module is used to determine the quality assessment information of the candidate content, wherein the quality assessment information is related to the metadata of the candidate content used for quality assessment. The recall module is used to determine the target content corresponding to the query information from the candidate content based on the semantic information and quality assessment information corresponding to the candidate content.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-11.

Images