Information screening method, device, equipment and storage medium

Abstract

The application provides an information screening method and device, equipment and a storage medium, and belongs to the technical field of information screening. The method comprises the following steps: determining at least one query word based on a search word; performing information query based on the query word to obtain a plurality of information query results; in response to the plurality of information query results containing redundant information and conflict information, performing an information screening operation to obtain an information screening result; the information screening operation comprises the following steps: acquiring the publishing time and the information source corresponding to all the information query results; determining an untrustworthy information source from the information sources based on the publishing time of each piece of redundant information; processing the redundant information in the plurality of information query results based on the untrustworthy information source to obtain a first screening result; and processing the conflict information in the first screening result based on the publishing time corresponding to each piece of conflict information and the untrustworthy information source to obtain the information screening result. The application can improve the accuracy of information screening.

Description

Technical Field

[0001] This application belongs to the field of information filtering technology, and more specifically, relates to an information filtering method, apparatus, device and storage medium. Background Technology

[0002] With the advent of the big data era, the amount of information available online is increasing, bringing convenience but also a series of problems. Taking investment promotion as an example, tools such as models can be used to obtain relevant company information based on user search needs, and then invitations can be sent to target companies. However, due to the sheer volume of data, the information collected from the internet using models often contains a large amount of redundancy or conflict. Purely manual screening is inefficient, and models are weak in handling redundancy and conflicts, thus leading to inaccurate information. Summary of the Invention

[0003] The purpose of this application is to provide an information filtering method, apparatus, device, and storage medium to improve the accuracy of information filtering.

[0004] A first aspect of this application provides an information filtering method, including:

[0005] At least one query term must be determined based on the search terms; the query terms must include at least the search terms.

[0006] Information search based on query terms yields multiple information search results;

[0007] In response to the presence of redundant and conflicting information in multiple information query results, an information filtering operation is performed to obtain the filtered information results;

[0008] The information filtering operation includes:

[0009] Retrieve the publication time and information source corresponding to all information query results;

[0010] At least based on the release time of each redundant piece of information, untrusted information sources can be identified from various information sources;

[0011] The redundant information in multiple information query results is processed based on untrusted information sources to obtain the first filtering result;

[0012] The conflicting information in the first screening result is processed based on the release time and untrusted information sources corresponding to each conflicting information to obtain the information screening result.

[0013] A second aspect of this application provides an information filtering device, comprising:

[0014] The term determination module is used to determine at least one query term based on the search terms; the query term must include at least the search term.

[0015] The information query module is used to perform information queries based on query terms and obtain multiple information query results;

[0016] The information filtering module is used to perform information filtering operations and obtain information filtering results when multiple information query results contain redundant or conflicting information.

[0017] The information filtering operation includes:

[0018] Retrieve the publication time and information source corresponding to all information query results;

[0019] At least based on the release time of each redundant piece of information, untrusted information sources can be identified from various information sources;

[0020] The redundant information in multiple information query results is processed based on untrusted information sources to obtain the first filtering result;

[0021] The conflicting information in the first screening result is processed based on the release time and untrusted information sources corresponding to each conflicting information to obtain the information screening result.

[0022] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the information filtering method described above.

[0023] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the information filtering method described above.

[0024] The beneficial effects of the information filtering method, apparatus, device, and storage medium provided in this application embodiment are as follows:

[0025] This application embodiment obtains the publication time and information source corresponding to all information query results. At least based on the publication time of each redundant piece of information, it identifies untrusted information sources from among the various information sources. Then, based on these untrusted information sources, it processes the redundant information in multiple information query results to obtain a first filtering result. By considering the factors of information publication time and information source, it can more accurately identify and remove redundant information, avoiding interference from redundant information that affects the filtering accuracy. Furthermore, this application embodiment also processes the conflicting information in the first filtering result based on the publication time and untrusted information sources corresponding to each conflicting piece of information to obtain the final information filtering result. When processing conflicting information, combining the publication time with the untrusted information sources already identified during the processing of redundant information allows for a more reasonable judgment of which conflicting information is reliable and which is unreliable, thereby effectively solving the problem of inaccurate filtering caused by conflicting information and improving the accuracy of information filtering. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating a first information filtering method provided in an embodiment of this application;

[0028] Figure 2 A flowchart illustrating a second information filtering method provided in an embodiment of this application;

[0029] Figure 3 A structural block diagram of an information filtering device provided in an embodiment of this application;

[0030] Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0033] Please refer to Figure 1 , Figure 1 The flowchart of a first information filtering method provided in an embodiment of this application is shown. It can be executed by an electronic device and may include: S101-S103.

[0034] S101: Determine at least one query term based on the search term.

[0035] In this embodiment, the search term refers to the words entered by the user, and the query term refers to the words generated based on the search term for actual information retrieval. The query term must include at least the user's original input, i.e., the search term. Since different information sources may describe the same thing differently, using only the search term for querying may easily miss relevant information. Therefore, in this embodiment, at least one query term can be determined based on the search term. For example, synonyms of the search term can be generated through an industry thesaurus or a pre-trained language model to obtain the query term. If there is no synonym for the search term in the industry thesaurus, the search term can be directly determined as the query term.

[0036] Taking the investment promotion industry as an example, if a user enters the search terms "new energy vehicles, core component enterprises", the corresponding query terms can be "new energy vehicles, core component enterprises, key component manufacturers, EV core component enterprises", etc.

[0037] S102: Perform information retrieval based on query terms to obtain multiple information query results.

[0038] In this embodiment, information can be searched from multiple publicly available information sources based on the obtained query terms, resulting in multiple query results. These information sources can be authoritative official sources, industry databases, or official corporate accounts, etc.

[0039] In this embodiment, the information query result refers to the set of information output by the query action. For example, if the query terms are "new energy vehicles, core component enterprises, key component manufacturers, EV core component enterprises", then the information query result can be multiple information entries such as "Company A belongs to the new energy vehicle industry", "Company B is a core component enterprise in the new energy vehicle industry", and "Company C is a key component manufacturer in the new energy vehicle industry".

[0040] S103: In response to the presence of redundant and conflicting information in multiple information query results, perform an information filtering operation to obtain the information filtering results.

[0041] In this embodiment, redundant information refers to consistent information provided by different information sources for the same attribute dimension of the same information subject; conflicting information refers to inconsistent information provided by different information sources for the same attribute dimension of the same information subject. The information subject refers to the entity described by each piece of information in the information query results. For example, "Company A," "Company B," and "Company C" mentioned above are all information subjects. The attribute dimension refers to a specific category describing the characteristics or attributes of the information subject, such as identity identification or business status.

[0042] refer to Figure 2 In this embodiment, if multiple information query results contain both redundant and conflicting information, the information filtering operation can be performed according to the following steps S1-S4:

[0043] S1: Retrieve the publication time and information source corresponding to all information query results.

[0044] S2: Identify untrusted information sources from among the various information sources, at least based on the release time of each redundant information.

[0045] In this embodiment, if one information source publishes a large amount of redundant information, and the time of publication of this redundant information is later than the publication time of other information sources, it can be identified as an untrusted information source. This means that such information sources have poor timeliness and high redundancy. In this embodiment, in addition to the publication time of the redundant information, other dimensions such as whether the information source publishing the redundant information is an official data source can also be used to determine untrusted information sources.

[0046] S3: Based on untrusted information sources, redundant information in multiple information query results is processed to obtain the first filtering result.

[0047] In this embodiment, after identifying untrusted information sources, redundant information in multiple information query results can be deleted. For example, all redundant information published by untrusted information sources can be deleted from each set of redundant information. If there is still remaining redundant information, any redundant information can be deleted, and only one of the identical redundant information can be retained. It should be noted that in this embodiment, only redundant information published by untrusted information sources is deleted; other information published by untrusted information sources is not processed in step S3. In this embodiment, considering that the redundant information is consistent, it is also possible to disregard whether its source is an untrusted information source and simply retain one piece of redundant information with consistent content. Most preferably, redundant information published by untrusted information sources can be deleted to reduce the number of untrusted information sources among the information sources of each piece of information in the first filtering result.

[0048] S4: Based on the release time and untrusted information sources corresponding to each conflicting information, process the conflicting information in the first screening result to obtain the information screening result.

[0049] In this embodiment, information from untrusted information sources can be removed from the conflicting information, and the remaining conflicting information can be processed. In this embodiment, the processing of the remaining conflicting information takes into account that the conflicting information may be caused by data updates rather than by errors in the information source's published information. Therefore, in this embodiment, the timeliness parameter and the preset credibility parameter can be determined based on the publication time of the conflicting information to determine whether to remove it and obtain the final information filtering result.

[0050] Specifically, the conflicting information in the first screening result is processed based on the release time and untrusted information sources corresponding to each conflicting piece of information to obtain the information screening result, including:

[0051] The conflicting information in the first screening result is classified according to the information subject corresponding to the conflicting information to obtain at least one conflict group; each conflict group contains at least two different conflicting information against the same information subject.

[0052] For each conflict group, from all the conflict information contained in the conflict group, the conflict information corresponding to the untrusted information source is removed to obtain the candidate conflict information set; for each candidate conflict information in the candidate conflict information set, the trust weight of the candidate conflict information is calculated based on the preset trust parameter of its corresponding information source and the timeliness parameter of the release time.

[0053] The information filtering results are determined based on the credibility weights of the candidate conflict information corresponding to each conflict group.

[0054] In this embodiment, each information source corresponds to a preset credibility parameter, which can be set by those skilled in the art based on historical data or whether it is an official source. In this embodiment, the credibility weight of the candidate conflicting information can be obtained by weighting the preset credibility parameter and the timeliness parameter. To ensure the same dimensions, the preset credibility parameter can be a value in the range of 0-1. Similarly, the timeliness parameter should also be a value in the range of 0-1. The timeliness parameter corresponding to different publication times and the current time can be determined based on a preset mapping relationship. The weights of the preset credibility parameter and the timeliness parameter can be set independently, for example, 0.6 and 0.4 respectively.

[0055] For example, the timeliness parameter can be determined as follows: if the time difference between the publication time and the current time is within one month, the timeliness parameter can be set to 1; if the time difference is between one and six months, the timeliness parameter can be set to 0.7; if the time difference is between six months and one and a half years, the timeliness parameter can be set to 0.5; and if the time difference exceeds one and a half years, the timeliness parameter can be set to 0.3. In addition to the above settings, linear decay and other methods can also be set, which are not limited in this embodiment. In this embodiment, the above steps can be implemented based on a model or other tools with retrieval and query functions.

[0056] As can be seen from the above, this embodiment of the application obtains the publication time and information source corresponding to all information query results, determines untrusted information sources from each information source based at least on the publication time of each redundant information, and then processes the redundant information in multiple information query results based on the untrusted information sources to obtain the first screening result. By considering the factors of information publication time and information source, it can more accurately identify and remove redundant information, avoiding the impact of redundant information interference on the screening accuracy. Furthermore, this embodiment of the application also processes the conflicting information in the first screening result based on the publication time and untrusted information sources corresponding to each conflicting information to obtain the final information screening result. When processing conflicting information, combining the publication time with the untrusted information sources already determined during the processing of redundant information allows for a more reasonable judgment of which conflicting information is reliable and which is unreliable, thereby effectively solving the problem of inaccurate screening caused by conflicting information and improving the accuracy of information screening.

[0057] In one embodiment of this application, determining untrusted information sources from various information sources based at least on the publication time of each piece of redundant information may include:

[0058] For each piece of redundant information, take that piece of redundant information as the target redundant information, filter out other redundant information whose publication time is earlier than that of the target redundant information, calculate the similarity between the text containing the target redundant information and the text containing other redundant information, and obtain the similarity corresponding to the target redundant information; take the average of the similarity corresponding to each target redundant information as the information similarity of the target information source corresponding to the target redundant information.

[0059] Calculate the proportion of redundant information published by the target information source in all redundant information to obtain the redundancy contribution of the target information source;

[0060] The average difference between the release time of redundant information in the target information source and the release time of corresponding redundant information in other information sources is calculated to obtain the update lag of the target information source.

[0061] The score of the target information source is determined based on information similarity, redundancy contribution, and update lag.

[0062] Untrusted information sources are identified from among the various information sources based on the scores of each target information source.

[0063] In this embodiment, other redundant information refers to redundant information that was published earlier than the target redundant information and whose content belongs to the same subject and has the same attribute. In this embodiment, the text containing the target redundant information refers to the source of the target redundant information. For example, the target redundant information obtained from the target information source is "Company A belongs to the new energy vehicle industry," and the title of the text containing it is "Leading Enterprise in the New Energy Vehicle Industry." Another information source also obtains the redundant information "Company A belongs to the new energy vehicle industry," but the title of the text containing it is "The Development Trend of Company A in the New Energy Vehicle Industry." Although the information obtained from both information sources is "Company A belongs to the new energy vehicle industry," and both are redundant information, the texts of the two information sources are not the same. Therefore, in the process of determining untrusted information sources, not only redundant information should be considered, but also the similarity of the text containing the redundant information should be considered.

[0064] In this embodiment, information similarity refers to the average similarity of all redundant information published by a certain information source. It can reflect the originality of the information source. The higher the information similarity, the more the redundant information of the source relies on copying and the worse its originality. It is believed that the information source does not screen or verify external information and has low credibility.

[0065] Redundancy contribution refers to the proportion of redundant information published by a certain target information source to the total amount of redundant information from all information sources. It reflects the amount of redundant output of that information source. The higher the contribution, the more likely that the source is the main source of redundant information and the higher the proportion of invalid information.

[0066] Update lag refers to the average difference between the release time of redundant information from a target information source and the release time of corresponding redundant information from other information sources. It reflects the timeliness of the information source; a higher lag indicates slower information updates and a greater likelihood of outdated information. In this embodiment, corresponding redundant information from other information sources refers to information identical to the target redundant information from other information sources.

[0067] In this embodiment, the information similarity, redundancy contribution, and update lag of the target information source can be weighted and calculated to obtain a score for the target information source. The weights in the weighting calculation process can be evenly distributed. The higher the score, the lower the credibility of the target information source. A preset score threshold can be set, and information sources that exceed the threshold can be identified as untrustworthy information sources.

[0068] As can be seen from the above, this embodiment evaluates the credibility of information sources from multiple different dimensions by calculating information similarity, redundancy contribution, and update lag, avoiding the limitations of a single dimension and making the evaluation of untrusted information sources more accurate.

[0069] In one embodiment of this application, for each piece of candidate conflict information in the candidate conflict information set, a credibility weight is calculated based on a preset credibility parameter of its corresponding information source and a timeliness parameter of its publication time, including:

[0070] For each candidate conflict information in the candidate conflict information set, the initial credibility weight of the candidate conflict information is obtained by weighting the preset credibility parameter of the corresponding information source and the timeliness parameter of the publication time.

[0071] Obtain the information source's information similarity, redundancy contribution, and update lag;

[0072] The initial credibility weight of the candidate conflict information is updated based on the information similarity, redundancy contribution, and update lag of the information source, thus obtaining the credibility weight of the candidate conflict information.

[0073] In this embodiment, considering that the information similarity, redundancy contribution and update lag of some information sources have been calculated in the process of processing redundant information, and that there is no other relevant credibility data to support the calculation of credibility weight of conflicting information except for the preset credibility parameters, data reuse is considered in this embodiment.

[0074] In this embodiment, the process of calculating the initial credibility weight will not be described in detail, and the determination of the timeliness parameter can be referred to the aforementioned text. The process of updating the initial credibility weight of the candidate conflicting information based on the information similarity, redundancy contribution, and update lag of the information source can be implemented based on the following formula:

[0075] ,in, The final credibility weight represents the candidate conflict information. As the initial credibility weight for candidate conflict information, The weighting coefficient for the i-th indicator can be either evenly distributed or set manually. This refers to the negative indicators of information sources after standardization (including information similarity, redundancy contribution, and update lag), for example... Due to the similarity of information, it can be used directly. This is a redundant contribution, which can be used directly. The standardized update lag is denoted as . , , Indicates the update lag. The maximum lag threshold can be preset, for example, it can be set to six months.

[0076] In this embodiment, by The negative impacts of information similarity, redundancy contribution, and standardization update lag are integrated into a single comprehensive value. The initial credibility weight is adjusted by discounting the correction coefficient. The greater the overall impact, the less reliable the information source. The smaller the correction coefficient, the lower the final credibility weight.

[0077] In this embodiment, since only the information similarity, redundancy contribution, and update lag of some information sources are calculated in the first calculation, if the information similarity, redundancy contribution, and update lag of the information source cannot be obtained in this embodiment, the initial credibility weight of the candidate conflict information can be used as the credibility weight of the candidate conflict information.

[0078] As can be seen from the above, the embodiments of this application take into account that the information similarity, redundancy contribution, and update lag of some information sources have already been calculated during the processing of redundant information. This technical solution reuses these data when calculating the credibility weight of conflicting information. The above data reuse method avoids repeated calculations, saves computing resources and time costs. At the same time, considering the preset credibility parameters of the information sources and the timeliness parameters of the release time, as well as the information similarity, redundancy contribution, and update lag of the information sources, the credibility of candidate conflicting information can be evaluated more comprehensively and accurately, avoiding the evaluation bias caused by relying on only a single parameter, thereby effectively improving the quality of information screening and ensuring that the screened information is more reliable and accurate.

[0079] In one embodiment of this application, the information filtering result is determined based on the credibility weight of the candidate conflict information corresponding to each conflict group, including:

[0080] Perform a judgment operation for each conflict group to obtain the judgment result for each conflict group;

[0081] By integrating the judgment results from various conflict groups, information filtering results are obtained;

[0082] Specifically, for each conflict group, a judgment operation is performed, including:

[0083] If the credibility weights of the candidate conflict information corresponding to the conflict group are all higher than the preset minimum weight threshold, and the difference between the highest credibility weight and the second highest credibility weight among the candidate conflict information corresponding to the conflict group is greater than the preset weight threshold, the judgment result of the conflict group is: retain the candidate conflict information corresponding to the highest credibility weight.

[0084] If the difference between the highest and second-highest trusted weights among the trusted weights of the candidate conflict information corresponding to the conflict group is not greater than a preset weight threshold, and / or the trusted weights of all candidate conflict information are not higher than a preset minimum weight threshold, the judgment result of the conflict group is: all candidate conflict information in the conflict group is marked as information to be manually verified.

[0085] In this embodiment, the preset minimum weight threshold refers to a pre-set baseline score for information reliability. If the reliability weight is greater than this value, the information is basically reliable; if it is less than or equal to this value, the information is not reliable enough. The preset weight threshold is a pre-set standard for the difference between the highest and second-highest reliability weights. If the difference is greater than this value, it indicates a significant difference in reliability between the two pieces of information; if it is less than or equal to this value, it indicates a small difference, making it impossible to determine which is more accurate.

[0086] In this embodiment, if the credibility weights of all candidate conflict information corresponding to the conflict group are higher than the preset minimum weight threshold, and the difference between the highest and second-highest credibility weights among the candidate conflict information corresponding to the conflict group is greater than the preset weight threshold, it indicates that all information is basically reliable, and the reliability difference between the two pieces of information is significant. In this case, the most reliable information can be directly retained. Conversely, if the difference between the highest and second-highest credibility weights of the conflict group is not greater than the preset weight threshold and / or the credibility weights of all candidate conflict information in the conflict group are not higher than the preset minimum weight threshold, it indicates that the reliability difference is small and the reliability of all information is low. It is impossible to determine which conflict information to accept. To avoid judgment errors, uncertain information can be marked for manual review.

[0087] In this embodiment, when the credible weight of some candidate conflict information in a conflict group is greater than the preset minimum weight threshold, and the other part is less than or equal to the preset minimum weight threshold, the information less than or equal to the preset minimum weight threshold can be removed first, and then the judgment operation can be performed on the remaining conflict information.

[0088] As can be seen from the above, in this embodiment, when the credibility weights of the candidate conflict information corresponding to the conflict group are all higher than the preset minimum weight threshold, and the difference between the highest credibility weight and the second highest credibility weight is greater than the preset weight threshold, the candidate conflict information corresponding to the highest credibility weight is directly retained, which can filter out the most reliable information among the conflict information. At the same time, if the difference between the highest credibility weight and the second highest credibility weight is not greater than the preset weight threshold, and / or the credibility weights of all candidate conflict information are not higher than the preset minimum weight threshold, each candidate conflict information in the conflict group is marked as information to be manually verified, which avoids blindly selecting information when the information reliability difference is small or the overall reliability is low, and prevents the risk caused by inaccurate information.

[0089] In one embodiment of this application, the information filtering method further includes:

[0090] In response to information query results containing redundant information but not conflicting information, retrieve the publication time and information source corresponding to all redundant information.

[0091] The redundant information in the information query results is processed based on the release time and information source of all redundant information to obtain the information filtering results.

[0092] In this embodiment, if the information query results contain only redundant information and no conflicting information, the information can be filtered based on the publication time of the redundant information and the corresponding information source. For example, the redundant information with the earliest publication time can be retained, or the redundant information with the highest preset credibility from each information source can be retained, or a comprehensive judgment can be made on both, such as determining which redundant information to retain through weighted calculation, and finally obtaining the information filtering result.

[0093] In response to a query result containing no redundant information but containing conflicting information, retrieve the information source corresponding to all conflicting information.

[0094] The conflicting information in the information query results is processed based on the information sources corresponding to all conflicting information to obtain the information filtering results.

[0095] In this embodiment, if the information query results all contain conflicting information but no redundant information, it is impossible to judge the conflicting information based on the results of the redundant information processing. Therefore, the retention or rejection of conflicting information can be determined based on the preset credibility parameter and timeliness parameter corresponding to the information source of the conflicting information. In this embodiment, the process of determining the timeliness parameter of conflicting information is the same as the process of determining the timeliness parameter of conflicting information when the information query results contain both redundant and conflicting information, and will not be described again.

[0096] Corresponding to the information filtering method in the above embodiments, Figure 3 This is a structural block diagram of an information filtering device provided according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. Referring to the figure, the information filtering device 20 includes: a search term determination module 21, an information query module 22, and an information filtering module 23.

[0097] The term determination module 21 is used to determine at least one query term based on the term; the query term includes at least the term.

[0098] Information query module 22 is used to perform information queries based on query terms and obtain multiple information query results;

[0099] Information filtering module 23 is used to perform information filtering operations and obtain information filtering results when multiple information query results contain redundant and conflicting information.

[0100] The information filtering operation includes:

[0101] Retrieve the publication time and information source corresponding to all information query results;

[0102] At least based on the release time of each redundant piece of information, untrusted information sources can be identified from various information sources;

[0103] The redundant information in multiple information query results is processed based on untrusted information sources to obtain the first filtering result;

[0104] The conflicting information in the first screening result is processed based on the release time and untrusted information sources corresponding to each conflicting information to obtain the information screening result.

[0105] In one embodiment of this application, the information filtering module 23 is specifically used to, for each piece of redundant information, use the redundant information as the target redundant information, filter out other redundant information whose publication time is earlier than the publication time of the target redundant information, calculate the similarity between the text where the target redundant information is located and the text where other redundant information is located, and obtain the similarity corresponding to the target redundant information; and take the average of the similarity corresponding to each target redundant information as the information similarity of the target information source corresponding to the target redundant information.

[0106] Calculate the proportion of redundant information published by the target information source in all redundant information to obtain the redundancy contribution of the target information source;

[0107] The average difference between the release time of redundant information in the target information source and the release time of corresponding redundant information in other information sources is calculated to obtain the update lag of the target information source.

[0108] The score of the target information source is determined based on information similarity, redundancy contribution, and update lag.

[0109] Untrusted information sources are identified from among the various information sources based on the scores of each target information source.

[0110] In one embodiment of this application, conflicting information refers to inconsistent information provided by different information sources for the same attribute dimension of the same information subject;

[0111] The information filtering module 23 is further used to classify the conflicting information in the first filtering result according to the information subject corresponding to the conflicting information, and obtain at least one conflict group; each conflict group contains at least two different conflicting information against the same information subject;

[0112] For each conflict group, from all the conflict information contained in the conflict group, the conflict information corresponding to the untrusted information source is removed to obtain the candidate conflict information set; for each candidate conflict information in the candidate conflict information set, the trust weight of the candidate conflict information is calculated based on the preset trust parameter of its corresponding information source and the timeliness parameter of the release time.

[0113] The information filtering results are determined based on the credibility weights of the candidate conflict information corresponding to each conflict group.

[0114] In one embodiment of this application, the information filtering module 23 is further configured to perform a judgment operation for each conflict group to obtain the judgment result of each conflict group;

[0115] By integrating the judgment results from various conflict groups, information filtering results are obtained;

[0116] Specifically, for each conflict group, a judgment operation is performed, including:

[0117] If the credibility weights of the candidate conflict information corresponding to the conflict group are all higher than the preset minimum weight threshold, and the difference between the highest credibility weight and the second highest credibility weight among the candidate conflict information corresponding to the conflict group is greater than the preset weight threshold, the judgment result of the conflict group is: retain the candidate conflict information corresponding to the highest credibility weight.

[0118] If the difference between the highest and second-highest trusted weights among the trusted weights of the candidate conflict information corresponding to the conflict group is not greater than a preset weight threshold, and / or the trusted weights of all candidate conflict information are not higher than a preset minimum weight threshold, the judgment result of the conflict group is: all candidate conflict information in the conflict group is marked as information to be manually verified.

[0119] In one embodiment of this application, the information filtering module 23 is further used to perform weighted calculations on each candidate conflict information in the candidate conflict information set, based on the preset credibility parameter of the corresponding information source and the timeliness parameter of the publication time, to obtain the initial credibility weight of the candidate conflict information.

[0120] Obtain the information source's information similarity, redundancy contribution, and update lag;

[0121] The initial credibility weight of the candidate conflict information is updated based on the information similarity, redundancy contribution, and update lag of the information source, thus obtaining the credibility weight of the candidate conflict information.

[0122] In one embodiment of this application, the information filtering module 23 is further configured to, in response to the information query results containing redundant information but not conflicting information, obtain the publication time and corresponding information source of all redundant information;

[0123] The redundant information in the information query results is processed based on the release time and information source of all redundant information to obtain the information filtering results.

[0124] In one embodiment of this application, the information filtering module 23 is further configured to, in response to the information query result containing no redundant information but containing conflicting information, obtain the information source corresponding to all conflicting information.

[0125] The conflicting information in the information query results is processed based on the information sources corresponding to all conflicting information to obtain the information filtering results.

[0126] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 4 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of the search term determination module 21, information query module 22, and information filtering module 23 are shown.

[0127] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0128] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0129] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0130] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the information filtering method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0131] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0132] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0133] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules, units, or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or modules / units, or it may be an electrical, mechanical, or other form of connection.

[0136] The modules / units described as separate components may or may not be physically separate. Similarly, the components shown as modules / units may or may not be physical modules / units; they may be located in one place or distributed across multiple network modules / units. Some or all of the modules / units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0137] Furthermore, the functional modules / units in the various embodiments of this application can be integrated into one processing module / unit, or each module / unit can exist physically separately, or two or more modules / units can be integrated into one module / unit. The integrated modules / units described above can be implemented in hardware or in the form of software functional modules / units.

[0138] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An information filtering method, characterized in that, include: At least one query term is determined based on the search terms; The query terms include at least the search terms; Based on the query terms, information is searched to obtain multiple information search results; In response to the presence of redundant and conflicting information in the multiple information query results, an information filtering operation is performed to obtain the information filtering results; The conflicting information refers to inconsistent information provided by different information sources for the same attribute dimension of the same information subject; The information filtering operation includes: Retrieve the publication time and information source corresponding to all information query results; At least based on the release time of each redundant piece of information, untrusted information sources can be identified from various information sources; Based on the untrusted information source, redundant information in the multiple information query results is processed to obtain the first filtering result; Based on the release time of each conflicting piece of information and the untrusted information source, the conflicting information in the first filtering result is processed to obtain the information filtering result; The process of identifying untrusted information sources from various information sources based at least on the publication time of each redundant piece of information includes: For each piece of redundant information, this piece of redundant information is used as the target redundant information. Other redundant information whose publication time is earlier than that of the target redundant information is filtered out. The similarity between the text containing the target redundant information and the text containing the other redundant information is calculated to obtain the similarity corresponding to the target redundant information. The average of the similarity corresponding to each target redundant information is taken as the information similarity of the target information source corresponding to the target redundant information. Calculate the proportion of redundant information published by the target information source in all redundant information to obtain the redundancy contribution of the target information source; The average difference between the release time of redundant information in the target information source and the release time of corresponding redundant information in other information sources is calculated to obtain the update lag of the target information source. The score of the target information source is determined based on the information similarity, the redundancy contribution, and the update lag. The untrusted information sources are determined from each information source based on the scores of each target information source; The step of processing the conflict information in the first filtering result based on the release time corresponding to each conflict information and the untrusted information source to obtain the information filtering result includes: The conflicting information in the first screening result is classified according to the information subject corresponding to the conflicting information to obtain at least one conflict group; each conflict group contains at least two different conflicting information against the same information subject. For each conflict group, the conflict information corresponding to the untrusted information source is removed from all the conflict information contained in the conflict group to obtain a candidate conflict information set; for each candidate conflict information in the candidate conflict information set, the initial trust weight of the candidate conflict information is calculated based on the preset trust parameter of its corresponding information source and the timeliness parameter of the release time. The initial credibility weight of the candidate conflict information is updated based on the information similarity, redundancy contribution and update lag of the information source to obtain the credibility weight of the candidate conflict information. The information filtering results are determined based on the credibility weights of the candidate conflict information corresponding to each conflict group.

2. The information filtering method as described in claim 1, characterized in that, The determination of the information filtering result based on the credibility weight of the candidate conflict information corresponding to each conflict group includes: Perform a judgment operation for each conflict group to obtain the judgment result for each conflict group; The information filtering results are obtained by integrating the judgment results of each conflict group; Specifically, for each conflict group, a judgment operation is performed, including: If the credibility weights of the candidate conflict information corresponding to the conflict group are all higher than the preset minimum weight threshold, and the difference between the highest credibility weight and the second highest credibility weight among the candidate conflict information corresponding to the conflict group is greater than the preset weight threshold, the judgment result of the conflict group is: retain the candidate conflict information corresponding to the highest credibility weight. If the difference between the highest and second-highest trusted weights among the trusted weights of the candidate conflict information corresponding to the conflict group is not greater than a preset weight threshold, and / or the trusted weights of all candidate conflict information are not higher than a preset minimum weight threshold, the judgment result of the conflict group is: all candidate conflict information in the conflict group is marked as information to be manually verified.

3. The information filtering method as described in claim 1, characterized in that, For each candidate conflict message in the candidate conflict information set, based on the preset credibility parameter of its corresponding information source and the timeliness parameter of its publication time, the initial credibility weight of the candidate conflict message is calculated, including: For each candidate conflict information in the candidate conflict information set, the initial credibility weight of the candidate conflict information is obtained by weighting the preset credibility parameter of the corresponding information source and the timeliness parameter of the publication time.

4. The information filtering method as described in claim 1, characterized in that, Also includes: In response to the fact that the information query results contain redundant information but no conflicting information, the release time and information source corresponding to all redundant information are obtained. The redundant information in the information query results is processed based on the release time and information source of all redundant information to obtain the information filtering results.

5. The information filtering method as described in claim 1, characterized in that, Also includes: In response to the fact that the information query results do not contain redundant information but contain conflicting information, the information sources corresponding to all conflicting information are obtained. The conflicting information in the information query results is processed based on the information sources corresponding to all conflicting information to obtain the information filtering results.

6. An information filtering device, characterized in that, include: The term determination module is used to determine at least one query term based on the search terms. The query terms include at least the search terms; The information query module is used to perform information queries based on the query terms and obtain multiple information query results; The information filtering module is used to perform an information filtering operation and obtain information filtering results in response to the presence of redundant and conflicting information in the multiple information query results; The conflicting information refers to inconsistent information provided by different information sources for the same attribute dimension of the same information subject; The information filtering operation includes: Retrieve the publication time and information source corresponding to all information query results; At least based on the release time of each redundant piece of information, untrusted information sources can be identified from various information sources; Based on the untrusted information source, redundant information in the multiple information query results is processed to obtain the first filtering result; Based on the release time of each conflicting piece of information and the untrusted information source, the conflicting information in the first filtering result is processed to obtain the information filtering result; The process of identifying untrusted information sources from various information sources based at least on the publication time of each redundant piece of information includes: For each piece of redundant information, this piece of redundant information is used as the target redundant information. Other redundant information whose publication time is earlier than that of the target redundant information is filtered out. The similarity between the text containing the target redundant information and the text containing the other redundant information is calculated to obtain the similarity corresponding to the target redundant information. The average of the similarity corresponding to each target redundant information is taken as the information similarity of the target information source corresponding to the target redundant information. Calculate the proportion of redundant information published by the target information source in all redundant information to obtain the redundancy contribution of the target information source; The average difference between the release time of redundant information in the target information source and the release time of corresponding redundant information in other information sources is calculated to obtain the update lag of the target information source. The score of the target information source is determined based on the information similarity, the redundancy contribution, and the update lag. The untrusted information sources are determined from each information source based on the scores of each target information source; The step of processing the conflict information in the first filtering result based on the release time corresponding to each conflict information and the untrusted information source to obtain the information filtering result includes: The conflicting information in the first screening result is classified according to the information subject corresponding to the conflicting information to obtain at least one conflict group; each conflict group contains at least two different conflicting information against the same information subject. For each conflict group, the conflict information corresponding to the untrusted information source is removed from all the conflict information contained in the conflict group to obtain a candidate conflict information set; for each candidate conflict information in the candidate conflict information set, the initial trust weight of the candidate conflict information is calculated based on the preset trust parameter of its corresponding information source and the timeliness parameter of the release time. The initial credibility weight of the candidate conflict information is updated based on the information similarity, redundancy contribution and update lag of the information source to obtain the credibility weight of the candidate conflict information. The information filtering results are determined based on the credibility weights of the candidate conflict information corresponding to each conflict group.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images