Processing method of report information and related device

Abstract

The invention discloses a report information processing method and a related device, and relates to the field of text data processing, and the method comprises the steps: judging whether second historical report information consistent with a reported person in current report information exists in first historical report information or not after the current report information is received; when the second historical report information exists, processing the current report information and the second historical report information to obtain a current report information vector and a second historical report information vector, and then calculating a text similarity set of the current report information vector and the second historical report information vector; and finally, inputting the highest text similarity in the text similarity set into the decision tree, and performing repetition determination on the current report information based on a preset determination rule to obtain a repetition determination result. According to the method, the historical report information is screened, the text similarity is calculated, and the highest text similarity is input into the decision tree for repeated judgment, so that the judgment coverage rate and the success rate are improved while the judgment range is narrowed step by step.

Description

Technical Field

[0001] The present application relates to the technical field of text data processing, and in particular to a method for processing report information and related devices. Background Art

[0002] Nowadays, all industries have reporting acceptance systems. The reporting acceptance system can accept the reporting information fed back by users and then process it.

[0003] However, some whistleblowers, due to dissatisfaction with the historical reporting results, in order to strengthen the reporting efforts, or simply to vent their emotions, will submit multiple duplicate or similar reporting information for the same whistleblower. These duplicate reporting information will enter the reporting acceptance system for acceptance and processing. If these duplicate reporting information are accepted without distinction, it will cause a waste of acceptance resources. Therefore, it is necessary to judge the duplication of reporting information and label the duplicate reporting information as a duplicate report for the reporting acceptance system to refer to when accepting it, so as to improve the operational efficiency of the reporting acceptance system.

[0004] The known methods of checking duplicate reporting information are that only when the reporting information is about the same person and the content of the report is exactly the same, will it be considered as a duplicate report. The coverage and success rate of this method of checking duplicate reporting information are very low, and only a small part of the reporting information can be labeled as duplicate reports. In addition, the reporter can easily bypass this known technology of checking duplicate reporting mechanism by simply modifying the reporting information and get it accepted. Summary of the invention

[0005] In view of the above problems, this application provides a method and related device for processing reported information to achieve the purpose of improving the coverage rate and success rate of duplicate detection. The specific scheme is as follows:

[0006] The first aspect of the present application provides a method for processing report information, comprising:

[0007] Receive current reporting information;

[0008] Determine whether there is second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information;

[0009] When the second historical reporting information exists in the first historical reporting information, the current reporting information and the second historical reporting information are processed to obtain a current reporting information vector and a second historical reporting information vector; a text similarity set between the current reporting information vector and the second historical reporting information vector is calculated; and the text similarity set includes at least one text similarity;

[0010] The highest text similarity in the text similarity set is input into the decision tree, and the current report information is repeatedly judged based on the preset judgment rules to obtain the repeated judgment result.

[0011] In a possible implementation, determining whether there is second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information includes:

[0012] Determine whether the person being reported in the current reporting information exists in the set of historical persons being reported in the first historical reporting information.

[0013] In a possible implementation, the method further includes:

[0014] When the person being reported in the current reporting information does not exist in the set of historical reporters in the first historical reporting information, the current reporting information is determined to be a non-duplicate report, and the current reporting information is sent to the reporting acceptance module.

[0015] In a possible implementation, the highest text similarity in the text similarity set is input into a decision tree, and repeated determination is performed based on a preset determination rule, including:

[0016] Determining whether the highest text similarity is greater than a first preset threshold;

[0017] When the highest text similarity is greater than a first preset threshold, the current report information is determined to be a duplicate report;

[0018] When the highest text similarity is less than the first threshold value, the proportion of repeated reports in the second historical report information is calculated;

[0019] The comprehensive score of the historical repetition rate of the person being reported is calculated using the percentage and the highest text similarity;

[0020] Determine whether the comprehensive score is greater than a second preset threshold;

[0021] When the comprehensive score is greater than the second preset threshold, the current report information is determined to be a repeated report;

[0022] When the comprehensive score is less than the second preset threshold, the current report information is determined to be a non-duplicate report.

[0023] In a possible implementation, the method further includes:

[0024] When the comprehensive score is less than the second preset threshold and is within the preset confidence interval, the comprehensive score is used as the duplicate report confidence value, and a non-duplicate report confidence value is generated based on the comprehensive score;

[0025] The duplicate report trust value and the non-duplicate report trust value are sent to the acceptance module as acceptance reference values.

[0026] In a possible implementation, the method further includes:

[0027] Regularly obtain the actual duplication determination results within a preset time period from the acceptance module;

[0028] The first preset threshold and the second preset threshold are updated according to the pre-repeat determination result corresponding to the actual repeat determination result and the actual repeat determination result.

[0029] In a possible implementation, updating the first preset threshold and the second preset threshold according to the pre-repeat determination result corresponding to the actual repeat determination result and the actual repeat determination result includes:

[0030] Determine the false positive rate and the missed positive rate based on the pre-repeated determination result and the actual repeated determination result;

[0031] When the false positive rate is greater than the false negative rate, the first preset threshold and the second preset threshold are increased according to the preset adjustment rule;

[0032] When the false positive rate is less than the false negative rate, the first preset threshold and the second preset threshold are adjusted down according to the preset adjustment rule;

[0033] When the false positive rate is equal to the false negative rate, the first preset threshold and the second preset threshold are not adjusted.

[0034] A second aspect of the present application provides a reporting information processing device, including:

[0035] Receiving unit, judging unit, calculating unit and determining unit; wherein:

[0036] A receiving unit, used for receiving current reporting information;

[0037] A judgment unit, used to judge whether there is a second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information;

[0038] A calculation unit is used to process the current report information and the second historical report information when the first historical report information contains the second historical report information, to obtain the current report information vector and the second historical report information vector; calculate and obtain a text similarity set between the current report information vector and the second historical report information vector; the text similarity set includes at least one text similarity;

[0039] The determination unit is used to input the highest text similarity in the text similarity set into the decision tree, and perform repeated determination on the current reporting information based on the preset determination rules to obtain repeated determination results.

[0040] The third aspect of the present application provides a reporting information processing device, comprising at least one processor and a memory connected to the processor, wherein:

[0041] The memory is used to store computer programs;

[0042] The processor is used to execute a computer program so that the report information processing device can implement the report information processing method as described in any one of the above.

[0043] The fourth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement a method for processing reporting information as described in any one of the above.

[0044] By means of the above technical scheme, the method for processing report information and the related device provided by the present application, after receiving the current report information, determine whether there is a second historical report information in the first historical report information that is consistent with the person being reported in the current report information; when the second historical report information exists, the current report information and the second historical report information are processed to obtain the current report information vector and the second historical report information vector, and then, the text similarity set of the current report information vector and the second historical report information vector is calculated. Finally, the highest text similarity in the text similarity set is input into the decision tree, and the current report information is repeatedly judged based on the preset judgment rules to obtain the repeated judgment result. In the present application, by screening the historical report information, calculating the text similarity, and inputting the highest text similarity into the decision tree for repeated judgment, the judgment range is gradually narrowed while the coverage rate and success rate of repeated judgment are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0046] Figure 1 A flowchart of a method for processing reported information provided for this application;

[0047] Figure 2 An example diagram of a Gaussian distribution with preset confidence intervals provided for this application;

[0048] Figure 3 A determination flow chart of the method for handling the reported information provided for this application;

[0049] Figure 4 An example diagram of the decision process of the decision tree module provided for this application;

[0050] Figure 5 A schematic diagram of a structure of a device for processing report information provided in this application;

[0051] Figure 6 A schematic diagram of the structure of the reporting information processing device provided for this application. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] Nowadays, when accepting reports, the reporting acceptance systems of various industries often process the report information repeatedly, which leads to a waste of acceptance resources. In order to solve the problem of waste of acceptance resources, the known technology first uses duplicate judgment to process the report information and then inputs it into the report acceptance system. The report acceptance system will process the report information based on the result of the duplicate judgment.

[0056] The duplicate detection processing method in the known technology has the following problems:

[0057] The coverage rate of duplicate reporting is low: only reporting information with exactly the same person being reported and the same reporting content will be considered as duplicate reporting.

[0058] The success rate of duplicate detection is low: the reporter can easily bypass the duplicate detection mechanism by modifying the reporting information and submit similar reporting information.

[0059] In order to solve the above problems, the present application provides a method and related device for processing reporting information.

[0060] Optional, see Figure 1 , a flowchart of a method for processing reported information provided in this application, such as Figure 1 As shown, the method for handling reported information includes the following steps:

[0061] Step 101, receiving current reporting information.

[0062] It should be noted that the current reporting information is the reporting information submitted by the user;

[0063] Optionally, receive report information submitted by the current user.

[0064] Step 102: Determine whether there is second historical reporting information in the first historical reporting information that is consistent with the person being reported in the current reporting information.

[0065] It should be noted that the first historical reporting information is all historical reporting information, and the second historical reporting information is the historical reporting information of the same person being reported in the current reporting information.

[0066] This step is mainly to determine whether the second historical reporting information that is consistent with the person being reported in the current reporting information has been processed before. If the second historical reporting information that is consistent with the person being reported in the current reporting information does not exist in the first historical reporting information, the current reporting information will be directly judged as a non-duplicate report and the current reporting information will be directly input into the reporting acceptance module; if the second historical reporting information that is consistent with the person being reported in the current reporting information exists in the first timely reporting information, it means that the reporting acceptance system has previously received reporting information from the same person being reported, and there may be historical reporting information in the second historical reporting information that is very similar to the current reporting information, that is, there is a possibility that the current reporting information is a duplicate report.

[0067] Optionally, determine whether the person being reported in the current reporting information exists in the set of historical reporters of the first historical reporting information.

[0068] Specifically, assume that the current report information is , the person being reported is , the historical reporting information set of the first historical reporting information is , the historical set of reported persons of the first historical reporting information is .

[0069] Determine the historical reported person set of the first historical reporting information as Is there a person who is being reported in the current report information? If not, the current report information is directly determined as a non-duplicate report and sent to the report acceptance module. If it exists, it means that the current report information may be a duplicate report. See step 103 for subsequent processing steps.

[0070] Step 103: When the second historical reporting information exists in the first historical reporting information, the current reporting information and the second historical reporting information are processed to obtain a current reporting information vector and a second historical reporting information vector; a text similarity set between the current reporting information vector and the second historical reporting information vector is calculated; and the text similarity set includes at least one text similarity.

[0071] Optionally, when there is second historical reporting information in the first historical reporting information, the current reporting information and the second historical reporting information are processed to obtain a current reporting information vector and a second historical reporting information vector, and then a text similarity set is calculated based on the current reporting information vector and the second historical reporting information vector. There is at least one second historical reporting information, so the text similarity set includes at least one text similarity.

[0072] In the process of calculating the text similarity between the reporting information and the second historical reporting information, the cosine value of the reporting vector is used as a measure of the similarity.

[0073] Specifically, when comparing the similarity between two reports, the similarity needs to be converted into a quantitative value before it can be measured. A string-based measurement method can be used, that is, after the text is segmented, the word frequency vector of the text is obtained, and then the cosine value of the two text word frequency vectors is calculated to quantify the text similarity. The closer the cosine value is to 1, the higher the similarity between the two texts.

[0074] For example, sentence 1: This cup is big, that cup is good.

[0075] Sentence 2: This water cup is not small, I want that water cup.

[0076] First, segment the above two sentences.

[0077] Sentence 1: This / water cup / is big, that / water cup / is good.

[0078] Sentence 2: This / cup / is / not / small, / I / want / that / cup / .

[0079] Then, list all the words to form a word set:

[0080] {This cup is too big. That one is better. No, it should be smaller.}

[0081] After that, calculate the frequency of each word in the sentence according to the word set order:

[0082] Sentence 1: This 1, water cup 2, big 1, that 1, good 1, not 0, small 0, need 0

[0083] Sentence 2: This 1, water cup 2, big 0, that 1, good 0, no 1, small 1, want 1

[0084] Get the word frequency vector:

[0085] Sentence 1: [1, 2, 1, 1, 1, 0, 0, 0]

[0086] Sentence 2: [1, 2, 0, 1, 0, 1, 1, 1]

[0087] Finally, calculate the cosine value of the above two word frequency vectors.

[0088] Assume that the two word frequency vectors are n-dimensional vectors [A 1 ,A 2 ,...,A n ] and [B 1 ,B 2 ,...,B n ] , the general formula for calculating the cosine value can be:

[0089]

[0090] After obtaining the cosine values ​​of sentence 1 and sentence 2 through the above formula, 3 decimal places can be retained, and the calculated cosine value is 0.707, that is, the similarity between sentence 1 and sentence 2 is 0.707.

[0091] Similarly, the similarity between the current reporting information and each piece of historical reporting information in the second historical reporting information is calculated based on the above similarity calculation method.

[0092] Suppose the set of the second historical reporting information is , select one historical report information and the current report information in the second historical report information set in turn Perform word segmentation to obtain word set .

[0093] Then, we get the word frequency vector of the word segmentation compared with the n historical reporting information: [A k,i ],[B k,i ],i = 1 , 2 ,...,n .

[0094] Finally, according to the general formula for calculating the cosine value above, Cosine value of text similarity with m historical reporting information .

[0095] Step 104: input the highest text similarity in the text similarity set into the decision tree, perform a duplicate determination on the current report information based on a preset determination rule, and obtain a duplicate determination result.

[0096] The highest text similarity is the text similarity with the largest median value of each text similarity calculated between the current reporting information and each historical reporting information in the second historical reporting information set.

[0097] The judgment process based on the preset judgment rules is as follows:

[0098] Determine whether the highest text similarity is greater than a first preset threshold value. When the highest text similarity is greater than the first preset threshold value, determine that the current report information is a duplicate report. When the highest text similarity is less than the first preset threshold value, calculate the proportion of duplicate reports in the second historical report information. Then, use the calculated proportion and the highest text similarity to calculate a comprehensive score of the historical repetition rate of the person being reported in the current report information.

[0099] Finally, determine whether the comprehensive score is greater than the second preset threshold. When the comprehensive score is greater than the second preset threshold, determine that the current report information is a duplicate report; when the comprehensive score is less than the second preset threshold, determine that the current report information is a non-duplicate report.

[0100] For example, let the highest text similarity be , the highest text similarity is input into the decision tree for repeated judgment. The judgment process is as follows:

[0101] First, the highest text similarity Similarity threshold with text For comparison, if , the current report information is judged as a repeated report; otherwise, the next step of judgment and processing is carried out;

[0102] Statistically calculate the proportion of repeated reports in the second historical reporting information corresponding to the person being reported in the current information , and based on this percentage, calculate the comprehensive score of the historical repetition rate of the person being reported:

[0103]

[0104] Finally, the comprehensive score of the historical repetition rate is compared with the comprehensive score threshold For comparison, if , then the current report information is determined as a duplicate report, otherwise the current report information is determined as a non-duplicate report.

[0105] Optionally, the highest text similarity is input into a decision tree to perform duplicate determination according to the above preset determination rules to obtain a result of duplicate determination.

[0106] It should be noted that when the comprehensive score is less than the second preset threshold, a further judgment process is included:

[0107] When the comprehensive score is less than the second preset threshold, it is determined whether the comprehensive score is within a preset confidence interval, where the preset confidence interval is [T2-σ, T2], where is the second preset threshold, which can also be called the comprehensive scoring threshold, and σ is the standard deviation of the deviation introduced in the word segmentation and text similarity calculation process, which can be obtained through sample statistics. According to the characteristics of Gaussian distribution, data in the interval [T2-σ, T2] has a high probability of being less than the threshold due to statistical deviation. Therefore, it is necessary to give a duplicate report confidence value and a non-duplicate report confidence value to the report information within this interval, and send these two values ​​to the report acceptance module as a reference value when the report acceptance module processes the current report information whose comprehensive score is within the preset confidence interval. Specifically, let the comprehensive score be , the repeated reporting trust value can be , correspondingly, the non-duplicate report trust value is .

[0108] For example, see Figure 2 , an example diagram of Gaussian distribution with preset confidence intervals provided in this application.

[0109] like Figure 2 As shown, σ is the standard deviation and the preset confidence interval is [T2-σ, T2].

[0110] In summary, when using the pre-set threshold in the decision tree for judgment, this application introduces a confidence interval, and calculates the trust values ​​of repeated reports and non-repeated reports for the data in important intervals, which can improve the situation where the boundary value near the threshold is judged incorrectly when the absolute threshold is judged.

[0111] Furthermore, the first preset threshold and the second preset threshold can be updated; the updating process is as follows:

[0112] Regularly obtain actual duplicate determination results within a preset time period from the reporting acceptance system, and update the first preset threshold and the second preset threshold based on the actual duplicate determination results and the preliminary duplicate determination results of duplicate determination according to steps 101 to 104 provided in the present application corresponding to the actual duplicate determination results.

[0113] First, the false positive rate and the false negative rate are determined based on the pre-repeat determination result and the actual repeat determination result.

[0114] Among them, judging a non-duplicate report as a duplicate report is called a misjudgment, and the misjudgment rate is . Determining repeated reports as non-duplicate reports is called missed judgment, and the missed judgment rate is . Lower the first preset threshold and a second preset threshold , can improve the missed judgment situation of the entire pre-repeated judgment process from step 101 to step 104, that is, reduce the missed judgment rate However, it will also increase the number of misjudgments, that is, the misjudgment rate is It can be seen that these two kinds of misjudgment cannot be reduced at the same time. The two are in a trade-off relationship. In addition, it should be noted that when the misjudgment rate and missed detection rate The closer they are, the smaller the error rate of the entire pre-repetition determination process.

[0115] Specifically, the actual duplication judgment results of the report acceptance module in the acceptance stage are compared with the pre-duplicate judgment results through the decision tree, which are divided into three situations:

[0116] The actual duplication judgment result of the report acceptance module in the acceptance link is consistent with the pre-duplicate judgment result. The judgment is correct, and the number of correct judgments is updated in real time. Suppose the number of correct judgments is ;

[0117] When the actual duplication judgment result of the acceptance link is a non-duplicate report, and the pre-duplicate judgment result of the decision tree is a duplicate report, it is a misjudgment. The number of misjudgments is updated in real time. Set the number of misjudgments to ;

[0118] When the actual duplication judgment result of the acceptance link is a duplicate report, and the pre-duplicate judgment result of the decision tree is a non-duplicate report, it is a missed judgment. The number of missed judgments is updated in real time. Set the number of missed judgments to .

[0119] False positive rate The calculation formula is:

[0120] ;

[0121] Missed Detection Rate The calculation formula is:

[0122] ;

[0123] When the false positive rate is greater than the false negative rate, the first preset threshold and the second preset threshold are increased according to the preset adjustment rule, that is, when > When and ;

[0124] When the false positive rate is less than the false negative rate, the first preset threshold and the second preset threshold are adjusted down according to the preset adjustment rule, that is, when < When and ;

[0125] When the false positive rate is equal to the false negative rate, the first preset threshold and the second preset threshold are not adjusted, that is, when = hour, and No adjustments are made.

[0126] It should be noted that the first preset threshold and the second preset threshold are adjusted according to a fixed period, and the preset adjustment rules include the threshold adjustment range determined through machine learning training, specifically including the threshold adjustment range that makes the adjusted false positive rate and missed positive rate approach each other, rather than the result of a reverse jump that makes the adjusted false positive rate and missed positive rate have an opposite relationship in size.

[0127] It should be noted that the first preset threshold and a second preset threshold The initial value of can be set by expert evaluation.

[0128] In summary, the method for processing report information provided in the present application determines whether there is a second historical report information that is consistent with the person being reported in the current report information in the first historical report information after receiving the current report information; when the second historical report information exists, the current report information and the second historical report information are processed to obtain the current report information vector and the second historical report information vector, and then the text similarity set of the current report information vector and the second historical report information vector is calculated. Finally, the highest text similarity in the text similarity set is input into the decision tree, and the current report information is repeatedly judged based on the preset judgment rules to obtain the repeated judgment result. In the present application, by screening the historical report information, calculating the text similarity, and inputting the highest text similarity into the decision tree for repeated judgment, the judgment range is gradually narrowed while the coverage rate and success rate of repeated judgment are improved.

[0129] Optional, see Figure 3 , a determination flow chart of the method for processing reporting information provided in this application.

[0130] like Figure 3 As shown, the method for processing the report information can include a report submission module, a duplicate preprocessing module, and a text similarity

[0131] The calculation module, decision tree judgment module, historical judgment feedback module and report acceptance module have been completed.

[0132] Among them, the report submission module is used to receive the report information submitted by the user;

[0133] The duplicate detection preprocessing module is used to determine whether there is any historical report information about the same person reported in the current report information submitted by the user; when there is no historical report information about the same person reported in the current report information submitted by the user, the current report information is determined to be a non-duplicate report and is input into the report acceptance module; when there is any historical report information about the same person reported in the current report information submitted by the user, the current report information and the historical report information about the same person reported are segmented to obtain a report content vector;

[0134] The text similarity calculation module is used to calculate the text similarity between the current report information and the historical report information of the same person being reported based on the report content vector obtained by the duplicate judgment preprocessing module, and specifically uses the cosine value of the report vector as the measurement value of the similarity.

[0135] The decision tree judgment module is used to receive the text similarity calculated by the text similarity calculation module, as well as the historical repeated reporting rate and judgment failure feedback from the historical judgment feedback module. For details, see Figure 4 , the decision process example diagram of the decision tree module provided in this application. Figure 4 As shown, first determine the highest text similarity calculated in the text similarity module and the text similarity threshold When the highest text similarity is greater than the text similarity threshold , the current report information is judged as a repeated report; when the highest text similarity is less than the text similarity threshold When calculating the comprehensive score taking into account the historical repetition rate, the comprehensive score and the comprehensive score threshold are judged. When the comprehensive score is greater than the comprehensive score threshold When the comprehensive score is less than the comprehensive score threshold, the current report information will be judged as a repeated report. , the current report information is judged as a non-duplicate report. In addition, the decision tree module will regularly obtain the historical repeated report rate and judgment failure information from the historical judgment feedback module, and retrospectively update the text similarity threshold based on the wrong judgment in this information. and comprehensive score threshold .

[0136] The historical judgment feedback module is used to collect historical judgment information, calculate the repeated reporting ratio of the current person being reported and the correctness of the repeated judgments fed back by the reporting acceptance module, and provide it to the decision tree judgment module for use.

[0137] The report acceptance module is used to receive repeated reports from the judgment preprocessing module and repeated reports or non-duplicate reports from the decision tree judgment module, and accept and process them.

[0138] In summary, this application proposes a method for duplication determination of reporting information based on a decision tree and historical feedback data, and uses text similarity for calculation, so as to improve the coverage and success rate of duplicate reporting and reduce the probability of bypassing the duplication determination mechanism by modifying the reporting content.

[0139] A method for processing report information provided in an embodiment of the present application is introduced above, and a device for executing the above-mentioned method for processing report information will be introduced below.

[0140] See also Figure 5 , Figure 5 A schematic diagram of the structure of the reporting information processing device provided in this application. Figure 5 As shown, the device comprises:

[0141] Receiving unit 10, judging unit 20, calculating unit 30 and determining unit 40; wherein:

[0142] A receiving unit 10, used to receive current reporting information;

[0143] A judgment unit 20, used to judge whether there is a second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information;

[0144] The calculation unit 30 is used to process the current report information and the second historical report information when the first historical report information contains the second historical report information, to obtain the current report information vector and the second historical report information vector; to calculate the text similarity set between the current report information vector and the second historical report information vector; the text similarity set includes at least one text similarity;

[0145] The determination unit 40 is used to input the highest text similarity in the text similarity set into the decision tree, perform a duplication determination on the current reporting information based on a preset determination rule, and obtain a duplication determination result.

[0146] In one implementation, the determination unit 20 is specifically configured to:

[0147] Determine whether the person being reported in the current reporting information exists in the set of historical persons being reported in the first historical reporting information.

[0148] In one embodiment, the device further includes a sending unit, and the sending unit is specifically configured to:

[0149] When the person being reported in the current reporting information does not exist in the set of historical reporters in the first historical reporting information, the current reporting information is determined to be a non-duplicate report, and the current reporting information is sent to the reporting acceptance module.

[0150] In one implementation, the determination unit 40 is specifically configured to:

[0151] Determining whether the highest text similarity is greater than a first preset threshold;

[0152] When the highest text similarity is greater than a first preset threshold, the current report information is determined to be a duplicate report;

[0153] When the highest text similarity is less than the first threshold value, the proportion of repeated reports in the second historical report information is calculated;

[0154] The comprehensive score of the historical repetition rate of the person being reported is calculated using the percentage and the highest text similarity;

[0155] Determine whether the comprehensive score is greater than a second preset threshold;

[0156] When the comprehensive score is greater than the second preset threshold, the current report information is determined to be a repeated report;

[0157] When the comprehensive score is less than the second preset threshold, the current report information is determined to be a non-duplicate report.

[0158] In one embodiment, the device further includes an acceptance reference value generating unit and a high acceptance reference value generating unit, which are specifically used to:

[0159] When the comprehensive score is less than the second preset threshold and is within the preset confidence interval, the comprehensive score is used as the duplicate report confidence value, and a non-duplicate report confidence value is generated based on the comprehensive score;

[0160] The duplicate report trust value and the non-duplicate report trust value are sent to the acceptance module as acceptance reference values.

[0161] In one embodiment, the device further includes a threshold updating unit, wherein the threshold updating unit is specifically configured to:

[0162] Regularly obtain the actual duplication determination results within a preset time period from the acceptance module;

[0163] The first preset threshold and the second preset threshold are updated according to the pre-repeat determination result corresponding to the actual repeat determination result and the actual repeat determination result.

[0164] In one implementation, the threshold updating unit is specifically configured to:

[0165] Determine the false positive rate and the missed positive rate based on the pre-repeated determination result and the actual repeated determination result;

[0166] When the false positive rate is greater than the false negative rate, the first preset threshold and the second preset threshold are increased according to the preset adjustment rule;

[0167] When the false positive rate is less than the false negative rate, the first preset threshold and the second preset threshold are adjusted down according to the preset adjustment rule;

[0168] When the false positive rate is equal to the false negative rate, the first preset threshold and the second preset threshold are not adjusted.

[0169] The present application also provides a device for processing report information. Figure 6 As shown, it shows a schematic diagram of the structure of a processing device suitable for implementing the reporting information in the embodiment of the present application. The processing device for reporting information in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The reporting information processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0170] like Figure 6 As shown, the report information processing device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the report information processing device is powered on, various programs and data required for the operation of the report information processing device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0171] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a memory card, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the reporting information processing device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The report information processing device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0172] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any method for processing reporting information provided in an embodiment of the present application.

[0173] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0174] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the method of each embodiment of the present application.

[0175] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0176] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state hard disk (SSD)), etc.

Claims

1. A method for processing report information, characterized in that: include: Receive current reporting information; Determine whether there is second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information; When the second historical reporting information exists in the first historical reporting information, the current reporting information and the second historical reporting information are processed to obtain a current reporting information vector and a second historical reporting information vector; a text similarity set between the current reporting information vector and the second historical reporting information vector is calculated; the text similarity set includes at least one text similarity; The highest text similarity in the text similarity set is input into the decision tree, and a duplication determination is performed on the current reporting information based on a preset determination rule to obtain a duplication determination result.

2. The method for processing report information according to claim 1, characterized in that: The determining whether there is second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information includes: Determine whether the person being reported in the current reporting information is included in the set of historical persons being reported in the first historical reporting information.

3. The method for processing report information according to claim 2, characterized in that: Also includes: When the person being reported in the current reporting information does not exist in the set of historical reporters in the first historical reporting information, it is determined that the current reporting information is a non-duplicate report, and the current reporting information is sent to the reporting acceptance module.

4. The method for processing report information according to claim 1, characterized in that: The step of inputting the highest text similarity in the text similarity set into a decision tree and performing repeated determination based on a preset determination rule comprises: Determining whether the highest text similarity is greater than a first preset threshold; When the highest text similarity is greater than the first preset threshold, determining that the current report information is a duplicate report; When the highest text similarity is less than the first threshold value, calculating the proportion of repeated reports in the second historical report information; The proportion and the highest text similarity are used to calculate a comprehensive score of the historical repetition rate of the reported person; Determining whether the comprehensive score is greater than a second preset threshold; When the comprehensive score is greater than the second preset threshold, determining that the current report information is a repeated report; When the comprehensive score is less than the second preset threshold, the current report information is determined to be a non-duplicate report.

5. The method for processing report information according to claim 4, characterized in that: Also includes: When the comprehensive score is less than the second preset threshold and is within the preset confidence interval, the comprehensive score is used as the duplicate report confidence value, and a non-duplicate report confidence value is generated based on the comprehensive score; The duplicate report trust value and the non-duplicate report trust value are sent to the acceptance module as acceptance reference values.

6. The method for processing report information according to claim 4, characterized in that: Also includes: Regularly obtain the actual duplication determination results within a preset time period from the acceptance module; The first preset threshold and the second preset threshold are updated according to a pre-repeat determination result corresponding to the actual repeat determination result and the actual repeat determination result.

7. The method for processing report information according to claim 6, characterized in that: The updating of the first preset threshold and the second preset threshold according to the pre-repeat determination result corresponding to the actual repeat determination result and the actual repeat determination result includes: Determine the false positive rate and the false negative rate based on the pre-repeat determination result and the actual repeat determination result; When the false positive rate is greater than the false negative rate, increasing the first preset threshold and the second preset threshold according to a preset adjustment rule; When the false positive rate is less than the false negative rate, reducing the first preset threshold and the second preset threshold according to the preset adjustment rule; When the false positive rate is equal to the false negative rate, the first preset threshold and the second preset threshold are not adjusted.

8. A device for processing report information, characterized in that: include: Receiving unit, judging unit, calculating unit and determining unit; wherein: The receiving unit is used to receive current reporting information; The judging unit is used to judge whether there is a second historical reporting information in the first historical reporting information that is consistent with the reported person in the current reporting information; The calculation unit is used for processing the current reporting information and the second historical reporting information to obtain a current reporting information vector and a second historical reporting information vector when the second historical reporting information exists in the first historical reporting information; calculating a text similarity set between the current reporting information vector and the second historical reporting information vector; the text similarity set includes at least one text similarity; The determination unit is used to input the highest text similarity in the text similarity set into a decision tree, and perform a repeat determination on the current reporting information based on a preset determination rule to obtain a repeat determination result.

9. A device for processing report information, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the reporting information processing device can implement the reporting information processing method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the method for processing reporting information as described in any one of claims 1 to 7.

Images