Information auditing method, electronic equipment and computer program product

Abstract

The invention provides an information auditing method, electronic equipment and a computer program product. The method disclosed by the invention comprises the following steps: acquiring to-be-audited information; auditing the to-be-audited information by using the primary auditing model to obtain a preliminary auditing result; determining a model weight value corresponding to each partner model in the partner group based on the to-be-audited information or the preliminary audit result; inputting the to-be-audited information and the related attribute information of the partner role into a corresponding partner model to obtain a plurality of partner audit results; and inputting the preliminary auditing result, the reviewer auditing result and the model weight value into a judge model, and generating a final auditing result of the to-be-audited information.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to information verification methods, electronic devices, and computer program products. Background Technology

[0002] With the widespread application of artificial intelligence (AI) technology, automated review systems based on AI models have become core support for business scenarios such as credit risk control, content security, and transaction anti-fraud. These systems typically rely on a single model or rule engine, quickly outputting a binary "pass" or "reject" decision by extracting features and matching patterns from input data. While efficient and low-cost, their "black box" decision-making and single judgment dimension make them highly susceptible to misjudgment when faced with complex, ambiguous, or adversarial samples, potentially misclassifying high-risk content as compliant and leading to serious errors.

[0003] Current technologies are constantly being improved. For example, continuous optimization of model algorithms can improve the accuracy of individual models, but this faces the risks of diminishing marginal returns and overfitting. Alternatively, adding manual review processes can improve accuracy, but this significantly increases operating costs and sacrifices the efficiency of automated review. Therefore, a solution is needed to enhance the information review capabilities of models. Summary of the Invention

[0004] This disclosure provides information verification methods, electronic devices, and computer program products.

[0005] According to a first aspect of this disclosure, an information review method is provided. The method specifically includes: acquiring information to be reviewed; reviewing the information using a presiding judge model to obtain preliminary review results; determining the model weight values ​​corresponding to each juror model in the jury based on the information to be reviewed or the preliminary review results; inputting the information to be reviewed and juror role-related attribute information into the corresponding juror models to obtain multiple juror review results; and inputting the preliminary review results, juror review results, and model weight values ​​into a judge model to generate the final review result for the information to be reviewed.

[0006] According to the above scheme, by intelligently identifying the type and scenario of the information to be reviewed through semantic tags, and dynamically adjusting the weights of jurors accordingly, the system can concentrate computing resources on the most relevant review dimensions, avoiding full review and thus significantly improving processing efficiency while ensuring accuracy. Since all samples (regardless of the main reviewer's result) must be reviewed by a multi-role jury, this scheme can effectively intercept misjudgments by the main reviewer model, fundamentally solving the problem of misjudgments caused by model illusions in existing technologies.

[0007] According to at least one embodiment of this disclosure, the jury model includes at least one of: rule engine jury, model jury, user behavior jury, and business strategy jury.

[0008] According to at least one embodiment of this disclosure, determining the model weight value corresponding to each juror model in the jury based on the information to be reviewed or the preliminary review result includes: determining the model weight value corresponding to each juror model in the jury based on the preset weight value carried by the information to be reviewed; or, obtaining the semantic tags carried in the preliminary review result; wherein the semantic tags are used to characterize the data type or business scenario of the information to be reviewed; obtaining a preset weight mapping table, wherein the weight mapping table contains the correspondence between the semantic tags and the model weight values ​​of each juror model; querying the weight mapping table according to the semantic tags to obtain the model weight value of each juror model that matches the information to be reviewed.

[0009] According to the above scheme, by combining semantic tags with a weight mapping table, accurate and dynamic configuration of review resources is achieved. This not only ensures that the most relevant review model is available in various business scenarios, effectively reducing misjudgments caused by a single model, but also enables the system to have self-learning and self-optimization capabilities through the dynamic adjustment mechanism of weights, continuously improving the overall review capabilities and effectiveness.

[0010] According to at least one embodiment of this disclosure, when the juror model is a rule engine juror, the information to be reviewed and the juror role-related attribute information are input into the corresponding juror model to obtain multiple juror review results, including: matching the information to be reviewed with a preset rule base, wherein the rule base includes blacklists, whitelists, regular expressions, and hard threshold rules; judging based on the matching results, if the information to be reviewed matches any rejection rule in the blacklist or whitelist, the review result is output as "failed" and the corresponding failure reason identifier is output; if no rejection rule is matched, the review result is output as "passed".

[0011] According to the above scheme, the rule engine juror, as a key role in the jury, provides a certain and highly reliable basic review capability, which can effectively block obvious violations and avoid misjudgments by the system at the basic rule level.

[0012] According to at least one embodiment of this disclosure, when the juror model is a model juror, the information to be reviewed and the attribute information related to the juror role are input into the corresponding juror model to obtain multiple juror review results, including: extracting features from the information to be reviewed to generate feature vectors; inputting the feature vectors into the juror model to obtain the risk score predicted by the model; determining and outputting the review result based on the comparison result of the risk score and a preset score threshold; if the review result is unsuccessful, outputting the key features that led to the high risk score as the reason for failure.

[0013] According to the above scheme, the model juror, through machine learning technology, can identify complex risk patterns that are difficult for rule engines to capture, significantly improving the system's ability to identify gray-scale samples and novel risks. Its output risk score, rather than a simple binary judgment, makes the review results more accurate and quantifiable; while the output mechanism of key features provides strong interpretability support for the review results, facilitating not only manual review. As a member of the jury, the model juror's judgment complements that of other roles (such as rule engines and behavioral analysis), forming a multi-perspective, multi-dimensional risk assessment, fundamentally reducing the risk of misjudgment by a single model.

[0014] According to at least one embodiment of this disclosure, when the juror model is a user behavior juror, the information to be reviewed and the relevant attribute information of the juror role are input into the corresponding juror model to obtain multiple juror review results, including: obtaining the historical behavior sequence corresponding to the user identifier based on the user identifier in the information to be reviewed; analyzing the behavior anomaly score of the current behavior of the user corresponding to the user identifier using an anomaly detection algorithm and the historical behavior sequence; determining and outputting the review result based on the comparison result of the behavior anomaly score and the behavior threshold: if the behavior anomaly score exceeds the behavior threshold, the review result is output as "failed" and the abnormal behavior type is output as the reason for failure; otherwise, "passed" is output.

[0015] According to the above scheme, user behavior jurors can identify risks that are difficult to detect based on a single request by analyzing users' historical behavior patterns, significantly improving the ability to detect abnormal behavior. As key members of the jury, the judgments of user behavior jurors can complement those of other roles such as rule engines and model jurors, forming a multi-dimensional risk assessment system.

[0016] According to at least one embodiment of this disclosure, when the juror model is a business strategy juror, the information to be reviewed and the relevant attribute information of the juror role are input into the corresponding juror model to obtain multiple juror review results, including: obtaining the current business scenario, channel source or activity identifier; and a set of business strategy rules matching the current scenario; executing the business strategy rules one by one, and judging according to the execution results. If any rule is violated, the review result is output as "failed", and the violated strategy identifier is output as the reason for failure; if the behavior abnormality score does not exceed the behavior threshold, the review result is output as "passed".

[0017] According to the above scheme, business strategy jurors enable the review system to flexibly adapt to constantly changing business needs without frequent modifications to the underlying code. As key members of the jury, the judgments of business strategy jurors complement those of other technical jurors (such as rule engines and model analysis), effectively preventing review errors caused by missing or mismatched business strategies.

[0018] According to at least one embodiment of this disclosure, preliminary review results, juror review results, and model weight values ​​are input into a judge model to generate a final review result for the information to be reviewed. This includes: receiving preliminary review results, multiple juror review results, and model weight values ​​corresponding to each juror model; determining whether any juror's review result is unsuccessful and its corresponding juror model is preset as a key role model; if so, directly generating a failed final review result; if not, calculating a weighted comprehensive score based on each juror's review result and its corresponding model weight value; comparing the weighted comprehensive score with a comprehensive threshold; if the weighted comprehensive score is greater than the comprehensive threshold, generating a passed final review result; if the weighted comprehensive score is not greater than the comprehensive threshold, generating a failed final review result.

[0019] After generating the final review result, the process also includes: if the final review result is unsuccessful, summarizing the reasons for failure corresponding to all the jurors' review results that were deemed unsuccessful; and sending the summarized reasons for failure along with the final review result to the user.

[0020] According to the above scheme, the judge model achieves dual protection for review and judgment through a decision-making mechanism that combines a key role's veto power with a confidence-weighted scoring system. The mandatory rejection mechanism of key roles ensures zero tolerance for high-risk content. The confidence-weighted scoring mechanism enables the system to make more refined and reasonable judgments on gray-scale samples, avoiding misjudgments that may be caused by simple majority decisions.

[0021] According to at least one embodiment of this disclosure, the training process of the juror model and the judge model includes: collecting historical review samples, which include: sample information, manually annotated final review results, and reasons for failure; training each juror model to be trained using the historical review samples to obtain an initial juror model; determining the initial model weight values; and training the judge model to be trained using the initial jury results of the initial juror model, the corresponding initial model weight values, and the corresponding historical review samples to obtain a judge model and generate a weight mapping table.

[0022] According to the above scheme, by using real historical samples for training, it is ensured that each juror model and judge model can accurately capture risk patterns in business scenarios; secondly, the generation mechanism of the weight mapping table enables the system to intelligently allocate the speaking power of each juror according to different types of review tasks, which significantly improves the targeting and accuracy of the review; the entire training process is not limited to the optimization of model parameters, but also includes the dynamic adjustment of weight configuration, so that the system can be optimized as the business environment changes.

[0023] According to a second aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs a first aspect of any embodiment of this disclosure.

[0024] According to a third aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement a first aspect of any embodiment of this disclosure.

[0025] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a first aspect of any embodiment of this disclosure. Attached Figure Description

[0026] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0027] Figure 1 This is a flowchart illustrating the information verification method proposed in this disclosure.

[0028] Figure 2 This is a flowchart illustrating the model weight determination method provided in an embodiment of this disclosure.

[0029] Figure 3 A schematic diagram of the rule engine juror review process provided in this embodiment of the disclosure.

[0030] Figure 4 A schematic diagram of the rule engine juror review process provided in this embodiment of the disclosure.

[0031] Figure 5 This is a schematic diagram of the user behavior juror review process provided in an embodiment of the present disclosure.

[0032] Figure 6 This is a schematic diagram of the business strategy juror review process provided in an embodiment of the present disclosure.

[0033] Figure 7 This is a schematic diagram of the judge model review process provided in an embodiment of this disclosure.

[0034] Figure 8 This is a schematic flowchart of the model training method provided in the embodiments of this disclosure.

[0035] Figure 9 This is a schematic block diagram of the structure of an information verification device according to one embodiment of the present disclosure.

[0036] Figure 10 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0037] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0038] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0039] Figure 1 This is a flowchart illustrating the information verification method proposed in this disclosure. Figure 1 The method shown includes steps 101 to 105. This method can be executed by an electronic device (such as a local computer, cloud server, etc.).

[0040] Specifically, Figure 1 The method shown includes step 101: obtaining information to be reviewed.

[0041] In practical applications, the system receives pending review information from users or upstream business systems (such as content publishing platforms, credit application systems, risk control engines, etc.). This pending review information includes not only core review data (e.g., user-submitted text content, transaction amount, user identity information, etc.) but also request context (e.g., the source of the request, the current business activity ID, request timestamp, user device fingerprint, etc.). This allows the juror model to utilize the contextual information for accurate review.

[0042] Step 102: Use the lead reviewer model to review the information to be reviewed and obtain preliminary review results.

[0043] In practical applications, upon receiving information to be reviewed, a preliminary review is first conducted using a master review model. The role of the master review model is not to make a final decision, but rather to quickly screen and classify the input information to be reviewed, providing directional guidance for subsequent jury review. The master review model can be a lightweight AI model or a hybrid system combining rules and models; the key is its fast response speed and low resource consumption, so as not to affect the overall review efficiency in high-concurrency scenarios. Taking common text content review as an example, the master review model can choose a lightweight BERT model (such as TinyBERT) that has undergone distillation and compression, or a high-efficiency TextCNN model. When receiving a user-submitted comment text, the master review model first vectorizes the text, then extracts semantic features through its internal neural network layers, ultimately outputting two key results: the first is the preliminary review result, which is a binary judgment; for example, if the model calculates the probability that the comment is high-risk content to be 0.82, and the preset threshold is 0.7, then the preliminary review result is "not approved." The second is the semantic label, which is generated by the master review model during the inference process. For example, during the judgment process, the lead review model might detect frequent occurrences of words like "part-time job" and "rebate" in the text. Combining this with the knowledge learned during training, it can not only assess risk but also identify the "data type" of the comment as "user-generated content" and its "business scenario" as potentially involving "suspected fraudulent advertising or order manipulation." This tagging information is then output in a structured manner. In a financial transaction scenario, the lead review model might be an XGBoost-based gradient boosting tree model. It receives transaction data containing the amount, payee, and user ID, and quickly calculates a risk score. Besides outputting a preliminary review result (e.g., a score of 0.65, below the threshold of 0.8, resulting in a pass), it can also deduce semantic tags based on activated key decision paths. These semantic tags are then fed into a role weighting allocator to dynamically adjust the weights of subsequent jurors. For instance, when the tag indicates "suspected fraud," the weights of "rule engine jurors" and "behavioral jurors" are automatically increased, giving them greater influence in the final decision. In this way, the presiding judge model not only completed the initial risk filtering, but also pointed out the direction for subsequent refined and scenario-based review through semantic tags, thereby making the operation of the entire jury mechanism more accurate and efficient.

[0044] Step 103: Determine the model weight values ​​for each juror model in the jury based on the information to be reviewed or the preliminary review results. It should be noted that the jury mentioned here contains multiple juror models, each with its own unique review capabilities. For example, some models are used for reviewing according to rules, while others are used for reviewing user behavior, etc.

[0045] There are two ways to determine the model weights for each juror model: one is by configuring the model weights for each juror model by staff, and the other is by determining the weights based on the semantic tags carried in the preliminary review results output by the lead reviewer model. The semantic tags are used to characterize the data type or business scenario of the information to be reviewed.

[0046] Specifically, semantic tags are input into a role weight allocator. This allocator, based on a pre-defined weight mapping table, queries and determines the weight values ​​that each juror model should be assigned to in the scenario represented by the current semantic tag. For example, when the semantic tag is "text comment + sensitive topic discussion," the weight of the "model juror" (skilled in NLP analysis) will be increased; while when the tag is "transaction order + large transfer," the weights of the "rule engine juror" and the "behavioral juror" will be increased. Through the dynamic allocation of model weight values, the accuracy of the entire jury mechanism can be effectively improved.

[0047] Step 104: Input the information to be reviewed and the relevant attribute information of the juror role into the corresponding juror model to obtain multiple juror review results.

[0048] It should be noted that the juror role-related attribute information mentioned here can be understood as additional information required by a specific juror model in addition to the core data when performing its review tasks. For example, a behavioral juror needs a "user ID" to query historical behavior, while a business strategy juror needs a "channel ID" to match the corresponding strategy.

[0049] Based on the configuration, the information to be reviewed, along with contextual information related to each juror's role (i.e., juror role-related attribute information, such as user historical behavior data, current activity strategies, etc.), is distributed in parallel to each juror model in the jury. Each juror model independently executes its proprietary review logic and outputs its own review result (pass / fail) and (in the case of failure) the specific reason for failure. The types of juror models include, but are not limited to: rule engine jurors, model jurors, user behavior jurors, and business strategy jurors. In practical applications, juror models can be added or removed according to business needs. For example, a financial risk juror model can be added, or an image authenticity verification model can be added.

[0050] Step 105: Input the preliminary review results, juror review results, and model weight values ​​into the judge model to generate the final review results of the information to be reviewed.

[0051] The judge model, acting as the final decision-maker, aggregates the preliminary review results from the presiding module, the review results from all juror models, and their corresponding model weights. The judge model does not judge the raw data; instead, it makes a comprehensive decision based on pre-defined decision rules (such as veto power by key juror models, weighted scoring rules, majority voting rules, etc.), outputting a unique final review result. For example, if any juror pre-defined as a "key role" (such as a rule engine juror) is deemed unqualified, the judge model directly outputs "unqualified"; otherwise, the judge model calculates a weighted comprehensive score and compares it with a threshold to reach a conclusion.

[0052] Based on the publicly available solutions described above, by intelligently identifying the type and scenario of the information to be reviewed through semantic tags and dynamically adjusting juror weights accordingly, computational resources can be concentrated on the most relevant review dimensions, avoiding full-scale review and thus significantly improving processing efficiency while ensuring accuracy. Since all samples (regardless of the presiding judge's decision) must be reviewed by a multi-role jury, and the judge model supports the rule of veto power for key roles, this solution can effectively intercept misjudgments by the presiding judge model, fundamentally solving the problem of model illusion-induced misjudgments in existing technologies. Each juror's output includes a reason for failure; this information, after being aggregated by the judge module, is not only used for final decision-making but also provides clear and traceable data support for subsequent manual review, model optimization, and compliance audits.

[0053] In one or more embodiments of this disclosure, such as Figure 2 This is a flowchart illustrating the model weight determination method provided in an embodiment of this disclosure. Figure 2 As shown, step 103, which involves determining the model weight values ​​corresponding to each juror model in the jury based on the information to be reviewed or the preliminary review results, includes: Step 1031: Determining the model weight values ​​corresponding to each juror model in the jury based on the preset weight values ​​carried by the information to be reviewed. Alternatively, Step 1032: Obtaining the semantic tags carried in the preliminary review results; wherein the semantic tags are used to characterize the data type or business scenario of the information to be reviewed. Step 1033: Obtaining a preset weight mapping table, wherein the weight mapping table contains the correspondence between semantic tags and the model weight values ​​of each juror model. Step 1034: Querying the weight mapping table according to the semantic tags to obtain the model weight values ​​of each juror model that matches the information to be reviewed.

[0054] In practical applications, as mentioned above, the model weight values ​​can be pre-configured by the user. That is, when obtaining the information to be reviewed, the model weight values ​​corresponding to each juror model can be obtained at the same time. These model weight values ​​are relatively fixed.

[0055] In addition, there is another approach that allows for dynamic adjustment of model weights. Specifically, this requires maintaining a weight mapping table. This table is a structured data configuration that stores the mapping relationships between various preset semantic label combinations and the weight values ​​of each juror model. Here, semantic labels refer to classification identifiers generated by the lead review model to characterize the inherent attributes of the information to be reviewed, such as "data type" (e.g., text comments, transaction orders) and "business scenario" (e.g., new user registration). "Juror models" refer to independent review units in the jury, such as rule engine jurors and model jurors, each representing different review logics. "Model weight value" is a numerical indicator representing the influence of the juror model in the final decision.

[0056] When it is necessary to determine the weight for a review, it performs the following operations: obtains the semantic tags associated with the information to be reviewed, and then uses these semantic tags as query keys to perform a matching search in the weight mapping table.

[0057] It will find the set of configurations that best matches the current semantic tag and extract the model weight values ​​corresponding to each juror model (such as rule engine, NLP model, etc.). For example, when the semantic tag is "content type: user comment" and "business scenario: sensitive topic discussion", it may find and apply a set of configurations with high weights over "NLP model juror" and "rule engine juror"; while when the tag is "data type: transaction order" and "business scenario: large transfer", it will load another set of configurations with high weights over "behavioral juror" and "risk scoring model".

[0058] It's important to note that this weighting mapping table is not static. It supports dynamic adjustments—it can be updated through operational configuration or automatically optimized by the system based on historical review data. For example, the system can analyze the accuracy of each juror model over a period of time. If a particular model consistently performs well in a specific scenario, its weight under the corresponding semantic tag can be automatically increased. This dynamic adjustment capability allows the entire review system to evolve with changes in the business environment and risk profile, without requiring frequent code modifications or architecture refactoring.

[0059] Based on the publicly available solutions described above, combining semantic tags with a weight mapping table enables precise and dynamic configuration of review resources. This not only ensures the most relevant review model across various business scenarios, effectively reducing misjudgments caused by a single model, but also, through the dynamic weight adjustment mechanism, gives the system self-learning and self-optimization capabilities, continuously improving overall review capabilities and effectiveness.

[0060] In one or more embodiments of this disclosure, such as Figure 3This is a schematic diagram of the rule engine juror review process provided in an embodiment of this disclosure. Figure 3 As described in step 104, when the juror model is a rule engine juror, the information to be reviewed and the relevant attribute information of the juror role are input into the corresponding juror model to obtain multiple juror review results, including: Step 1041: Matching the information to be reviewed with a preset rule base, wherein the rule base includes blacklists / whitelists, regular expressions, and hard threshold rules. Step 1042: Judging based on the matching results, if the information to be reviewed matches any rejection rule in the blacklist / whitelist, the review result is output as "failed," and the corresponding failure reason identifier is output. Step 1043: If no rejection rule is matched, the review result is output as "passed."

[0061] In practical applications, the information to be reviewed and the relevant attribute information of the jurors are input into the juror model of the rule engine, and the following operations are performed: First, the system calls the preset rule library, which contains three core rule components: blacklists and whitelists, regular expressions, and hard threshold rules (it should be noted that the rules in this rule library can be added, deleted, or modified according to actual business needs). Blacklists and whitelists refer to pre-configured lists of allowed or prohibited entities, such as user ID blacklists, device fingerprint whitelists, and sensitive word libraries. Regular expressions are used to match content with specific formats or patterns, such as ID card number format verification and matching of specific risk keywords. Hard threshold rules refer to insurmountable numerical boundary conditions, such as "the amount of a single transaction shall not exceed 1 million yuan" or "the number of users registered from the same IP address shall not exceed 50 per day."

[0062] The rule engine's juror works by matching the information to be reviewed against the aforementioned rule base one by one. For example, in content review scenarios, the text content submitted by the user is matched against a blacklist of sensitive words to check for any prohibited terms; in financial risk control scenarios, the transaction amount is compared with a preset hard threshold to determine if it exceeds a safe range. If the matching process finds that the information to be reviewed matches any rejection rule in the blacklist or whitelist (such as containing blacklisted sensitive words, the user ID being on the rejection list, or the amount exceeding the threshold), the rule engine juror immediately terminates the subsequent matching and outputs the review result as "failed," while generating a corresponding "failure reason identifier." If the information to be reviewed successfully passes all rule checks and does not match any rejection rules, the rule engine juror outputs the review result as "passed."

[0063] Rule engine jurors can be implemented in various ways, such as: Drools rule engine: an open-source business rule management system that supports Java-based rule definition and execution; custom rule matching systems: efficient matching algorithms built on data structures such as hash tables, tries, or AC automata; lightweight script engines: such as Lua or JavaScript engines, used to execute dynamically configured rule scripts; and database rule tables: storing rules in a database for fast matching via SQL queries.

[0064] Taking content security review as an example, when a user submits a real estate review, the rules engine's reviewers will first check whether the review contains words on the sensitive word blacklist (such as phrases like "negotiable agent fees" or "private transactions"). They will also verify whether the review's length complies with platform regulations (hard threshold rules) and confirm whether the publisher is on the whitelist. If the review contains keywords related to "private transactions," it will be immediately rejected, and the reason for failure will be marked.

[0065] Based on the publicly available schemes mentioned above, it is known that the rule engine juror, as a key role in the jury, provides a certain and highly reliable basic review capability, which can effectively block obvious violations and avoid misjudgments by the system at the basic rule level.

[0066] In one or more embodiments of this disclosure, such as Figure 4 This is a schematic diagram of the rule engine juror review process provided in an embodiment of this disclosure. Figure 4 As shown, when the juror model is a model juror, as described in step 104, the information to be reviewed and the relevant attribute information of the juror role are input into the corresponding juror model to obtain multiple juror review results, including: Step 401: Extract features from the information to be reviewed to generate a feature vector. Step 402: Input the feature vector into the juror model to obtain the risk score predicted by the model. Step 403: Determine and output the review result based on the comparison result between the risk score and the preset score threshold. Step 404: If the output review result is a failure, output the key features that caused the high risk score as the reason for failure.

[0067] The information to be reviewed and the relevant attribute information of the jurors are input into the model juror. Feature extraction is then performed on the information to be reviewed, transforming the raw data into a structured numerical representation. Feature extraction here refers to the process of identifying and quantifying key attributes that influence the review decision from the raw data. For example, in a text content review scenario, feature extraction may include word frequency statistics, sentiment analysis, and topic modeling; in a financial transaction review scenario, it may include the calculation of features such as transaction amount, time distribution, and user behavior patterns. These features are organized into a feature vector, that is, an ordered array of multiple numerical features, which serves as the input to the model.

[0068] The generated feature vectors are then input into a pre-trained model juror, which performs inference calculations and outputs a risk score. The risk score is a value between 0 and 1, representing the probability or degree of risk in the information to be reviewed; a higher value indicates a greater risk. The model juror can be implemented using models such as: XGBoost / LightGBM models: machine learning models based on gradient boosting trees, adept at handling structured data and widely used in financial risk control; deep neural networks: architectures such as DNN, CNN, or RNN, suitable for handling high-dimensional and complex data; NLP models: pre-trained language models such as BERT and RoBERTa, used for risk identification of text content; and ensemble learning models: models that fuse the prediction results of multiple base models to improve overall performance.

[0069] After calculating the risk score, it is compared with a preset scoring threshold. This threshold can be a pre-configured cutoff value used to transform continuous risk scores into a binary decision. If the risk score is greater than the threshold, the audit result is "fail"; if the risk score is less than or equal to the threshold, the audit result is "approved".

[0070] It's important to note that when the model juror determines a review to be unsuccessful, it further analyzes the key features that led to the high-risk score and outputs these features as reasons for failure. These key features refer to the features that contribute most to the risk score and can be obtained through interpretable AI techniques such as SHAP values, LIME interpretation, and gradient analysis. For example, in a content moderation scenario, if a user comment is judged to be high-risk, the model juror might output "Reason for failure: Contains negative sentiment keywords: fraud, order manipulation, and the text similarity is 85% with historical violations." In a financial transaction scenario, it might output "Reason for failure: Abnormal transaction time (3 AM), recipient is a newly registered account, transaction amount deviates from the user's historical average by 3 standard deviations."

[0071] Taking property listing review as an example, when a user submits a property description, the model reviewer first extracts text features (such as title keywords, description length, number of images, etc.) and user behavior features (such as posting frequency, historical violation records, etc.) to form a feature vector; then the vector is input into a pre-trained NLP model to obtain a risk score of 0.82; since the score exceeds the preset threshold of 0.75, it is judged as "failed" and the key feature is output as the reason for failure: "The title contains the keyword 'urgent sale without tax'".

[0072] Based on the publicly available solutions, it is evident that model jurors, through machine learning technology, can identify complex risk patterns that are difficult for rule engines to capture, significantly improving their ability to identify gray-scale samples and novel risks. Their output risk scores, rather than simple binary judgments, make the review results more accurate and quantifiable; while the key feature output mechanism provides strong interpretability support for the review results, facilitating not only manual review. As a member of the jury, the judgments of model jurors complement those of other roles (such as rule engines and behavioral analysis), forming a multi-perspective, multi-dimensional risk assessment, fundamentally reducing the risk of misjudgment by a single model.

[0073] In one or more embodiments of this disclosure, such as Figure 5 This is a schematic diagram of the user behavior juror review process provided in an embodiment of this disclosure. Figure 5 As shown, when the juror model is a user behavior juror, step 104 involves inputting the information to be reviewed and the relevant attribute information of the juror role into the corresponding juror model to obtain multiple juror review results, including: Step 501: Obtain the historical behavior sequence corresponding to the user identifier based on the user identifier in the information to be reviewed. Step 502: Analyze the behavior anomaly score of the current behavior of the user corresponding to the user identifier using an anomaly detection algorithm and the historical behavior sequence. Step 503: Determine and output the review result based on the comparison result between the behavior anomaly score and the behavior threshold: if the behavior anomaly score exceeds the behavior threshold, the review result is output as "failed," and the abnormal behavior type is output as the reason for failure. Step 504: if the behavior anomaly score does not exceed the behavior threshold, the review result is output as "passed."

[0074] In practical applications, the information to be reviewed and the relevant attribute information of the juror role are input into the user behavior juror database. The user identifier is extracted from the information to be reviewed; this identifier can be a field that uniquely identifies the user entity, such as user ID, device ID, or account identifier. Based on this user identifier, historical behavior sequences are retrieved from the behavior database. These historical behavior sequences refer to the juror role-related attribute information of the juror model, which can be a complete record of the user's behavior over a past period (e.g., 30 days), including but not limited to structured data such as login time, operation type, transaction frequency, and access path. This data is typically stored in the form of a time series table, recording the regularity and stability of user behavior.

[0075] After obtaining historical behavior sequences, user behavior jurors analyze them using anomaly detection algorithms to calculate anomaly scores. These anomaly detection algorithms are statistical or machine learning methods specifically designed to identify outliers in data. Examples include: Isolation Forest: an unsupervised anomaly detection algorithm based on decision trees, particularly suitable for high-dimensional data; Local Anomaly Factor (LOF): identifying anomalies by comparing the local density of data points with their neighbors; Gaussian Distribution Model: assuming normal behavior follows a Gaussian distribution, calculating the degree of deviation of the current behavior from the distribution; and Temporal Anomaly Detection Models: such as LSTM-AE (Long Short-Term Memory-Autoencoder), specifically used to analyze anomalous patterns in time series data.

[0076] Anomaly score is a quantitative indicator, usually normalized to the range of 0-1, representing the degree of deviation between the current behavior and the user's historical behavior patterns. The higher the value, the greater the likelihood of anomaly. For example, it may calculate the distance between the user's current login location and frequently used locations, the deviation of the current operation frequency from the historical average, the standard deviation of the current transaction amount from the historical distribution, and combine these factors into a final anomaly score.

[0077] After obtaining the abnormal behavior score, it is compared with a preset behavior threshold. The behavior threshold is a configurable cutoff value used to convert consecutive abnormal scores into a binary decision. If the abnormal behavior score exceeds the behavior threshold, the audit result is output as "failed," and the abnormal behavior type is output as the reason for failure. The abnormal behavior type refers to the specific abnormal pattern identified by the system, such as sudden changes in login location, abnormal operation frequency, or sudden increase in transaction amount. If the abnormal behavior score does not exceed the behavior threshold, the audit result is output as "approved."

[0078] This user behavior juror can be implemented using models such as: a time-series behavior analysis model, such as an LSTM-based sequence model specifically designed to capture the temporal dependencies of behavior patterns; or a rule-plus-model hybrid system, combining expert rules and machine learning models for comprehensive judgment.

[0079] Taking a financial risk control scenario as an example, when a user initiates a large transfer request, the user behavior judge first obtains the user's unique identifier (such as user ID), and then queries their behavior sequence over the past 30 days, including login location, transaction amount distribution, and transaction frequency. Using the Isolation Forest algorithm, it was found that the user's current login location is more than 2000 kilometers away from their usual location, and the transaction amount exceeds the historical average by 5 standard deviations, resulting in a behavioral anomaly score of 0.85. Since this score exceeds the preset behavioral threshold of 0.7, it is judged as a failure, and the reason for failure is output: "Abnormal behavior type: Login from a different location + Abnormal transaction amount".

[0080] Based on the publicly available solutions, user behavior jurors, by analyzing users' historical behavior patterns, can identify risks that are difficult to detect based on a single request, significantly improving the ability to detect abnormal behavior. As key members of the jury, the judgments of user behavior jurors can complement those of other roles such as rule engines and model jurors, forming a multi-dimensional risk assessment system.

[0081] In one or more embodiments of this disclosure, such as Figure 6 This is a schematic diagram of the business strategy juror review process provided in an embodiment of this disclosure. Figure 6 As shown, when the juror model is a business strategy juror, step 104 involves inputting the information to be reviewed and the juror role-related attribute information into the corresponding juror model to obtain multiple juror review results, including: Step 601: Obtaining the current business scenario, channel source, or activity identifier; and the business strategy rule set matching the current scenario. Step 602: Executing each business strategy rule one by one and judging based on the execution result. If any rule is violated, the review result is output as "failed," and the violated strategy identifier (strategy ID) is output as the reason for failure. Step 603: If all rules are satisfied, the result is output as "passed."

[0082] Extract the current business scenario, channel source, or activity identifier from the juror role-related attribute information. The current business scenario refers to the specific business environment in which this review takes place, such as new user registration, transactions during a major promotional period, or the posting of second-hand housing listings; the channel source refers to the channel through which the request was initiated, such as an app, a mini-program, or a partner interface; and the activity identifier refers to the identifier of an ongoing marketing activity or a special business period, such as the 618 shopping festival or Spring Festival special offers. This information collectively constitutes the key context for business strategy jurors to understand the review background.

[0083] Based on the above information, a set of business strategy rules matching the current scenario is then obtained. This set of rules is a dynamically updated storage system for business rules. For example, during the 18.18 promotion, the review standards for new users are relaxed; applications from channel A require additional mobile phone number verification; and during the night (11:00 PM to 6:00 AM), stricter reviews are implemented for high-risk product transactions. These rules are typically configured by the business team based on market strategies, compliance requirements, and risk appetite.

[0084] After obtaining the business strategy rule set, the business strategy reviewers will execute these rules one by one. If any violation of a business strategy rule is found during execution, subsequent rule checks will be immediately terminated, and the review result will be output as "failed," along with the violated strategy ID as the reason for failure. The strategy ID is a unique identifier for each business strategy rule, facilitating subsequent tracking and analysis. If all business strategy rules are satisfied, the review result will be output as "passed."

[0085] Business strategy jurors can be implemented in the following ways, for example: Business rule engines: such as Drools and Easy Rules, which are open source rule engines that support complex condition judgments and rule chains.

[0086] Strategy Decision Table: Business strategies defined in tabular form, making them easy for business personnel to configure and understand.

[0087] For example, when a user submits a property listing, the business strategy reviewer first identifies the current business scenario as routine review and the source as the app. Then, they query the strategy configuration center to obtain the set of business strategy rules matching this scenario, including rules such as requiring app-submitted listings to contain at least 3 images and limiting ordinary users to a maximum of 5 listings per day. Next, they execute these rules one by one and find that the listing only contains 2 images, violating the first rule. Therefore, the review result is "failed," with the reason for failure marked as: insufficient listing images.

[0088] Based on the publicly available solutions, it is clear that business strategy jurors enable the review system to flexibly adapt to constantly changing business needs without frequent modifications to the underlying code. As key members of the jury, the judgments of business strategy jurors complement those of other technical jurors (such as rule engines and model analysis), effectively preventing review errors caused by missing or mismatched business strategies.

[0089] In one or more embodiments of this disclosure, such as Figure 7 This is a schematic diagram of the judge model review process provided in an embodiment of this disclosure. Figure 7 As shown, step 105 involves inputting the preliminary review results, juror review results, and model weight values ​​into the judge model to generate the final review result for the information to be reviewed. This includes: Step 1051: Receiving the preliminary review results, multiple juror review results, and the model weight values ​​corresponding to each juror model. Step 1052: Determining if any juror's review result is "fail" and their corresponding juror model is preset as a key role model. Step 1053: If yes, directly generating a "fail" final review result. Step 1054: If no, calculating a weighted comprehensive score based on each juror's review result and their corresponding model weight values. Step 1055: Comparing the weighted comprehensive score with a comprehensive threshold; if the weighted comprehensive score is greater than the comprehensive threshold, generating a "pass" final review result. Step 1056: If the weighted comprehensive score is not greater than the comprehensive threshold, generating a "fail" final review result.

[0090] After generating the final review result, the method further includes: Step 1057: If the final review result is unsuccessful, summarize the reasons for failure corresponding to all the jurors' review results that were deemed unsuccessful. Step 1058: Send the summarized reasons for failure along with the final review result to the user.

[0091] In practical applications, the preliminary review results, the review results of multiple jurors, and the model weight values ​​corresponding to each juror model are input into the judge model. The preliminary review results refer to the binary judgment (pass / fail) output by the presiding judge module; the juror review results refer to the judgment results output by each juror model after independent review; and the model weight values ​​are numerical values ​​that are dynamically determined based on semantic tags and represent the importance of each juror's opinion.

[0092] After receiving the above input, the judge model performs the first level of judgment: checking if any juror's review result is negative and if their corresponding juror model is preset as a key role model. It should be noted that the key role model here refers to a specific juror model that is granted veto power in the system configuration, such as a rule engine juror. If such a situation exists, the judge model immediately terminates subsequent judgments and directly generates a final negative review result. This mechanism ensures strict control over high-risk content.

[0093] If there is no veto by a key figure, the judge model will proceed to the second level of judgment: calculating a weighted composite score based on each juror's review results and their corresponding model weight values. The weighted calculation considers not only the preset model weight values ​​but also the confidence level of each juror's output. For example, for each juror, their weighted contribution score = model weight value × review result confidence level. Here, the review result confidence level refers to the probability that the juror model considers its output result, typically a value between 0 and 1. For example, a rule engine juror might have a confidence level of 1.0 when hitting a blacklist, while a model juror might output a risk score confidence level of 0.85. By introducing a confidence factor, the reliability of different jurors' judgments can be more finely differentiated, making the final decision more scientific and reasonable.

[0094] After obtaining the weighted composite score, the judge model compares it with a preset composite threshold. The composite threshold is a configurable cutoff value used to determine the standard for approval. If the weighted composite score is greater than the composite threshold, the final approval result is "approved"; if the weighted composite score is not greater than the composite threshold, the final approval result is "disapproved".

[0095] When the final review result is determined to be unsuccessful, the reasons for failure corresponding to the review results of all jurors who determined it to be unsuccessful are summarized. The reasons for failure refer to the specific reason identifier attached by each juror when outputting the unsuccessful result, such as "Abnormal behavior type: Login from a different location + Abnormal transaction amount". These reasons for failure are then structured and sent to the user or business system along with the final review result.

[0096] Based on the publicly available schemes described above, the judge model achieves dual safeguards for review and judgment through a decision-making mechanism that combines a veto by key roles with a confidence-weighted scoring system. The mandatory rejection mechanism by key roles ensures zero tolerance for high-risk content. The confidence-weighted scoring mechanism enables the system to make more refined and reasonable judgments on gray-scale samples, avoiding misjudgments that might result from simple majority decisions.

[0097] In one or more embodiments of this disclosure, such as Figure 8 This is a schematic flowchart illustrating the model training method provided in an embodiment of this disclosure. Figure 8 As shown, the training process for the jury model and judge model includes: Step 801: Collect historical review samples, which include sample information, manually annotated final review results, and reasons for failure. Step 802: Train each jury model to be trained using the historical review samples to obtain the initial jury model. Step 803: Determine the initial model weights. Step 804: Train the judge model to be trained using the initial jury results and corresponding initial model weights of the initial jury model, as well as the corresponding historical review samples, to obtain the judge model and generate a weight mapping table.

[0098] Historical review samples refer to complete datasets accumulated by the system during past review processes and verified by manual reviewers. Each sample contains three key pieces of information: first, sample information, i.e., the original data to be reviewed (such as text content submitted by users, transaction records, etc.); second, the final review result manually annotated, i.e., the correct judgment result (pass / fail) confirmed by professional reviewers; and third, the reason for failure, i.e., the specific rejection reasons output by each juror module when the sample is judged to be unsuccessful. This sample data usually comes from review requests marked as controversial or high-risk samples during the system's daily operation, as well as the results of periodic sampling reviews.

[0099] Based on collected historical review samples as training samples, training is conducted for each juror model to be trained. For rule engine jurors, the training process mainly optimizes their rule base and matching algorithm. By analyzing the characteristics of content that has been manually identified as violating in historical samples, the blacklists, whitelists, regular expressions, and threshold rules are continuously expanded and adjusted. For model jurors (such as NLP models and risk scoring models), supervised learning methods are used, with sample information as input and manually labeled results as labels for training, optimizing model parameters. For user behavior jurors, the training focus is on tuning the anomaly detection algorithm to more accurately identify anomalies that deviate from users' historical behavior patterns. For business strategy jurors, the main focus is on matching and verifying business rules with successful / failed cases in historical samples to optimize strategy configuration.

[0100] When training the juror model, it is also necessary to dynamically train the initial model weights. This process is based on the performance of each juror model on the historical sample validation set. By calculating metrics such as accuracy, recall, and F1 score, an initial weight is assigned to each juror model. For example, in a content moderation scenario, if the NLP model juror performs well on the validation set (F1 score of 0.85), while the rule engine performs only moderately (F1 score of 0.75), it may be assigned a higher initial weight.

[0101] The training process of the judge model simulates the actual review process: First, the initial jury model is used to predict historical samples to obtain the review results and reasons for failure for each juror; then, these results are combined with initial weight values ​​to generate weighted decision signals; finally, using manually labeled final review results as labels, the judge model is trained to learn how to correctly integrate these signals. During the judge model training process, a weight mapping table is simultaneously generated. This table records the mapping relationship between different semantic labels (such as content type: user comments, business scenario: sensitive topic discussion) and the optimal weights of each jury model. The generation of the weight mapping table is based on analysis: under specific semantic labels, which jury models perform best and should be assigned higher weights.

[0102] It should be noted that the training samples for the master review model also come from historical review data. During training, the sample information is used as input, and the manually labeled final review results are used as labels. The master review model is trained through supervised learning. To enable the master review model to output semantic labels, a multi-task learning mechanism is introduced during training: the main task is to predict the review results, and the auxiliary tasks are to predict the data type and business scenario.

[0103] Based on the publicly available solutions, it is known that by using real historical samples for training, the system ensures that each juror and judge model can accurately capture risk patterns in business scenarios. Secondly, the generation mechanism of the weight mapping table enables the system to intelligently allocate the speaking power of each juror according to different types of review tasks, significantly improving the targeting and accuracy of the review. The entire training process is not limited to the optimization of model parameters, but also includes the dynamic adjustment of weight configuration, enabling the system to optimize with changes in the business environment.

[0104] Based on any of the above embodiments, this disclosure also provides an information verification device. Figure 9 This is a schematic block diagram of the structure of an information verification device according to one embodiment of this disclosure. Figure 9 As shown, the information review device includes: an acquisition module 91, used to acquire information to be reviewed.

[0105] The first review module 92 is used to review the information to be reviewed using the master review model and obtain preliminary review results.

[0106] The weight determination module 93 is used to determine the model weight values ​​corresponding to each juror's model in the jury based on the information to be reviewed or the preliminary review results.

[0107] The second review module 94 is used to input the information to be reviewed and the relevant attribute information of the juror role into the corresponding juror model to obtain multiple juror review results.

[0108] The third review module 95 is used to input the preliminary review results, the jurors' review results, and the model weight values ​​into the judge model to generate the final review results of the information to be reviewed.

[0109] Optionally, the juror model includes at least one of the following: rule engine jurors, model jurors, user behavior jurors, and business strategy jurors.

[0110] The weight determination module 93 is used to determine the model weight value corresponding to each juror model in the jury based on the preset weight value carried by the information to be reviewed; or, to obtain the semantic tags carried in the preliminary review results; wherein the semantic tags are used to characterize the data type or business scenario of the information to be reviewed; to obtain a preset weight mapping table, wherein the weight mapping table contains the correspondence between the semantic tags and the model weight values ​​of each juror model; and to query the weight mapping table according to the semantic tags to obtain the model weight values ​​of each juror model that match the information to be reviewed.

[0111] When the juror model is a rule engine juror, the second review module 94 is used to match the information to be reviewed with a preset rule base, which includes blacklists, whitelists, regular expressions, and hard threshold rules. Based on the matching results, if the information to be reviewed matches any rejection rule in the blacklist or whitelist, the review result is output as "failed" and the corresponding failure reason identifier is output; if no rejection rule is matched, the review result is output as "passed".

[0112] When the juror model is a model juror, the second review module 94 is used to extract features from the information to be reviewed and generate feature vectors; input the feature vectors into the juror model to obtain the risk score predicted by the model; determine and output the review result of passing or failing based on the comparison result of the risk score and the preset score threshold; if the result is failing, output the key features that caused the high risk score as the reason for failure.

[0113] When the juror model is a user behavior juror, the second review module 94 is used to obtain the historical behavior sequence corresponding to the user identifier in the information to be reviewed; using the anomaly detection algorithm and the historical behavior sequence, analyze the behavior anomaly score of the current behavior of the user corresponding to the user identifier; and determine and output the review result based on the comparison result of the behavior anomaly score and the behavior threshold: if the behavior anomaly score exceeds the behavior threshold, the review result is output as "failed" and the abnormal behavior type is output as the reason for failure; otherwise, the result is output as "passed".

[0114] When the juror model is a business strategy juror, the second review module 94 is used to obtain the current business scenario, channel source or activity identifier; and the business strategy rule set matching the current scenario; execute the business strategy rules one by one, and judge according to the execution result. If any rule is violated, the review result is output as "not passed", and the violating strategy identifier is output as the reason for failure; if all rules are satisfied, the result is output as "passed".

[0115] The third review module 95 is used to receive the preliminary review results, the review results of multiple jurors, and the model weight values ​​corresponding to each juror's model; determine whether there is any juror whose review result is unsuccessful and whose corresponding juror model is preset as a key role model; if so, directly generate the final review result of unsuccessful; if not, calculate the weighted comprehensive score based on the review results of each juror and their corresponding model weight values; compare the weighted comprehensive score with the comprehensive threshold; if the weighted comprehensive score is greater than the comprehensive threshold, generate the final review result of passing; if the weighted comprehensive score is not greater than the comprehensive threshold, generate the final review result of failing.

[0116] The third review module 95 is used to summarize the reasons for failure of all jurors' review results that were judged to be unsuccessful if the final review result is unsuccessful; and to send the summarized reasons for failure along with the final review result to the user.

[0117] Optionally, it also includes a training module 96 for collecting historical review samples, which include: sample information, manually annotated final review results, and reasons for failure; training each juror model to be trained using the historical review samples to obtain an initial juror model; determining the initial model weight values; training the judge model to be trained using the initial jury results of the initial juror model and the corresponding initial model weight values, as well as the corresponding historical review samples, to obtain a judge model and generate a weight mapping table.

[0118] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0119] The entity executing the information verification method in the specific embodiments of this disclosure may be an electronic device such as a server (including a local server or a cloud computing platform).

[0120] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the information verification method of any of the embodiments described above.

[0121] Figure 10 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure.

[0122] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0123] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0124] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0125] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0126] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0127] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0132] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0133] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. An information verification method, characterized in that, The method includes: Obtain information pending review; The information to be reviewed is reviewed using the lead review model to obtain preliminary review results; Based on the information to be reviewed or the preliminary review results, determine the model weight values ​​corresponding to each juror's model in the jury; The information to be reviewed and the attribute information related to the juror role are input into the corresponding juror model to obtain multiple juror review results; The preliminary review results, the juror review results, and the model weight values ​​are input into the judge model to generate the final review results for the information to be reviewed.

2. The information verification method according to claim 1, characterized in that, The jury model includes at least one of the following: rule engine jury, model jury, user behavior jury, and business strategy jury.

3. The information verification method according to claim 2, characterized in that, When the juror model is the juror of the rule engine. The process involves inputting the information to be reviewed and the relevant attribute information of the juror roles into the corresponding juror model to obtain multiple juror review results, including: The information to be reviewed is matched with a preset rule base, wherein the rule base includes blacklists, whitelists, regular expressions, and hard threshold rules; Based on the matching results, if the information to be reviewed matches any rejection rule in the blacklist or whitelist, the review result will be output as "not approved", and the corresponding failure reason identifier will be output. If the rejection rule is not met, the audit result will be "passed".

4. The information verification method according to claim 2, characterized in that, When the jury model is the model juror The process involves inputting the information to be reviewed and the relevant attribute information of the juror roles into the corresponding juror model to obtain multiple juror review results, including: Feature extraction is performed on the information to be reviewed to generate a feature vector; The feature vector is input into the juror model to obtain the risk score predicted by the model. Based on the comparison between the risk score and the preset scoring threshold, the audit result is determined and output. If the audit result is unsuccessful, the key features that led to the high-risk score will be output as the reason for failure.

5. The information verification method according to claim 2, characterized in that, When the jury model is the user behavior jury. The process involves inputting the information to be reviewed and the relevant attribute information of the juror roles into the corresponding juror model to obtain multiple juror review results, including: Based on the user identifier in the information to be reviewed, obtain the historical behavior sequence corresponding to the user identifier; Using anomaly detection algorithms and the historical behavior sequences, analyze the behavioral anomaly scores of the user corresponding to the user identifier; Based on the comparison between the abnormal behavior score and the behavior threshold, the audit result is determined and output: if the abnormal behavior score exceeds the behavior threshold, the audit result is "failed" and the abnormal behavior type is output as the reason for failure. If the abnormal behavior score does not exceed the behavior threshold, the audit result is "passed".

6. The information verification method according to claim 2, characterized in that, When the jury model is a business strategy jury. The process involves inputting the information to be reviewed and the relevant attribute information of the juror roles into the corresponding juror model to obtain multiple juror review results, including: Obtain the current business scenario; and the set of business strategy rules that match the current business scenario; Execute the business strategy rules in the aforementioned business strategy rule set one by one; The review result is judged based on the execution result. If any of the business strategy rules are violated, the review result will be "not approved" and the violating strategy identifier will be output as the reason for failure. If all the business strategy rules are satisfied, the output will be "pass".

7. The information verification method according to claim 1, characterized in that, The step of inputting the preliminary review results, the juror review results, and the model weight values ​​into the judge model to generate the final review results for the information to be reviewed includes: Receive the preliminary review results, the review results of multiple jurors, and the model weight values ​​corresponding to each juror model; Determine whether there exists any jury member whose review result is "fail" and whose corresponding jury member model is preset as a key role model; If so, then a final review result of failure will be generated directly; If not, then a weighted composite score is calculated based on the review results of each juror and their corresponding model weight values; The weighted composite score is compared with a comprehensive threshold. If the weighted composite score is greater than the comprehensive threshold, the final review result is "passed". If the weighted composite score is not greater than the composite threshold, the final review result will be "not approved".

8. The information verification method according to claim 1, characterized in that, Based on the information to be reviewed or the preliminary review results, determine the model weight values ​​corresponding to each juror's model in the jury, including: Based on the preset weight values ​​carried by the information to be reviewed, determine the model weight values ​​corresponding to each juror's model in the jury; or, Obtain the semantic tags carried in the preliminary review results; wherein, the semantic tags are used to characterize the data type or business scenario of the information to be reviewed; Obtain a preset weight mapping table, wherein the weight mapping table contains the correspondence between the semantic labels and the model weight values ​​of each juror model; The model weight value of each juror model that matches the information to be reviewed is obtained by querying the weight mapping table based on the semantic tags.

9. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images