Distributed risk control auditing method based on multi-modal content and related equipment

Abstract

The invention belongs to the field of risk control, and discloses a distributed risk control auditing method based on multi-modal content and related equipment, which can improve the accuracy and efficiency of risk identification. The method comprises the following steps: receiving original heterogeneous data, wherein the original heterogeneous data comprises text data, picture data, video data and two-dimensional code data; performing structured packaging on the original heterogeneous data of each business scene to obtain an auditing detail list with a uniform data structure; grouping the auditing details in the auditing detail list to obtain an execution task set; issuing the execution task to a working thread in a thread pool for parallel processing, and processing the execution task according to the task type of the execution task to obtain feature data; performing risk identification on the feature data based on a preset risk identification model to obtain a risk analysis result; and determining a final auditing result according to the feature data and the risk analysis result.

Description

Technical Field

[0001] This application relates to the field of risk control, and in particular to a distributed risk control audit method and related equipment based on multimodal content. Background Technology

[0002] With the surge in business volume on e-commerce platforms specializing in local services and home furnishing after-sales, these platforms need to process massive amounts of user-generated content, including order notifications, feedback messages, and user-posted text, images, and videos. This diverse content comes from complex sources and may contain illegal information and fraudulent activities, posing significant challenges to content security and risk control for these platforms.

[0003] Existing risk control methods are mainly based on static mechanisms of rules and lists, and adopt separate review pipelines for different modalities of content. For example, text, image and video content usually use independent detection processes and strategies, and for the detection of duplicate orders or fraudulent order behavior, simple keyword matching or hash value comparison methods are often used.

[0004] However, the fragmented multimodal audit pipeline makes it difficult to unify risk control strategies and aggregate results. Simple keyword matching or hash value comparison cannot capture the hidden characteristics of fraudulent groups across accounts or devices, which leads to a large number of misjudgments and omissions, resulting in low overall audit efficiency and low accuracy. Summary of the Invention

[0005] This application provides a distributed risk control audit method and related equipment based on multimodal content, which can improve the accuracy and efficiency of risk identification.

[0006] The first aspect of this application proposes a distributed risk control and review method based on multimodal content, including: Receive raw heterogeneous data from each of N business scenarios, where the raw heterogeneous data includes text data, image data, video data, and QR code data, and N is an integer greater than or equal to 1. The original heterogeneous data of each business scenario is structurally encapsulated to obtain a unified audit detail list. The audit details in the audit details list are grouped to obtain the execution task set; The execution tasks in the execution task set are distributed to worker threads in the thread pool for parallel processing, and the execution tasks in the execution task set are processed according to the task type of each execution task to obtain the feature data of each execution task. Risk identification is performed on the feature data of each execution task based on a preset risk identification model to obtain the risk analysis results of each execution task; The final audit result is determined based on the characteristic data of each execution task and the risk analysis results of each execution task.

[0007] In one possible design, processing the execution tasks in the execution task set according to the task type of each execution task to obtain the feature data of each execution task includes: When there is a first execution task of text type in the execution task set, determine the feature vector of the order text content corresponding to the first execution task, and determine the feature data of the first execution task based on the feature vector; When there is a second execution task of image type in the execution task set, the QR code image corresponding to the second execution task is processed to obtain the feature data of the second execution task; When there is a third execution task of video type in the execution task set, extract the key frame list of the target video corresponding to the third execution task, and determine the key frame list as the feature data of the third execution task.

[0008] In one possible design, determining the feature data of the first execution task based on the feature vector includes: Based on the feature vectors, an approximate nearest neighbor search is performed in the vector database according to a preset time window to obtain an initial order list; Starting with the first order in the initial order list, the graph database is traversed to determine the set of historical orders in the graph database that are associated with each order in the initial order list. Extract the structured association evidence corresponding to each historical order in the historical order set; The initial order list is aggregated with the historical order set to obtain a similar order list; The list of similar orders and the structured association evidence are identified as the feature data of the first execution task.

[0009] In one possible design, processing the QR code image corresponding to the second execution task to obtain the feature data of the second execution task includes: Identify the QR code content of the QR code image corresponding to the second execution task; The QR code content is decoded using a cascaded decoding strategy to obtain the webpage link URL; The main domain name is extracted from the URL and compared with a dynamically maintained domain whitelist to obtain the risk level of the main domain name; The QR code content, the main domain name, and the risk level are determined as the feature data of the second execution task.

[0010] In one possible design, the extraction of the keyframe list of the target video corresponding to the third execution task includes: Step 1: Determine the first video frame in the target video as the initial keyframe and record the timestamp of the initial keyframe; Step 2: Take the second video frame in the target video as the current video frame, and determine the grayscale histogram difference between the current video frame and the previous video frame. Step 3: When the difference in the grayscale histogram is greater than a preset threshold, determine whether the time interval between the second timestamp of the current video frame and the first timestamp corresponding to the previous keyframe of the current video reaches a first preset value. Step 4: When the time interval reaches the first preset value, the second video frame is determined as a key frame; Step 5: When the time interval does not reach the first preset value, iteratively execute steps 2 to 4 until the preset termination condition is reached to obtain the keyframe list.

[0011] In one possible design, the preset termination conditions include the completion of processing of every video frame of the target video, the total processing time of the video frames in the target video reaching a second preset value, or the number of processed video frames in the target video reaching a third preset value.

[0012] In one possible design, determining the final audit result based on the characteristic data of each execution task and the risk analysis results of each execution task includes: The risk analysis results are then analyzed to obtain analytical data. The parsed data, the feature data, and the underlying context corresponding to the original heterogeneous data are fused to obtain fused data; The fused data is mapped to standard fields in a database table to obtain the final audit result.

[0013] In one possible design, before processing the execution tasks in the execution task set according to the task type of each execution task to obtain the feature data of each execution task, the method further includes: For each of the execution tasks, an idempotent key is set, and the idempotent key carries an expiration time; Generate a tracing identifier and inject the tracing identifier into the context of each worker thread in the thread pool.

[0014] A second aspect of this application provides a distributed risk control and review device based on multimodal content, comprising: The receiving module is used to receive the raw heterogeneous data of each of the N business scenarios. The raw heterogeneous data includes text data, image data, video data and QR code data, where N is an integer greater than or equal to 1. The encapsulation module is used to encapsulate the original heterogeneous data of each business scenario in a structured manner to obtain a unified audit detail list. The grouping module is used to group the various audit details in the audit detail list to obtain a set of execution tasks; The dispatch module is used to dispatch the execution tasks in the execution task set to the worker threads in the thread pool for parallel processing. A parallel processing module is used to process the execution tasks in the execution task set according to the task type of each execution task, so as to obtain the feature data of each execution task; The risk identification module is used to identify risks in the feature data of each execution task based on a preset risk identification model, so as to obtain the risk analysis results of each execution task; The determination module is used to determine the final audit result based on the characteristic data of each execution task and the risk analysis results of each execution task.

[0015] A third aspect of this application proposes a distributed risk control and review device based on multimodal content, comprising: processor; Memory, used to store computer programs; When the processor executes the computer program, it implements the distributed risk control audit method based on multimodal content as described in any of the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the distributed risk control audit method based on multimodal content as described in any of the above embodiments.

[0017] The fifth aspect of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the distributed risk control audit method based on multimodal content as described in any of the above embodiments.

[0018] In the embodiments provided in this application, multimodal heterogeneous data from N business scenarios are uniformly encapsulated into a standardized audit detail list, and a grouped parallel processing mechanism combined with thread pool scheduling is adopted to achieve high throughput and low latency asynchronous risk control audit of massive data. Based on a preset risk identification model, in-depth analysis is performed on the multidimensional feature data extracted in parallel, and the feature data and risk analysis results are integrated for comprehensive decision-making. While improving the accuracy of risk identification, the system's computing load and operation and maintenance costs are effectively reduced through distributed architecture and resource optimization strategies. Attached Figure Description

[0019] Figure 1 A system architecture diagram of a distributed risk control and review system based on multimodal content provided in an embodiment of this application; Figure 2 A flowchart illustrating a distributed risk control review method based on multimodal content, provided as an embodiment of this application; Figure 3 A schematic diagram of the structure of a distributed risk control review device based on multimodal content provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application; The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when an element is “connected” or “coupled” to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein may include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0022] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0023] Please see Figure 1 , Figure 1 The system architecture diagram of the distributed risk control and review system based on multimodal content provided in the embodiments of this application includes: 1. Multimodal input layer: This layer receives raw input data and encapsulates it into a unified audit detail structure. It outputs the data using a standardized data structure (such as a standard table format). The fields of this standardized data structure include: relation_id, main_relation_id, content_type, qr_content, account_id, account_level, risk_scene, etc. The raw data includes text, images, videos, QR codes, and audio. 2. The review pipeline is used to call worker threads in the thread pool to perform the following operations in parallel according to the content type of each review detail in the review detail structure: GraphRAG enhanced retrieval (including Milvus vectors + Neo4j graph, user / device / content association mining), order similarity detection (including feature extraction, vectorization, time window filtering and threshold binning), video keyframe analysis, QR code decoding, Promp version cache management, and multimodal risk identification (generating models and adding risk labels).

[0024] 3. Result aggregation and tag structuring: All audit results are aggregated and stored in the database, and distributed through a message queue. A unified trace identifier (trace_id) is used throughout the process, forming a complete closed loop from data input and risk identification to result output and governance.

[0025] The following description focuses on the distributed risk control and auditing method based on multimodal content provided in this application, from the perspective of a distributed risk control and auditing device based on multimodal content. This distributed risk control and auditing device based on multimodal content can be a server or a service unit within a server, without any specific limitation. For the sake of simplicity, the following description uses a server as an example to illustrate the distributed risk control and auditing device based on multimodal content.

[0026] Reference Figure 2, Figure 2 A flowchart illustrating the distributed risk control review method based on multimodal content provided in this application embodiment, the method comprising: 201. Receive the raw heterogeneous data of each of the N business scenarios.

[0027] In this embodiment, the server can receive raw heterogeneous data from each of N different business scenarios (e.g., order cancellation). The raw heterogeneous data includes text data, image data, video data, and QR code data, where N is an integer greater than or equal to 1.

[0028] 202. Structure and encapsulate the original heterogeneous data for each business scenario to obtain a unified audit details list.

[0029] In this embodiment, after obtaining the original heterogeneous data, the server can encapsulate the original heterogeneous data of each business scenario into a standardized data structure, namely the audit details list. The audit details list includes key metadata such as relation_id (current detail ID), main_relation_id (main order ID), content_type (content type), qr_content (original content of QR code), account_id / type / level (account context information), risk_scene (risk scene label), and timestamp.

[0030] 203. Group the audit details in the audit details list to obtain the set of execution tasks.

[0031] In this embodiment, after determining the list of review details, the server groups the review details in the list according to main_relation_id or other strategies to form a set of execution tasks that can be executed in parallel. This set of execution tasks includes, but is not limited to, text tasks, image tasks, and video tasks.

[0032] 204. Distribute the tasks in the task set to worker threads in the thread pool for parallel processing.

[0033] In this embodiment, after the server obtains the set of tasks to be executed, it can use ThreadPoolExecutor to distribute the tasks in the set to multiple worker threads in the thread pool and call multiple worker threads to process the tasks in parallel.

[0034] It should be noted that before each task is executed, an idempotent key with an expiration time (the key is relation_id, etc.) is set using the Redis SETNX command. This prevents the same content from being reviewed multiple times due to retries, message duplication, etc. At the same time, the unique trace_id generated by the HTTP request entry is injected into the context of each worker thread to ensure log traceability in a distributed environment.

[0035] 205. Process the execution tasks in the execution task set according to the task type of each execution task to obtain the feature data of each execution task.

[0036] In this embodiment, the server can first determine the task type of each execution task, which includes text, image, and video types. Then, based on the task type of each execution task, the server processes the execution tasks in the execution task set to obtain the feature data of each execution task. Specifically: When there is a first execution task of text type in the execution task set, determine the feature vector of the order text content corresponding to the first execution task, and determine the feature data of the first execution task based on the feature vector; When there is a second execution task of image type in the execution task set, the QR code image corresponding to the second execution task is processed to obtain the feature data of the second execution task; When there is a third execution task of video type in the execution task set, extract the key frame list of the target video corresponding to the third execution task, and determine the key frame list as the feature data of the third execution task.

[0037] The following sections provide a detailed explanation of the execution task processing within the task set, categorized by task type: image, text, and video. 1. When the task type is text, perform task processing: When there is a first execution task of text type in the execution task set, determine the feature vector of the order text content corresponding to the first execution task, and determine the feature data of the first execution task based on the feature vector.

[0038] When the server determines that there is a first execution task of text type in the execution task set, it identifies the order text content corresponding to the first execution task, extracts feature text from the order text content, and then cleans, removes abnormal characters, normalizes whitespace characters, and uses a large model (such as Qwen Embeddings) to transform the feature text into a high-dimensional vector to obtain the feature vector of the order text content corresponding to the first execution task. Afterwards, the server performs an approximate nearest neighbor search in the vector database based on the feature vectors according to a preset time window to obtain the initial order list. That is, an approximate nearest neighbor search is performed in the vector database (Milvus). At the same time, a time window filter is applied during the search to query only the order vectors within the most recent period (such as N days) to exclude the interference of outdated historical data. Starting with the first order in the initial order list, the system traverses the graph database to determine the set of historical orders in the graph database that are associated with each order in the initial order list. In other words, the server starts with the order node in the historical order set and performs a multi-hop traversal in the graph database to find all historical order sets that are associated with List_A at the user or device level. This set is defined here as the historical order set List_B. The historical orders in List_B may be completely dissimilar to the currently retrieved order in terms of content, but they are closely connected through entity relationships. The server extracts structured association evidence corresponding to each historical order from the historical order set. This structured association evidence could be, for example, "5 orders with similar content were found. These orders were posted by 3 different users, but these 3 users all logged in from the same device (Device_ID: XYZ) in the past 7 days." The server aggregates the initial order list with the historical order set to obtain a similar order list. The server then combines List_A (orders with similar content) and List_B (orders with related entities) to form a final suspicious order set, namely the similar order list. The similar order list and the structured association evidence are identified as the feature data for the first execution task.

[0039] It should be noted that the server constructs and maintains a heterogeneous graph in a graph database (such as Neo4j). This heterogeneous graph contains three types of nodes and the relationships between them, with the node types as follows: Order node: Attributes include order_id, content_hash, timestamp, etc.

[0040] User node: Attributes include user_id, account_level, registration_time, etc.

[0041] Device node: Attributes include device_id, device_fingerprint, last_login_ip, etc.

[0042] The relation types are as follows: (User)-[SUBMITTED]->(Order): The user submitted an order.

[0043] (User)-[LOGGED_IN_FROM]->(Device): The user has logged in from a certain device.

[0044] (Order)-[CREATED_BY_DEVICE]->(Device): The order was created by a device (optional, more direct relationship).

[0045] The server uses vector retrieval to find a set of historical orders, List_A, that are similar in content to the currently reviewed orders in the initial order list. The server will then determine the risk level of List_A, such as whether the number of orders in List_A exceeds a certain order threshold or whether the highest similarity score is higher than a certain score threshold. If it exceeds the threshold, it is initially deemed suspicious and GraphRAG is triggered to conduct an in-depth investigation. If it does not exceed the threshold, it directly enters the standard risk control process to save resources.

[0046] 2. Handling tasks when the task type is image: When there is a second execution task of image type in the execution task set, the QR code image corresponding to the second execution task is processed to obtain the feature data of the second execution task.

[0047] The server identifies the QR code content of the QR code image corresponding to the second execution task and uses a cascading decoding strategy to decode the QR code content to obtain the webpage link URL. The cascading decoding strategy attempts sequentially to the reserved local database, the third-party "Caoliao" API, and the WeChat API. If the local database successfully decodes the QR code content, it returns the result directly without needing to continue calling the third-party "Caoliao" API and the WeChat API for decompression. If the cascading decoding strategy fails to decode the QR code content, an empty result is recorded to avoid blocking the overall review process due to the inability to decode individual QR codes. After extracting the URL from the QR code content, the server can further extract the main domain from the complete URL. For example, it can extract the main domain (xxx) from the URL "https: / / www.xxx.html" in the QR code content and compare the main domain with a dynamically maintained domain whitelist to determine the risk level of the main domain. If the main domain is not on the domain whitelist, the URL will be marked as a risky external link. If the main domain is on the domain whitelist, it is considered safe. Finally, the QR code content, the main domain, and the risk level are determined as the feature data for the second execution task.

[0048] 3. Handling tasks when the task type is video: When there is a third execution task of video type in the execution task set, extract the keyframe list of the target video corresponding to the third execution task, and determine the keyframe list as the feature data of the third execution task, as follows: Step 1: Determine the first video frame in the target video as the initial keyframe and record the timestamp of the initial keyframe; Step 2: Take the second video frame in the target video as the current video frame, and determine the grayscale histogram difference between the current video frame and the previous video frame. Step 3: When the difference in grayscale histograms is greater than a preset threshold, determine whether the time interval between the second timestamp of the current video frame and the first timestamp corresponding to the previous keyframe of the current video reaches a first preset value. Step 4: When the time interval reaches the first preset value, the second video frame is determined as the keyframe; Step 5: When the interval duration does not reach the first preset value, iteratively execute steps 2 to 4 until the preset termination condition is met, and obtain the keyframe list.

[0049] That is, for the target video, the frame sequence of the target video is F1, F2, F3, ..., Fn. The first video frame (F1) of the target video is unconditionally added to the keyframe list and set as the reference frame used for comparison. This is because the first frame usually contains initial important information and records the timestamp (T_ref) of the reference frame. The server sequentially reads each video frame (Fi) from the target video, starting from the next video frame. For each video frame Fi, the following calculations and judgments are performed: A. Calculate the pixel difference between the currently reviewed video frame and the baseline frame: Both Fi and the current reference frame are converted into grayscale histograms, and the difference in pixel intensity between the two video frames is calculated to obtain the grayscale histogram difference, which is usually obtained by using the mean absolute difference (MAD) or mean squared error (MSE); then it is determined whether the grayscale histogram difference is greater than a preset difference threshold. B. Calculate the time interval between the currently reviewed video frame and the baseline frame: Get the timestamp (Ti) of the currently reviewed video frame Fi, calculate the time interval between the currently reviewed video frame and the reference frame, and determine whether the time interval is greater than the preset time interval; Only when both conditions A and B are met will the currently reviewed video frame be defined as a keyframe and added to the keyframe list. If the currently reviewed video frame Fi is determined to be a keyframe, then Fi is set as the new reference frame, and its timestamp is updated to T_ref. Subsequent difference comparisons will use it as the new reference frame until a preset termination condition is met, resulting in a list of keyframes corresponding to the target video. Thus, each keyframe in this list represents a significant point of change in the target video, efficiently summarizing the main content of the video with a minimal number of images.

[0050] It should be noted that the preset termination conditions include the completion of processing of every video frame of the target video, the total processing time of video frames in the target video reaching a second preset value, or the number of processed video frames in the target video reaching a third preset value. In other words, in addition to terminating the process after each video frame in the target video has been reviewed, the server will monitor two global resource indicators in real time when extracting and reviewing keyframes of the target video: 1. Processed Frames Count: The total number of video frames that have passed the above-mentioned keyframe determination, as recorded by the server; 2. Total processing time (Elapsed Time): The time elapsed between the start of processing the first video frame of the target video and the current time.

[0051] The server compares these two metrics with preset circuit breaker thresholds (maximum number of frames processed and maximum processing time). When either global resource metric reaches the threshold, the circuit breaker mechanism is immediately triggered. The server interrupts the extraction of keyframes from the target video, stops reading and analyzing subsequent video frames of the target video, and releases the resources occupied by the video decoder, temporary files, etc. At the same time, the list of keyframes that has been generated is taken as the final result of this analysis.

[0052] It should be noted that during a circuit breaker event, the server records the relevant information corresponding to the circuit breaker event in the logs and audit clues (the record will include the trace_id), which helps operations and maintenance personnel identify malicious attack videos or optimize system parameters. At the same time, after the circuit breaker is triggered, the server uses the Auto-Cleanup mechanism to completely delete all temporary files generated by the target video (including extracted keyframe images) to prevent disk space leakage.

[0053] 206. Based on the preset risk identification model, risk identification is performed on the feature data of each execution task to obtain the risk analysis results of each execution task.

[0054] In this embodiment, after obtaining the feature data of each execution task, the server uses the feature data as context to call the preset risk identification model for final comprehensive risk assessment, thereby obtaining the risk analysis result of each execution task. Specifically, the server obtains the latest version of the Prompt template based on content_type and risk_scene, and dynamically populates the Prompt template with feature data (such as similar order information, QR code content, etc.) as variables, forming a customized inference instruction containing rich context. This instruction is then input into the preset risk identification model, and outputs an unstructured or semi-structured risk analysis result in natural language or JSON format, such as: risk_type: "specific risk type", sensitive_words: ["keyword 1", "keyword 2"], is_risk: true, etc.

[0055] 207. Determine the final audit result based on the characteristic data of each task and the risk analysis results of each task.

[0056] In this embodiment, after obtaining the risk analysis results and feature data for each task, the server parses the unstructured output of the risk analysis results, extracts explicit risk tags, keywords, etc., and executes a simple decision logic. The parsed risk tags and keywords are then fused with the feature data, and the fused result is mapped to a standard field in the database table to obtain the final audit result. The output format of this standard field is as follows: ai_audit_status: 0 (Passed) / 1 (Rejected) / 2 (Requires manual review) ai_audit_category: "Risk Category 1" / "Risk Category 2" / "Risk Category 3" / "Risk Category 4" ai_sensitive_words: "word A, word B" ai_sensitive_urls / domain: "http: / / xxx.com,xxx_domain.com" ai_qr_content: "Decoded content" similar_orders_json: [{"order_no": "123", "score": 0.95}, ...] (This field stores the results of vector retrieval and GraphRAG) It should be noted that before writing the final audit result to the database, the server checks the idempotent key (key: relation_id) in Redis to prevent duplicate writes due to network retries or other reasons. It also uses a batch insert operation to write the master record (ai_risk_audit_record) and the detailed record (ai_risk_audit_record_detail) in a single database event, ensuring data consistency. Thus, a complete and structured audit record is persisted to the database.

[0057] It should be noted that after obtaining the final review result and writing it into the database, the server creates a publication message. This publication message contains the most essential information, such as relation_id, main_relation_id, and ai_audit_status. The publication message is then published to a specific Topic (such as AI_AUDIT_RESULT_TOPIC) via a message queue (such as RocketMQ).

[0058] When sending message notifications, asynchronous message notifications are used. This facilitates the decoupling of the audit system from downstream business systems. Even if the downstream system is temporarily unavailable, the published message will not accumulate in the message queue (MQ).

[0059] It should also be noted that during the aforementioned risk control process, the server monitors all anomalies occurring in real time during the risk review process. Specifically, the server uses a built-in classification logic to perform real-time judgment on all captured anomalies (such as those originating from external service calls, like AI models or QR code APIs), categorizing them into two types: 1. Retryable exceptions: Judgment criteria: The exception type is network timeout, connection error, HTTP 5xx server internal error, third-party service rate limiting, etc. These exceptions are usually temporary, and retrying may succeed. This exception is unrelated to the specific content of the request and is caused by infrastructure or temporary load issues.

[0060] 2. Non-retry exception: Content violation error: This is identified by matching error codes returned by the pre-screening risk identification model or keywords in the response content (such as "violation", "sensitive", "insecure", etc.). For example, invalid request parameters or authentication failure. This exception is determined by the legality of the request content or itself, and retrying cannot change the result.

[0061] The server can trigger different execution strategies based on the classification results of the anomalies: For retryable exceptions, an exponential backoff retry strategy is adopted. For example, after the first failure, wait for a short time (e.g., 1 second) to retry; after the second failure, wait for a longer time (e.g., 2 seconds); and so on, until the preset maximum number of retries (e.g., 3 times) is reached; if the retry still fails, a system exception is thrown upwards, marking this audit task as a system error.

[0062] For non-retryable exceptions, a specific non-retryable exception instruction (such as DataInspectionException) is thrown directly. This exception instruction will be caught by the upper layer and a result of "review failed" or "content violation" will be generated immediately, skipping all subsequent retry logic.

[0063] It should also be noted that the server has set up an ai_risk_audit_record table in the system database, which maintains two status fields in parallel: 1. ai_audit_status: The initial status of AI auditing.

[0064] 2. manual_audit_status: The final status of manual audit (initially empty, updated after manual processing).

[0065] Specifically, the server performs paginated queries through query interfaces such as get_audit_status_diff_data(). It filters out all records with inconsistent statuses using the SQL condition WHERE ai_audit_status != manual_audit_status AND manual_audit_status IS NOT NULL, obtaining a list of discrepancies. This list includes detailed audit content, AI results, and manual results for front-end display and statistical analysis. The server also provides a correct_audit_status() interface, authorizing users to manually write the correct status (manual_audit_status) and classification information back to the database record for a discrepancy record. This operation overwrites the initial AI result, serving as the final ruling for that entry.

[0066] The server accurately calculates key performance indicators for AI review based on the corrected data, such as: Accuracy = (Number of correct AI tests) / (Total number of corrections) Recall rate = Number of violations successfully identified by AI / (Total number of violations confirmed by humans) Meanwhile, the server can also record discrepancies. For erroneous cases identified by the model, these erroneous cases can be directly used for subsequent model training, or the wording and structure of the Prompt can be adjusted to optimize the dynamic threshold or time window size of the order similarity detection process.

[0067] In summary, it can be seen that the embodiments provided in this application unify the multimodal heterogeneous data from N business scenarios into a standardized audit detail list, and adopt a grouped parallel processing mechanism combined with thread pool scheduling to achieve high throughput and low latency asynchronous risk control audit of massive data; based on a preset risk identification model, the multidimensional feature data extracted in parallel is deeply analyzed, and the feature data and risk analysis results are integrated for comprehensive decision-making. While improving the accuracy of risk identification, the distributed architecture and resource optimization strategies effectively reduce the system's computing load and operation and maintenance costs.

[0068] The embodiments of this application have been described above from the perspective of a distributed risk control and review method based on multimodal content. The embodiments of this application will now be described below from the perspective of a distributed risk control and review device based on multimodal content.

[0069] Please see Figure 3 , Figure 3 A virtual structure diagram of a distributed risk control review device based on multimodal content is provided for this application. The distributed risk control review device 300 based on multimodal content includes: The receiving module 301 is used to receive the original heterogeneous data of each of the N business scenarios. The original heterogeneous data includes text data, image data, video data and QR code data, where N is an integer greater than or equal to 1. The encapsulation module 302 is used to encapsulate the original heterogeneous data of each business scenario in a structured manner to obtain a unified audit detail list. Grouping module 303 is used to group the various audit details in the audit detail list to obtain a set of execution tasks; The dispatch module 304 is used to dispatch the execution tasks in the execution task set to the worker threads in the thread pool for parallel processing. The parallel processing module 305 is used to process the execution tasks in the execution task set according to the task type of each execution task, so as to obtain the feature data of each execution task. The risk identification module 306 is used to identify risks in the feature data of each execution task based on a preset risk identification model, so as to obtain the risk analysis results of each execution task; The determination module 307 is used to determine the final audit result based on the characteristic data of each execution task and the risk analysis results of each execution task.

[0070] In one possible design, the parallel processing module 305 is specifically used for: When there is a first execution task of text type in the execution task set, determine the feature vector of the order text content corresponding to the first execution task, and determine the feature data of the first execution task based on the feature vector; When there is a second execution task of image type in the execution task set, the QR code image corresponding to the second execution task is processed to obtain the feature data of the second execution task; When there is a third execution task of video type in the execution task set, extract the key frame list of the target video corresponding to the third execution task, and determine the key frame list as the feature data of the third execution task.

[0071] In one possible design, the parallel processing module 305 determines the feature data of the first execution task based on the feature vector, including: Based on the feature vectors, an approximate nearest neighbor search is performed in the vector database according to a preset time window to obtain an initial order list; Starting with the first order in the initial order list, the graph database is traversed to determine the set of historical orders in the graph database that are associated with each order in the initial order list. Extract the structured association evidence corresponding to each historical order in the historical order set; The initial order list is aggregated with the historical order set to obtain a similar order list; The list of similar orders and the structured association evidence are identified as the feature data of the first execution task.

[0072] In one possible design, the parallel processing module 305 processes the QR code image corresponding to the second execution task to obtain the feature data of the second execution task, including: Identify the QR code content of the QR code image corresponding to the second execution task; The QR code content is decoded using a cascaded decoding strategy to obtain the webpage link URL; The main domain name is extracted from the URL and compared with a dynamically maintained domain whitelist to obtain the risk level of the main domain name; The QR code content, the main domain name, and the risk level are determined as the feature data of the second execution task.

[0073] In one possible design, the parallel processing module 305 extracts the list of keyframes of the target video corresponding to the third execution task, including: Step 1: Determine the first video frame in the target video as the initial keyframe and record the timestamp of the initial keyframe; Step 2: Take the second video frame in the target video as the current video frame, and determine the grayscale histogram difference between the current video frame and the previous video frame. Step 3: When the difference in the grayscale histogram is greater than a preset threshold, determine whether the time interval between the second timestamp of the current video frame and the first timestamp corresponding to the previous keyframe of the current video reaches a first preset value. Step 4: When the time interval reaches the first preset value, the second video frame is determined as a key frame; Step 5: When the time interval does not reach the first preset value, iteratively execute steps 2 to 4 until the preset termination condition is reached to obtain the keyframe list.

[0074] In one possible design, the preset termination conditions include the completion of processing of every video frame of the target video, the total processing time of the video frames in the target video reaching a second preset value, or the number of processed video frames in the target video reaching a third preset value.

[0075] In one possible design, the determining module 307 is specifically used for: The risk analysis results are then analyzed to obtain analytical data. The parsed data, the feature data, and the underlying context corresponding to the original heterogeneous data are fused to obtain fused data; The fused data is mapped to standard fields in a database table to obtain the final audit result.

[0076] In one possible design, the parallel processing module 305 is further configured to: For each of the execution tasks, an idempotent key is set, and the idempotent key carries an expiration time; Generate a tracing identifier and inject the tracing identifier into the context of each worker thread in the thread pool.

[0077] Reference Figure 4 This application also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 4As shown. The computer device includes a processor, memory, network interface, and database connected via a bus. The processor is designed to provide computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores operations, computer programs, and the database. The memory provides an environment for the operation and execution of the computer programs in the non-volatile storage medium. The database stores data such as a distributed risk control audit method based on multimodal content. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps of a distributed risk control audit method based on multimodal content, which includes: Receive raw heterogeneous data from each of N business scenarios, where the raw heterogeneous data includes text data, image data, video data, and QR code data, and N is an integer greater than or equal to 1. The original heterogeneous data of each business scenario is structurally encapsulated to obtain a unified audit detail list. The audit details in the audit details list are grouped to obtain the execution task set; The execution tasks in the execution task set are distributed to worker threads in the thread pool for parallel processing, and the execution tasks in the execution task set are processed according to the task type of each execution task to obtain the feature data of each execution task. Risk identification is performed on the feature data of each execution task based on a preset risk identification model to obtain the risk analysis results of each execution task; The final audit result is determined based on the characteristic data of each execution task and the risk analysis results of each execution task.

[0078] An embodiment of this application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements a distributed risk control review method based on multimodal content. The distributed risk control review method based on multimodal content includes: Receive raw heterogeneous data from each of N business scenarios, where the raw heterogeneous data includes text data, image data, video data, and QR code data, and N is an integer greater than or equal to 1. The original heterogeneous data of each business scenario is structurally encapsulated to obtain a unified audit detail list. The audit details in the audit details list are grouped to obtain the execution task set; The execution tasks in the execution task set are distributed to worker threads in the thread pool for parallel processing, and the execution tasks in the execution task set are processed according to the task type of each execution task to obtain the feature data of each execution task. Risk identification is performed on the feature data of each execution task based on a preset risk identification model to obtain the risk analysis results of each execution task; The final audit result is determined based on the characteristic data of each execution task and the risk analysis results of each execution task.

[0079] In some possible implementations, various aspects of the methods provided in this application can also be implemented as a program product, which can be implemented using any combination of one or more readable media. The program product includes program code that, when run on a computer device, is configured to cause the computer device to perform the steps of the methods described above according to various exemplary embodiments of this application. The computer device can execute the distributed risk control audit method based on multimodal content described in the embodiments of this application, which includes: Receive raw heterogeneous data from each of N business scenarios, where the raw heterogeneous data includes text data, image data, video data, and QR code data, and N is an integer greater than or equal to 1. The original heterogeneous data of each business scenario is structurally encapsulated to obtain a unified audit detail list. The audit details in the audit details list are grouped to obtain the execution task set; The execution tasks in the execution task set are distributed to worker threads in the thread pool for parallel processing, and the execution tasks in the execution task set are processed according to the task type of each execution task to obtain the feature data of each execution task. Risk identification is performed on the feature data of each execution task based on a preset risk identification model to obtain the risk analysis results of each execution task; The final audit result is determined based on the characteristic data of each execution task and the risk analysis results of each execution task.

[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0081] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A distributed risk control and review method based on multimodal content, characterized in that, include: Receive raw heterogeneous data from each of N business scenarios, where the raw heterogeneous data includes text data, image data, video data, and QR code data, and N is an integer greater than or equal to 1. The original heterogeneous data of each business scenario is structurally encapsulated to obtain a unified audit detail list. The audit details in the audit details list are grouped to obtain the execution task set; The execution tasks in the execution task set are distributed to worker threads in the thread pool for parallel processing, and the execution tasks in the execution task set are processed according to the task type of each execution task to obtain the feature data of each execution task. Risk identification is performed on the feature data of each execution task based on a preset risk identification model to obtain the risk analysis results of each execution task; The final audit result is determined based on the characteristic data of each execution task and the risk analysis results of each execution task.

2. The method according to claim 1, characterized in that, The step of processing the execution tasks in the execution task set according to the task type of each execution task to obtain the feature data of each execution task includes: When there is a first execution task of text type in the execution task set, the feature vector of the order text content corresponding to the first execution task is determined, and the feature data of the first execution task is determined according to the feature vector; When there is a second execution task of image type in the execution task set, the QR code image corresponding to the second execution task is processed to obtain the feature data of the second execution task; When there is a third execution task of video type in the execution task set, extract the key frame list of the target video corresponding to the third execution task, and determine the key frame list as the feature data of the third execution task.

3. The method according to claim 2, characterized in that, The step of determining the feature data of the first execution task based on the feature vector includes: Based on the feature vectors, an approximate nearest neighbor search is performed in the vector database according to a preset time window to obtain an initial order list; Starting with the first order in the initial order list, the graph database is traversed to determine the set of historical orders in the graph database that are associated with each order in the initial order list. Extract the structured association evidence corresponding to each historical order in the historical order set; The initial order list is aggregated with the historical order set to obtain a similar order list; The list of similar orders and the structured association evidence are identified as the feature data of the first execution task.

4. The method according to claim 2, characterized in that, The process of processing the QR code image corresponding to the second execution task to obtain the feature data of the second execution task includes: Identify the QR code content of the QR code image corresponding to the second execution task; The QR code content is decoded using a cascaded decoding strategy to obtain the webpage link URL; The main domain name is extracted from the URL and compared with a dynamically maintained domain whitelist to obtain the risk level of the main domain name; The QR code content, the main domain name, and the risk level are determined as the feature data of the second execution task.

5. The method according to claim 2, characterized in that, The keyframe list extracted from the target video corresponding to the third execution task includes: Step 1: Determine the first video frame in the target video as the initial keyframe and record the timestamp of the initial keyframe; Step 2: Take the second video frame in the target video as the current video frame, and determine the grayscale histogram difference between the current video frame and the previous video frame. Step 3: When the difference in the grayscale histogram is greater than a preset threshold, determine whether the time interval between the second timestamp of the current video frame and the first timestamp corresponding to the previous keyframe of the current video reaches a first preset value. Step 4: When the time interval reaches the first preset value, the second video frame is determined as a key frame; Step 5: When the time interval does not reach the first preset value, iteratively execute steps 2 to 4 until the preset termination condition is reached to obtain the keyframe list.

6. The method according to claim 5, characterized in that, The preset termination conditions include: each video frame of the target video has been processed; the total processing time of the video frames in the target video reaches a second preset value; or the number of video frames processed in the target video reaches a third preset value.

7. The method according to any one of claims 1 to 6, characterized in that, The determination of the final audit result based on the characteristic data of each execution task and the risk analysis results of each execution task includes: The risk analysis results are then analyzed to obtain analytical data. The parsed data, the feature data, and the underlying context corresponding to the original heterogeneous data are fused to obtain fused data; The fused data is mapped to standard fields in a database table to obtain the final audit result.

8. The method according to any one of claims 1 to 6, characterized in that, Before processing the execution tasks in the execution task set according to the task type of each execution task to obtain the feature data of each execution task, the method further includes: For each of the execution tasks, an idempotent key is set, and the idempotent key carries an expiration time; Generate a tracing identifier and inject the tracing identifier into the context of each worker thread in the thread pool.

9. A distributed risk control and review device based on multimodal content, characterized in that, include: The receiving module is used to receive the raw heterogeneous data of each of the N business scenarios. The raw heterogeneous data includes text data, image data, video data and QR code data, where N is an integer greater than or equal to 1. The encapsulation module is used to encapsulate the original heterogeneous data of each business scenario in a structured manner to obtain a unified audit detail list. The grouping module is used to group the various audit details in the audit detail list to obtain a set of execution tasks; The distribution module distributes the execution tasks from the set of execution tasks to the worker threads in the thread pool for parallel processing; A parallel processing module is used to process the execution tasks in the execution task set according to the task type of each execution task, so as to obtain the feature data of each execution task; The risk identification module is used to identify risks in the feature data of each execution task based on a preset risk identification model, so as to obtain the risk analysis results of each execution task; The determination module is used to determine the final audit result based on the characteristic data of each execution task and the risk analysis results of each execution task.

10. A distributed risk control and review device based on multimodal content, characterized in that, include: processor; Memory, used to store computer programs; Wherein, when the processor executes the computer program, it implements the distributed risk control audit method based on multimodal content as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the distributed risk control and auditing method based on multimodal content as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the distributed risk control audit method based on multimodal content as described in any one of claims 1 to 8.

Images