A content moderation system for resource migration driven by review results
By designing a resource migration content review system driven by review results, the problem of the disconnect between review conclusions and resource storage locations in existing technologies is solved. It realizes automated and highly reliable linkage of resources between different storage areas, ensures the physical execution capability and consistency of resource access control, and eliminates the security risk of premature exposure.
Patent Information
- Application Number
- CN202611301173.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-26
- Publication Date
- 2026-11-17
AI Technical Summary
In existing content moderation systems, the review results are separated from the resource storage location, resulting in the inability to physically isolate illegal resources and the inability to restore services to compliant resources in a timely manner. This lack of physical access control capabilities leads to security vulnerabilities exposed before review and inconsistencies in cross-system status.
Design a resource migration content review system driven by review results. The system uses a resource isolation storage module to store the resources to be reviewed in physically isolated public storage areas, review intermediate storage areas, and evidence storage areas. The review engine module generates review results, the resource migration control module automatically generates migration task records based on the results, and the migration execution module asynchronously executes resource migration operations to ensure that the physical storage location of the resources to be reviewed is consistent with the review results.
It achieves automated and highly reliable linkage between review conclusions and the physical storage location of resources, solves the problem of resource access control failure, ensures the eventual consistency of resources across different storage areas, eliminates the security risk of premature exposure, and improves the physical execution capability of content risk control measures.
Smart Images

Figure CN123116299A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of content security technology, and in particular to a resource migration content review system driven by review results. Background Technology
[0002] Existing content moderation systems typically employ a third-party moderation service to identify violations in multimodal content and automatically freeze or block it based on the results. For example, Tencent Cloud Object Storage supports automatic review of uploaded files and automatic freezing of violating data, while Huawei Cloud's video moderation solution automatically triggers the content moderation service and replaces and dumps violating videos after they are received in the OBS bucket. However, these operations are usually one-off, stateless calls, lacking persistent records and reliability assurance mechanisms for migration tasks.
[0003] The fundamental flaw in existing technologies lies in the disconnect between the review conclusion and the physical storage location of the resources. The review system only outputs a business status field (such as PASS / BLOCK), while whether the actual storage location of the resources is publicly accessible or has been moved to a secure area is independent of the review process. When cross-system calls (such as notifying object storage services to perform migrations) fail due to network timeouts, service degradation, or lost callbacks, an inconsistency arises where the review status has been updated but the resource files remain in the public storage area, rendering content risk control measures ineffective. Furthermore, resources are typically still publicly accessible during the review period, presenting a security vulnerability that is exposed before review, and lacking physical-level access control capabilities.
[0004] While Chinese patent CN122112943A proposes a solution, it focuses on the intelligentization of the review process and does not address the linkage between review results and the storage layer. Existing broadcast control systems, although they divide the material lifecycle into stages and support migration management, rely on preset time strategies rather than real-time review results, making them unsuitable for high-frequency UGC review scenarios. In summary, existing technologies lack a technical solution that can achieve atomic, highly reliable, and automated linkage between review conclusions and resource physical storage locations to fundamentally solve the technical problem of resource access control failure across system states. Summary of the Invention
[0005] In view of this, embodiments of the present invention provide a resource migration content review system driven by review results, in order to solve the technical problem in the prior art where the review conclusion and the resource storage location are separated, resulting in the inability to physically isolate illegal resources and the inability to restore services to compliant resources in a timely manner, thus causing content risk control measures to remain at the level of logical judgment and lack physical execution capability.
[0006] This invention provides a resource migration content review system driven by review results, comprising: The resource isolation storage module is configured to store the resources to be reviewed in physically isolated public storage areas, intermediate review storage areas, and evidence storage areas according to their review status.
[0007] The review engine module is configured to receive the resource to be reviewed, perform content review on the resource to be reviewed, and generate review results;
[0008] The resource migration control module is configured to automatically generate and persist migration task records from the current storage area to the target storage area based on the audit results output by the audit engine module. The migration task records include at least the resource identifier, the source storage area identifier, the target storage area identifier, and the migration status.
[0009] The migration execution module is configured to asynchronously execute the resource migration operation corresponding to the migration task record, physically move the resource to be reviewed from the current storage area to the target storage area, and update the migration status according to the migration execution result; The resource isolation storage module updates the storage location information of the resource to be reviewed based on the migration execution result of the resource migration control module, so that the physical storage location of the resource to be reviewed is consistent with the review result.
[0010] The resource migration content review system driven by review results provided by this invention has the following beneficial effects: In this invention, by setting up physically isolated public storage areas, intermediate review storage areas, and evidence storage areas, and directly converting review results into physical migration instructions for resources between different storage areas, the review conclusions are given the ability to drive physical actions at the storage layer. This fundamentally solves the technical problems in the prior art, such as the separation between review results and the physical storage location of resources, the inability to physically isolate illegal content, the existence of "pre-review" security vulnerabilities in resources during the review period, and the failure of risk control measures due to inconsistencies in cross-system states. At the same time, by solidifying migration intentions in the form of persistent task records and combining them with transaction callbacks and multiple idempotent controls, the eventual consistency of cross-system states between the review system and the object storage service is guaranteed, realizing the leap from logical judgment to physical execution in content risk control. Attached Figure Description
[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.
[0012] Figure 1 This is a flowchart of a resource migration content review system driven by review results. Detailed Implementation
[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the orientation or positional relationships indicated by terms such as center, up, down, front, back, left, right, vertical, horizontal, top, bottom, inside, and outside are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the term "includes" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, the inclusion of a defined element by a statement does not preclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Where there is no conflict, embodiments of the present invention and the various features thereof can be combined with each other, all of which are within the scope of protection of the present invention.
[0014] Example 1 Please see Figure 1 This invention provides a resource migration content review system driven by review results, comprising: The resource isolation storage module is configured to store the resources to be reviewed in physically isolated public storage areas, intermediate review storage areas, and evidence storage areas according to their review status. Specifically, the resource isolation storage module pre-creates three logically independent and physically isolated storage buckets for each business unit within the Object Storage Service (OSS). The public storage area stores normal resources that have passed the review (PASS), and this area is publicly accessible to external business systems and ordinary users. The intermediate review storage area stores resources that have been submitted for review but have not yet received a final review conclusion. This area is configured to be inaccessible externally, preventing business units or users from accessing these resources during the review process, thus achieving access isolation of resources during the review at the physical storage level. The evidence storage area stores non-compliant resources that have failed the review (BLOCK). This area is configured to be accessible only to compliance auditors and is used to solidify evidence of violations. Each resource initially falls into the public storage area upon submission for review, and then migrates between different storage areas depending on the stage of the review.
[0015] The review engine module is configured to receive the resource to be reviewed, perform content review on the resource to be reviewed, and generate review results; Specifically, the review engine module provides a unified review interface. After receiving resources submitted by business users for review, it dynamically selects an automated review strategy based on the business configuration. The automated review strategy is dynamically routed to the corresponding third-party review service provider, such as Tencent Tianyu, Huawei Cloud, and Shumei, through the strategy factory (ApprovalStrategyFactory) based on the business user identifier and resource type. The review engine module defines a unified review strategy interface (ApprovalStrategy), and each vendor implements an independent strategy adapter class (such as TencentSendApprovalContentConsumer, HuaweiSendApprovalContentConsumer, and ShumeiSendApprovalConsumer) to handle the differences between vendors in API protocols, authentication methods, request parameter structures, response formats, and callback mechanisms. The audit engine module also supports serial and parallel strategies: Under the serial strategy, multiple suppliers are called sequentially according to a preset order. The call is terminated when the first supplier returns a BLOCK result, and that result is taken as the final audit result. Under the parallel strategy, multiple suppliers are called concurrently, and all returned results are merged according to a preset priority (BLOCK has a higher priority than PASS, and PASS has a higher priority than REVIEW) before the final audit result is output. After the audit is completed, this module outputs the audit result (PASS, BLOCK, or REVIEW) to the resource migration control module.
[0016] The resource migration control module is configured to automatically generate and persist migration task records from the current storage area to the target storage area based on the audit results output by the audit engine module. The migration task records include at least the resource identifier, the source storage area identifier, the target storage area identifier, and the migration status. Specifically, the resource migration control module maintains a migration task data table (pc_approval_resource_transfer_task). Each migration task record contains at least the following fields: resource identifier (resource_id), resource group identifier (group_id), source storage type (source_bucket), target storage type (target_bucket), migration status (status, values include INIT pending execution, EXEC executing, SUCC successful, FAIL failed), resource content MD5 value (md5_content), retry count (retry_times), and execution time (exec_time). After the audit engine module outputs the audit result, the resource migration control module automatically determines the migration path based on the audit result value: if the audit result is PASS, the target storage area is the public storage area; if the audit result is BLOCK, the target storage area is the evidence storage area. Subsequently, the module synchronously creates migration task records in the business transaction context and writes them to the data table. After successful writing, the migration message is triggered by the callback mechanism after transaction commit (TransactionSynchronizationManager.registerTransactionCallback) to ensure the atomicity between the persistence of the audit results and the creation of the migration task records. That is, the migration task will not be created if the transaction commit fails, and the migration task will definitely be created if the transaction commit succeeds.
[0017] The migration execution module is configured to asynchronously execute the resource migration operation corresponding to the migration task record, physically move the resource to be reviewed from the current storage area to the target storage area, and update the migration status according to the migration execution result; Specifically, the migration execution module acts as a message consumer (ResourceTransferDelayDomainConsumer and ResourceTransferFinishConsumer), listening to the message queue related to resource migration. Upon receiving a migration task message, the module reads the source storage address and target storage type from the migration task record, and calls the object storage service's migration interface (such as createRevertMiddleToNormalBucketTask or createMiddleToIllegalBucketTask) to physically migrate the resource files from the source storage to the target storage. After the migration operation is complete, the object storage service returns the migration execution result. The module updates the migration status in the migration task record based on this result (updated to SUCC on success, FAIL on failure, and the number of retries is incremented). Furthermore, the module uses an XXL-Job scheduled task (ResourceIllegalTransferTask) to scan the migration task data table for records with an INIT status and whose execution time has expired. This serves as a fallback mechanism to compensate for any exceptions and ensure that the migration task is ultimately executed.
[0018] The resource isolation storage module updates the storage location information of the resource to be reviewed based on the migration execution result of the resource migration control module, so that the physical storage location of the resource to be reviewed is consistent with the review result. Furthermore, the resource isolation storage module updates the storage location information of the resource to be reviewed based on the migration execution result of the resource migration control module. Specifically, after the migration execution module successfully completes the physical migration of the resource from the intermediate storage area to the target storage area, the resource isolation storage module synchronously updates the resource_location field in the resource details table (pc_approval_task_resource) to the type of the target storage area after migration (NORMAL, MIDDLE, or ILLEGAL).
[0019] In this embodiment, through the above mechanism, the system ensures that the physical storage location of the resource to be reviewed is ultimately consistent with the review result: the resource that passes the review is physically located in the public storage area and can be accessed normally, the resource that fails the review is physically located in the evidence storage area and is sealed and isolated, and the resource under review is physically located in the intermediate storage area and is not visible to the outside world, thereby realizing the automated and highly reliable driving of the physical storage location of the resource by the review result.
[0020] Furthermore, the resource migration control module controls the migration of the resource to be audited according to at least one of the following migration paths: In response to the submission of the resource to be reviewed, the resource to be reviewed is moved from the public storage area to the review intermediate storage area, which is an isolated storage space that is not accessible to the outside world; Specifically, when a business submits resources for review to the system, the resources are first written to the public storage area. After the resource submission message enters the ApprovalTaskConsumer, the migration task creation unit of the resource migration control module detects the submission trigger event and immediately creates a migration task record from the public storage area to the intermediate review storage area. The target storage area type (target_bucket) of this record is marked as MIDDLE, and the migration status (status) is marked as INIT. The migration execution module asynchronously executes this migration task, physically moving the resource files from the public storage area to the intermediate review storage area. The intermediate review storage area is configured with private read and write permissions in the object storage service, and its access policy (Bucket Policy) is set to deny all unauthorized access. Furthermore, the resource URLs in this storage area are not publicly exposed, thus achieving access isolation of resources during the review period at the physical storage level. Before the resource obtains the final review conclusion, no business party or ordinary user can access the resource through any means, fundamentally eliminating the security risk of "exposure before review". After the migration is completed, the resource migration control module updates the resource_location field in the resource details table to MIDDLE, indicating that the resource is currently located in the audit intermediate storage area.
[0021] In response to the audit result being approved, the resource to be audited is rolled back from the audit intermediate storage area to the public storage area; Specifically, when the audit engine module outputs an audit result of PASS (audit passed), the resource migration control module first calls the `cancelExistingDelayTransferTasks` method to query the migration task data table for an existing delayed migration task record in the INIT state under the same resource identifier (such as the delayed migration task from the intermediate storage area to the evidence storage area, which will be described later). If it exists, the delayed migration task is marked as cancelled to avoid conflicts between subsequent tasks and the current rollback task. Subsequently, the resource migration control module creates a rollback migration task record from the audit intermediate storage area to the public storage area in the transaction context. The target storage area type of this record is marked as NORMAL. The migration execution module triggers the execution of this rollback migration task through message-driven or timed scanning, calling the interface provided by the object storage service for rolling back from the intermediate storage area to the public storage area (such as `createRevertMiddleToNormalBucketTask`) to physically migrate the resource files from the audit intermediate storage area back to the public storage area. After the migration is successful, the resource isolation storage module will update the resource_location field to NORMAL, and the resource will be restored to an externally accessible state. Business systems can obtain the resource normally through the resource URL in the public storage area.
[0022] In response to the audit result being that the audit failed, the resource to be audited is moved from the audit intermediate storage area to the evidence storage area.
[0023] Specifically, when the audit engine module outputs an audit result of BLOCK (audit failed), the resource migration control module first queries the current storage location of the resource. If the resource is currently located in the audit intermediate storage area, the resource migration control module directly creates a migration task record from the audit intermediate storage area to the evidence storage area. If the resource is currently located in the public storage area (e.g., in specific business scenarios where the resource is not audited directly in the intermediate storage area), the resource migration control module first creates a migration task record from the public storage area to the audit intermediate storage area, and after that task is completed, it creates another migration task record from the audit intermediate storage area to the evidence storage area. This ensures that the non-compliant resource is isolated in the intermediate storage area before being solidified to the evidence storage area. The target storage area type of this migration task record is marked as ILLEGAL. When the migration execution module performs the migration operation, it calls the migration interface provided by the object storage service from the intermediate storage area to the evidence storage area (such as createMiddleToIllegalBucketTask) to physically move the resource file to the evidence storage area. The evidence storage area is configured as a private storage space in the object storage service that can only be accessed by compliance auditors. After the migration is successful, the resource isolation storage module updates the resource_location field to ILLEGAL. The resource is permanently sealed as evidence of violations, and business parties and ordinary users can no longer access the resource, thereby achieving physical-level irreversible isolation of the violation content.
[0024] In this embodiment, the automated linkage of three migration paths enables a complete closed loop throughout the entire review lifecycle: resources are submitted to the intermediate isolation area, returned to the public area after passing the review, and entered into the evidence area after failing the review. This ensures that the physical storage location of the resources to be reviewed is always synchronized with the review status.
[0025] Furthermore, the resource migration control module includes: The migration task creation unit is configured to synchronously create the migration task record in the transaction context used to persist the audit result or the submission record of the resource to be audited, and trigger the sending of the migration message through the callback mechanism after the transaction is committed, so as to ensure the atomicity between the persistence of the audit result and the creation of the migration task record.
[0026] Specifically, in scenarios where resource migration needs to be triggered due to a PASS or BLOCK audit result, when the business service method (such as the audit result aggregation method of ApprovalResourceGroupDomainBizServiceImpl) is executed, the Spring framework has already started a database transaction for the execution of this method. Within this transaction context, the service method first performs the persistence operation of the audit result, that is, updates the result and complete_time fields in the resource details table (pc_approval_task_resource), and writes the audit record to the audit record table (pc_approval_resource_check_record). Within the same database transaction, the migration task creation unit then constructs a migration task record object. This record object includes fields such as resource identifier (resource_id), source storage type (the storage area where the current resource is located, obtained by querying the resource_location field), target storage type (determined based on the approval result: PASS corresponds to NORMAL, BLOCK corresponds to ILLEGAL), migration status (initially set to INIT), and the MD5 value of the resource content (md5_content). This record is then inserted into the migration task data table (pc_approval_resource_transfer_task) through the data access layer (ApprovalResourceTransferTaskMapper). Because the above-mentioned approval result persistence operation and migration task record insertion operation are within the same database transaction, this transaction is committed uniformly by Spring's transaction manager when the business method completes normally, ensuring that the approval result and the migration task record are either written successfully simultaneously or rolled back simultaneously, maintaining data consistency between the two at all times.
[0027] Building upon this, the migration task creation unit triggers the sending of migration messages through a callback mechanism following transaction commit. Specifically, the migration task creation unit utilizes the TransactionSynchronizationManager provided by the Spring framework, registering a transaction synchronization callback object (TransactionSynchronizationAdapter) with it before the database transaction is committed, and overriding its afterCommit method. In the afterCommit method, the migration task creation unit constructs a migration message body (containing the migration task record ID and target storage type) and sends the migration message to the resource migration-related message queue (topic: delay_pc, tag: new_ugcResourceTransferDelay) via the message sending service (AliyunRocketMQDomainService). This callback mechanism ensures that migration messages are only actually sent after the underlying database transaction has been successfully committed, thus avoiding the inconsistencies of "transaction rollback but message sent" or "transaction commit successful but message not sent." Specifically, if a database transaction is rolled back due to an anomaly, the afterCommit callback will not be executed, the migration message will not be sent, the system state will remain in the "unapproved" state, and no "orphan" migration task without associated valid transaction will be generated; if the database transaction is successfully committed, the afterCommit callback will be automatically triggered, the migration message will be reliably sent to the message queue, the migration execution module will consume the message from the queue and perform the actual physical relocation of resources.
[0028] It should be noted that the aforementioned transaction callback mechanism also applies to the submission migration scenario when a resource awaiting review is submitted for review, moving from the public storage area to the intermediate review storage area. In this scenario, the migration task creation unit synchronously creates the submission migration task record within the same transaction context as the persistent resource submission record (i.e., writing to the resource details table pc_approval_task_resource), and sends the migration message through the afterCommit callback, thus ensuring the atomicity between the persistence of the submission record and the creation of the submission migration task. Through this mechanism, the system achieves transactional consistency between the persistence of the review result and the creation of the migration task record in a distributed environment, avoiding consistency issues that may arise from cross-system calls or asynchronous message sending, where the state has been updated but the migration has not been executed.
[0029] Then:
[0030] Furthermore, the resource migration control module also includes: The idempotent control unit is configured to check whether there is an existing migration task record under the same resource ID that is in the pending execution state before creating a new migration task record. If it exists, the existing migration task record is cancelled and a new migration task record is created. And / or, add a distributed lock based on the content feature value of the resource to be reviewed to prevent the same resource from being concurrently executed for migration operations; and / or, set a unique constraint in the database table corresponding to the migration task record to prevent the same resource from being repeatedly created for migration task records under the same migration type.
[0031] Furthermore, the idempotent control unit ensures the idempotency of the migration operation through a three-layer defense mechanism, preventing the same resource from being repeatedly migrated or causing conflicts due to concurrent sending audits, repeated callbacks, or business retries.
[0032] The first layer of defense is the cancellation of pre-approval tasks. Before creating a new migration task record, the idempotent control unit first queries the migration task data table (pc_approval_resource_transfer_task) to determine if there is an existing migration task record in a pending state (status: INIT) under the same resource identifier (resource_id) or the same resource content MD5 value (md5_content). If it exists, the idempotent control unit updates the status of the existing migration task record to CANCEL, and then creates a new migration task record. This mechanism is typically used in approved rollback scenarios: when a resource is rolled back from the intermediate storage area to the public storage area, the idempotent control unit first checks if there is a delayed migration task from the intermediate storage area to the evidence storage area that has not yet been executed. If so, it cancels the task and then creates a rollback migration task, thereby preventing subsequent delayed migration tasks from mistakenly migrating approved resources to the evidence storage area.
[0033] The second layer of defense is a distributed lock. Before creating a migration task record, the idempotent control unit generates a distributed lock key (e.g., transfer_resource_md5_{md5Content}) based on the content feature value of the resource to be reviewed (i.e., the MD5 value of the resource file), and acquires the distributed lock through RedissonClient (tryLock method, 5-second timeout, 15-second holding time). Since the MD5 value of the same resource file is the same regardless of which business party or user submits it, only one request can successfully acquire the lock and continue to execute the subsequent migration task creation and resource migration operations when all migration requests for the same resource are competing for the same lock. The remaining requests will retry after the lock is released or skip directly. This distributed lock mechanism effectively prevents the same resource from being migrated simultaneously by multiple threads or service instances in high-concurrency scenarios (such as a popular video being uploaded by millions of users at the same time), thereby avoiding problems such as duplicate migration of resource files, duplicate creation of migration task records, and overwriting and disorder of the final storage location of resources.
[0034] The third layer of defense is a unique index constraint in the database. At the database design level of the migration task data table (pc_approval_resource_transfer_task), the idempotent control unit achieves database-level duplicate creation interception by establishing a unique composite index (uk_group_resource_type, containing three fields: group_id, resource_id, and restore_type). When two migration task records have the same resource group ID, resource ID, and migration type (NORMAL, MIDDLE, or ILLEGAL), the database will trigger a unique key conflict exception when inserting the second record, thus directly preventing the persistence of duplicate migration task records. This mechanism acts as a fallback defense for the distributed lock, ensuring that migration task records under the same migration type are not duplicated even if the distributed lock fails due to network jitter or Redis failure.
[0035] In this embodiment, through the coordinated operation of three layers of idempotent defense mechanisms, the pre-task cancellation addresses task conflicts at the business logic level, the distributed lock addresses operation competition in concurrent scenarios, and the database unique index addresses duplicate writes in extreme cases. The system establishes a defense-in-depth system at three levels: the business logic layer, the distributed coordination layer, and the database storage layer, ensuring that migration tasks are not duplicated or executed in conflict under any abnormal scenario, thereby guaranteeing the final correctness of resource migration operations and data consistency.
[0036] Then:
[0037] Furthermore, the migration execution module includes: The delayed scheduling unit is configured to determine a delayed execution time based on the violation level of the audit result or the business entity identifier to which the resource to be audited belongs, and to set the delayed execution time for the migration task record, so that the migration operation is triggered and executed after the delayed execution time arrives; and The retry unit is configured to automatically re-execute the migration operation according to a preset retry strategy when the migration operation fails, until it succeeds or the maximum number of retries is reached.
[0038] Specifically, the delayed scheduling unit is configured to determine the delayed execution time based on the violation level of the audit result or the business entity identifier to which the resource to be audited belongs, and to set the delayed execution time for the migration task record, so that the migration operation is triggered and executed after the delayed execution time arrives. Specifically, for scenarios where resources with an audit result of BLOCK are migrated from the audit intermediate storage area to the evidence storage area, the delayed scheduling unit does not immediately execute the migration, but instead configures a delayed execution time for the migration task. The delayed scheduling unit determines this delayed execution time by reading the middlebucket retention hours configured by business entity dimension in the Apollo configuration center. For example, the UGC business is configured with 360 hours (15 days), and the product review business is configured with 72 hours (3 days). Different business entities can configure this independently based on their own compliance requirements and appeal window periods. Furthermore, the delay scheduling unit determines the delay execution time based on the violation level of the audit results: In the machine audit result details, if the violation confidence score exceeds a preset threshold (e.g., 0.95), the violation level of the resource is marked as SERIOUS (serious violation), and the delay scheduling unit configures a longer delay execution time (e.g., 90 days) for resources with serious violations to meet the long-term requirements of regulatory review of appeals and evidence preservation for serious violations; if the violation confidence score is below the threshold, the violation level is marked as NORMAL (normal violation), and a shorter delay execution time (e.g., 7 days) is configured to accelerate resource turnover in the intermediate storage area and reduce storage costs. After the delay execution time is determined, the delay scheduling unit writes this time into the exec_time field in the migration task record. When triggering a migration, the migration execution module determines whether to execute the migration operation by checking if the current time has reached `exec_time`. For migration tasks whose execution time has not yet arrived, the module does not execute them but reserves them for a later scheduling cycle. For migration tasks whose execution time has arrived, the module uses the delayed message mechanism of the message queue (AliyunRocketMQDomainService.sendDelayMessage) to deliver the migration message to the delayed queue. This delayed message is only retrieved by the consumer and triggers the actual physical resource migration operation after the set delay time has elapsed. Through this mechanism, the system achieves fine-grained and differentiated control over the timing of migration operation execution, providing a window for appeals for business users while effectively controlling storage costs. It should be noted that for scenarios where approved resources are rolled back from the intermediate review storage area to the public storage area, the delayed scheduling unit does not set a delay time but immediately triggers the rollback migration operation to ensure that compliant resources can resume external services in a timely manner.
[0039] The retry unit is configured to automatically re-execute the migration operation according to a preset retry strategy when the migration operation fails, until it succeeds or the maximum number of retries is reached. Specifically, when the migration execution module calls the migration interface of the object storage service, if the migration operation fails due to anomalies such as network timeout, service degradation, or connection pool exhaustion, the retry unit will capture the anomaly and increment the retry count field (retry_times) in the migration task record by 1, while keeping the migration status at INIT for subsequent retry scheduling. The retry unit automatically re-executes the migration operation according to an exponential backoff strategy: the first retry is executed immediately after failure, the second retry waits 2 seconds, the third waits 4 seconds, and so on, with the waiting time for each retry increasing in powers of 2. The maximum number of retries for the retry unit is configurable (default 10 times), and the migration interface of the object storage service is called again for each retry. If the migration succeeds in a retry, the retry unit updates the status of the migration task record to SUCC, records retry_times as the actual number of retries, and the retry process terminates. If the migration operation still fails after 10 retries (the maximum retry count), the retry unit updates the migration task's status to FAIL and sends a DingTalk alert message to the developers via the exception alert service (ExceptionDingTalkServiceImpl). The alert message includes key information such as resource identifier, source storage area, target storage area, failure error code, and the number of retries, allowing operations personnel to manually intervene for troubleshooting and recovery. Furthermore, the retry unit works in conjunction with a scheduled scanning compensation mechanism: for migration tasks with a FAIL status that have not exceeded the maximum retry count, the XXL-Job scheduled task will continue scanning and compensating in subsequent cycles, ensuring that the migration task has the potential to be executed under any abnormal scenario. Through this retry mechanism, the system significantly improves the success rate and reliability of cross-system migration operations, avoiding situations requiring manual intervention after migration failures due to temporary faults such as momentary network jitter.
[0040] Furthermore, the migration execution module includes: A callback consumption unit is configured to receive a migration completion callback message from the object storage service and update the migration status of the migration task record based on the callback message; and The timed scanning unit is configured to scan migration task records that are in a pending state and have reached their execution time through a timed task, and trigger the migration execution module to perform the corresponding migration operation.
[0041] Specifically, the callback consumption unit is configured to receive migration completion callback messages from the object storage service and update the migration status of the migration task record based on the callback messages. Specifically, when the object storage service (OSS) performs a physical migration operation of resource files, for asynchronous archive-type migration operations (such as migrating large files from an intermediate storage area to an evidence storage area), the OSS will proactively push an archive completion callback message to the system after the file migration is completed. This callback message is transmitted through a message queue (topic: normal_bs-storage-service, tag: archiveFinish). The callback consumption unit, acting as a message consumer (ResourceTransferFinishConsumer), listens to this message queue and pulls the callback message, parsing the resource identifier (such as resourceId or md5Content), migration task ID (taskId), and migration execution result (success or failure) from the callback message body. The callback consumption unit queries the migration task data table (pc_approval_resource_transfer_task) based on the parsed migration task ID. If the callback message indicates successful migration, the status of the migration task record is updated to SUCC, and the new_url field is updated to the new resource address carried in the callback message. If the callback message indicates migration failure, the status is updated to FAIL, and the error code and error description from the callback message are written to the failure reason field (fail_reason) of the migration task record for subsequent manual investigation. It should be noted that for synchronous migration operations, the callback consumption unit also supports updating the migration status by receiving the execution results synchronously returned by the object storage service; the two methods complement each other. Through the callback consumption unit, the system can promptly perceive and confirm the final execution result of remote resource migration operations, achieving reliable closed-loop confirmation of cross-system migration status.
[0042] The scheduled scanning unit is configured to scan migration task records that are in a pending execution state and have reached their execution time via scheduled tasks, triggering the migration execution module to execute the corresponding migration operation. Specifically, the scheduled scanning unit registers a scheduled task (ResourceIllegalTransferTask) through the XXL-Job distributed task scheduling platform. This scheduled task runs according to a preset scheduling cycle (e.g., once per minute). During each scheduling, the scheduled scanning unit queries the migration task data table (pc_approval_resource_transfer_task). The query conditions are: migration tasks with a migration status of INIT (pending execution) and an execution time (exec_time) less than or equal to the current system timestamp, i.e., migration tasks that have reached their execution time but have not yet been executed. For each migration task record found, the scheduled scanning unit calls the resource migration operation of the migration execution module one by one to perform the actual physical relocation of resources. In the normal process, the migration execution module triggers migration operations through the delayed message mechanism of the message queue. The scheduled scanning unit acts as a fallback mechanism, proactively scanning and triggering timed-out migration tasks when delayed messages fail to be consumed in time due to message queue backlog, abnormal consumer restarts, network partitions, or other reasons. This ensures that the migration operation is executed within a certain time window after the delay period. Furthermore, the scheduled scanning unit also has sharding capabilities. By configuring the batch size (default 100 messages) and maximum scan rounds (default 1000 rounds), it can process large amounts of data in batches, avoiding excessive database connection resource consumption by a single scan task. Through the collaborative pull mode (object storage proactive callback confirmation) and push mode (scheduled task proactive scanning and execution) of the callback consumption unit and the scheduled scanning unit, the system constructs a closed-loop confirmation and compensation execution dual guarantee mechanism for the migration execution module. Under the dual action of the message-driven path and the scheduled scanning path, migration tasks can ultimately be executed and receive status confirmation in any abnormal scenario, effectively ensuring the eventual consistency of resource migration operations.
[0043] Furthermore, the audit engine module includes: The multi-strategy machine review adaptation unit is configured to define a unified review strategy interface and implement corresponding strategy adapters for different third-party review service providers. The strategy adapters include at least adaptation logic for encapsulating the differences in request protocols, authentication methods, and response formats of each third-party review service provider. Specifically, the multi-strategy machine-based audit adaptation unit defines a unified audit strategy interface (ApprovalStrategy) within the system. This interface includes at least the sendApproval method (used to submit audit requests to third-party audit service providers) and the getModerationResult method (used to retrieve audit results). Based on this, the multi-strategy machine-based audit adaptation unit implements independent strategy adapter classes for each third-party audit service provider: For Tencent Tianyu service, it implements the TencentSendApprovalContentConsumer class, which encapsulates Tencent Tianyu's asynchronous callback plus active query dual-mode call logic, its unique signature authentication algorithm (such as TC3-HMAC-SHA256), and JSON format response parsing logic; for Huawei Cloud service, it implements the HuaweiSendApprovalContentConsumer class, which encapsulates Huawei Cloud's synchronous return mode call logic, AK / SK signature authentication method, and XML format response parsing logic; and for Shumei service, it implements the ShumeiSendApprovalConsumer class, which encapsulates the Shumei SDK's calling method and its independent error code system mapping logic. Each strategy adapter class internally converts the unique request parameter structures of third-party review service providers (such as image URL transmission methods, audio file upload methods, and callback parameter structures) into a unified internal request model. It also maps the violation tag systems returned by various providers (such as different encodings and names for tags like pornography, terrorism, and political sensitivity) to unified result enumeration values (PASS, BLOCK, REVIEW) within the system. This allows upper-layer business logic to obtain standardized review results without needing to be aware of the interface differences between various providers. Through this adapter pattern, the system achieves unified access for multiple providers. When adding a new provider, only one strategy adapter class needs to be added and its unified interface implemented; no modification to the core review process code is required.
[0044] The policy routing unit is configured to dynamically select a serial policy or a parallel policy based on business configuration information. The serial policy calls multiple third-party audit service providers in a preset order until the final audit result is obtained. The parallel policy calls multiple third-party audit service providers concurrently and merges the audit results of each provider according to a preset priority.
[0045] Specifically, the policy routing unit internally maintains a policy factory (ApprovalStrategyFactory), which registers serial policy instances (ThirdSerialApprovalStrategy) and parallel policy instances (ThirdParallelApprovalStrategy) through a ConcurrentHashMap. After receiving the business entity identifier (businessId) and the approval scenario information, the policy routing unit queries the policy configuration corresponding to that business entity from the business approval policy table (pc_business_approval_strategy). This configuration is stored in JSON format in the content field, containing parameters such as the supplier list (supplierList), the execution mode (executionMode, with a value of SERIAL or PARALLEL), and the order in which each supplier is called. If the strategy is configured as a serial strategy, the strategy routing unit retrieves a ThirdSerialApprovalStrategy instance from the strategy factory and executes it. This serial strategy calls the audit interface of the first supplier in the configured supplier order. If the first supplier returns a BLOCK result, the call is immediately terminated and the BLOCK is returned as the final audit result, without calling subsequent suppliers. If the first supplier returns a PASS or REVIEW result, the call continues to the second supplier, and so on, until a supplier returns a BLOCK or all suppliers have been called. If the strategy is configured as a parallel strategy, the strategy routing unit retrieves a ThirdParallelApprovalStrategy instance from the strategy factory and executes it. This parallel strategy concurrently calls the audit interfaces of all configured suppliers through a thread pool (such as CompletableFuture). After all suppliers return audit results, the parallel strategy merges the audit results of each supplier according to a preset priority (BLOCK has the highest priority, followed by PASS, and REVIEW has the lowest priority), and finally takes the result with the highest priority as the final audit result and returns it. Through the above-mentioned strategy routing mechanism, the system supports refined machine review strategy configuration based on business dimensions. High-risk businesses can be configured with parallel strategies to improve review accuracy, while low-risk businesses can be configured with serial strategies to reduce review costs, thus achieving a flexible balance between review accuracy and call costs.
[0046] Furthermore, the audit engine module also includes: The rate limiting and circuit breaking unit is configured to establish a token bucket rate limiter based on the dimensions of third-party audit service providers and resource types. When a call request to any third-party audit service provider triggers rate limiting, the submission message corresponding to the resource to be audited is resubmitted to the delay queue for retry. Specifically, the rate limiting and circuit breaking unit establishes an independent token bucket rate limiter instance for each combination of third-party review service provider and each resource type (text, image, audio, video). All rate limiter instances are centrally managed through a rate limiter registry (RateLimiterDomainContact.STRATEGY). The key format of the rate limiter is pc-ugc-{resourceType}-{operation}-{supplier}, for example, pc-ugc-image-push-tencent represents the image push rate limiter for Tencent Tianyu. Each token bucket rate limiter adds tokens to the bucket at a fixed rate according to a preset QPS threshold (dynamically configured through the Apollo configuration center, such as approval.tencent.image.push.rateLimit=400). When the review engine module is about to initiate a call request to the third-party review service provider, the rate limiting and circuit breaking unit first retrieves the corresponding token bucket rate limiter instance from the rate limiter registry and calls its tryAcquire method to attempt to obtain a token from the bucket. If the retrieval is successful, it indicates that the current QPS has not exceeded the rate limiting threshold, and the review engine module executes the call to the third-party service normally. If the retrieval fails, it indicates that the current request has triggered rate limiting. The rate limiting and circuit breaker unit does not immediately discard the request, but instead re-delivers the submission message corresponding to the resource to be reviewed to the delayed queue of the submission message queue (approvalContentTopic) through the message sending service, and sets a delay level (default delay of 5 seconds before re-consumption). The submission message will re-enter the rate limiting judgment process after the delay time expires. Through the above token bucket rate limiting mechanism, the system controls the call rate smoothly when the QPS quota of the third-party review service provider is limited, avoiding the breakdown of the third-party service gateway due to instantaneous traffic peaks. At the same time, the automatic rate reduction and retry of requests is achieved by re-entering the delayed queue to ensure that no requests are lost.
[0047] Additionally, when the failure rate of calls to any third-party audit service provider exceeds a preset threshold, a circuit breaker is triggered, switching subsequent call requests to a backup audit service provider.
[0048] Specifically, when the rate limiting and circuit breaking unit initiates a call to a third-party auditing service provider, it statistically analyzes the results of each call, recording the number of successes and failures, and maintains a sliding time window (e.g., a 30-second window) to calculate the failure rate within that window. When the failure rate (number of failures / total number of calls) within the window exceeds a preset threshold (e.g., 50%), the rate limiting and circuit breaking unit determines that the third-party auditing service provider is in an unhealthy state and triggers the circuit breaker mechanism. During the circuit breaker period (e.g., 30 seconds), all call requests originally routed to the faulty provider are no longer actually sent to that provider, but are automatically switched to the backup auditing service provider (i.e., the next-priority provider in the business's policy configuration). The rate limiting and circuit breaking unit writes the call failure information to a failure record table (pc_check_fail_record), and the record fields include resource identifier (resource_id), current provider (old_supplier), backup provider (next_supplier), third-party failure error code (third_code), and current retry count (retry_times) for subsequent retries and traceability. After the circuit breaker window expires, the rate limiting and circuit breaker unit enters a half-open probing state, allowing a small number of requests (e.g., one) to be forwarded to the original faulty provider to test whether it has recovered. If the probe request is successful, the circuit breaker is closed, and normal calls resume. If the probe request still fails, it re-enters the circuit breaker state and continues to use the backup provider. Furthermore, when the rate limiting and circuit breaker unit determines that a provider has triggered the circuit breaker, it also sends a DingTalk alert message to developers via the exception alert service (ExceptionDingTalkServiceImpl). The alert includes information such as the provider name, failure rate, circuit breaker duration, and the backup provider after the switch. Through this circuit breaker degradation mechanism, the system can quickly isolate exceptions when a third-party audit service provider experiences a failure or performance degradation, ensuring the continuous availability of the main audit process.
[0049] Furthermore, it also includes: The task assignment control module is configured to respond to the task assignment request of the auditor, add a distributed lock to the public task pool according to the audit scenario dimension and audit type dimension, and after the lock is successfully acquired, allocate the audit task from the public task pool to the auditor's personal task pool, and update the status of the audit task to be assigned. Specifically, when an auditor initiates a task assignment request through the human review workbench page in the management backend, the task assignment control module receives the request and, based on the review scenario (scene, such as GRAPHICS for text review, IMG for pure image review, and AUDIO for audio review) and review type (checkType, such as FIRST_EMP for first review, MANUAL_REVIEW for reverse review, and SPOT_REVIEW for spot check) in the request parameters, generates a distributed lock key (e.g., pc-ugc:approvalPool:sceneLock_{tenantId}_{scene}_{checkType}). This distributed lock is then acquired through RedissonClient (using the tryLock method, with a 5-second timeout and a 15-second holding period). Because all auditors under the same scenario and review type share the same lock, only one auditor can successfully acquire the lock and execute the task assignment operation at any given time. Other auditors' assignment requests are either in a waiting state or return a "system busy, please try again later" message before the lock is released. Once the auditor successfully acquires the distributed lock, the task requisition control module queries the public task pool (i.e., records in the audit task table pc_approval_task where receive_status is UN_RECEIVE and status is INIT) for audit tasks to be requisitioned. During the query, it prioritizes tasks previously requisitioned by the auditor but not yet completed (i.e., tasks where receive_status is RECEIVE and emp_id is the current auditor's ID). If insufficient, it replenishes the pool of unrequited tasks. The task requisition control module then batch-assigns the retrieved audit tasks to the auditor. Specifically, it inserts a task requisition record into the employee task pool table (pc_approval_task_emp_pool), with fields including task ID (task_id), auditor ID (emp_id), requisition time (receive_time), audit scenario, and audit type. Simultaneously, it updates the requisition status in the audit task table (pc_approval_task) to RECEIVE, the auditor ID to the current auditor's ID, and the requisition time to the current timestamp, and updates the resource details status corresponding to the audit task to EXEC (under review). After the task allocation is completed, the task acquisition control module releases the distributed lock and returns the list of acquired review tasks to the front end for reviewers to perform review operations.
[0050] The overdue return module is configured to scan the personal task pool for review tasks whose claim duration exceeds a preset threshold through a scheduled task, reset the status of the review tasks to unclaimed and return them to the public task pool. Specifically, the timeout return module registers a scheduled task (ApprovalTaskEmpPoolReturnTask) through the XXL-Job distributed task scheduling platform. This scheduled task scans the employee task pool table (pc_approval_task_emp_pool) according to a preset cycle (e.g., every 5 minutes). During the scan, the timeout return module subtracts the task receipt time (receive_time) from the current system time to calculate the duration of each task. If this duration exceeds a preset threshold (this threshold is configured differently according to the review type: 30 minutes for first review FIRST_EMP and second review SECOND_EMP, and 60 minutes for reverse review MANUAL_REVIEW and spot check SPOT_REVIEW), then the task is determined to have timed out. For timed-out tasks, the timeout return module first resets the claim status in the corresponding audit task table (pc_approval_task) to UN_RECEIVE, clears the auditor ID, sets the claim time to 0, and rolls back the corresponding resource details status to INIT. Then, it deletes the task record from the employee task pool table, returning the task to the public task pool for other auditors to claim. If an auditor has not submitted the task 5 minutes before the timeout, a timeout reminder pop-up will appear on the front-end page. The auditor can click the "Apply for Extension" button to submit an extension request. After approval by the audit team leader, the timeout return module extends the task's claim time by 30 minutes (maximum of 2 extensions). Through this timeout return mechanism, the system effectively avoids the problem of audit tasks being locked due to auditors not processing them for a long time after claiming them, ensuring the effective flow of tasks in the public task pool and the rational utilization of audit resources.
[0051] The sampling inspection scheduling module is configured to extract resources to be inspected from the resources that have been reviewed and approved according to a preset sampling inspection strategy. The sampling inspection strategy includes a periodic execution strategy and a single execution strategy. The sampling inspection scheduling module extracts data from the search engine by business party identifier, review result type and time range through cursor pagination, and generates sampling inspection tasks by evenly distributing the total number of extracted data according to the business party dimension and the review result dimension.
[0052] Specifically, the sampling scheduling module first allows operations managers to create sampling strategies through the sampling strategy management interface. These strategies are stored in the sampling strategy table (pc_spot_approval_strategy). Strategy types include periodic tasks (PERIOD) and single-time tasks (ONCE): periodic tasks are executed periodically via an XXL-Job scheduled task (SpotApprovalStrategyPeriodExecuteTask, triggered daily at 00:00), while single-time tasks are manually triggered by operations personnel for immediate execution. The sampling strategy configures the list of business entities to be sampled (businessIds), the approval scenario (approvalScene), the approval method (checkType, such as machine review THIRD, human review first-round FIRST_EMP), the total sampling volume (totalNum), and the sampling quantity configured according to the approval result type (PASS / BLOCK / REVIEW). When the strategy is triggered, the sampling scheduling module extracts the resources to be sampled from the audit task index of the ElasticSearch search engine. The extraction method uses search_after cursor pagination, and performs scrolling pagination query with the resource upload time (upload_time) and document ID (_id) as sort keys. The query conditions include the list of business party identifiers, audit result type, audit method and time range. Each batch of data (e.g. 200 records) is queried, and the cursor value (scrollId) is maintained to continuously extract data until the total number of samples is met or all candidate resources have been traversed. After data extraction is complete, the sampling scheduling module executes an equal distribution algorithm: First, it distributes the data by business entity (getSpotPlanSize), allocating the total sampling volume to each business entity based on their resource share. Then, within each business entity's quota, it distributes the data according to three audit result types: PASS, BLOCK, and REVIEW (getAllSpotTaskPlanSize). Specifically, it uses an algorithm that adds a base amount and a remainder, sorted by hash, to ensure that the remainder is strictly equal to the total sampling volume. After equal distribution, the sampling scheduling module constructs sampling tasks (checkType is SPOT_REVIEW or ALL_REVIEW), writes the sampling tasks to the audit task table (pc_approval_task) and the audit resource details table (pc_approval_task_resource), and generates a strategy task execution record (pc_spot_approval_strategy_task) to track the execution status and sampling quantity of this sampling.Through the above mechanism, the system realizes automated, configurable, and high-precision sampling and scheduling of resources that have been reviewed, effectively supporting the continuous monitoring and evaluation of the quality of manual review.
[0053] Furthermore, the resource migration control module also includes: The rollback compatibility unit is configured to support switching between rollback schemes for various object storage services via a configuration switch, so as to select the appropriate rollback scheme when the migration execution module performs the operation of rolling back from the audit intermediate storage area to the public storage area.
[0054] Specifically, in actual system deployment environments, object storage services may have multiple rollback schemes coexisting due to technical architecture upgrades, vendor switching, or hybrid cloud strategies. For example, initially, the system might use the self-developed Tinman storage service for resource migration and rollback operations. Later, due to business expansion needs, it might switch to the standard rollback interface of a public cloud object storage service (such as Alibaba Cloud OSS or Huawei Cloud OBS). The two schemes differ in API protocol, parameter structure, authentication method, and return format. The rollback compatibility unit sets a configuration switch item (such as biz.ugc.v2.transfer.use.tinman.rollback) in the Apollo configuration center. The value of this switch is true or false. When the system starts up or runs, it listens for change events of this configuration item through the @ApolloConfigChangeListener annotation. When the configuration value changes, the rollback compatibility unit can hot-load the new configuration value without restarting the service. When the migration execution module performs a rollback operation from the audit intermediate storage area to the public storage area (i.e., calls the `createRevertFromMiddleBucketTask` method), the rollback compatibility unit first reads the current value of the configuration switch: if the configuration switch is true, the rollback compatibility unit selects the old Tinman rollback scheme and calls the rollback interface provided by the Tinman storage service (such as `tinmanRollbackService.revertFromMiddleToNormal`), which uses a self-developed internal protocol and signature mechanism; if the configuration switch is false, the rollback compatibility unit selects the standard rollback scheme and calls the general rollback interface of the object storage service (such as `ossService.revertMiddleToNormalBucket`), which uses the cloud vendor's standard RESTful API and AK / SK authentication method. The two rollback schemes are encapsulated into a unified rollback interface (`RevertStrategy`) within the rollback compatibility unit. Each scheme implements an independent strategy class (`TinmanRevertStrategy` and `OssRevertStrategy`). The migration execution module only depends on this unified interface and does not need to be aware of the implementation details of the specific rollback scheme. Through the aforementioned configuration switch mechanism, the system can gradually switch traffic from the old solution to the new solution via a canary release approach during object storage service solution upgrades or switching: first, the new solution switch is enabled for a small number of business users to verify its effectiveness; after successful verification, the switch scope is gradually expanded until a full switch is completed; if any anomalies occur during the canary release period, the switch can be immediately rolled back to the old solution, achieving a fast and lossless rollback. This mechanism effectively reduces the coupling between the system and the underlying object storage service, ensuring a smooth transition for the migration module during the evolution of the storage infrastructure.
[0055] Example 2 The above embodiment 1 details the core architecture and the specific implementation of its main modules for the three-barrel resource migration driven by audit results. In practical applications, the present invention may also include the following preferred embodiments to further improve the functional completeness of the system and the feasibility of actual deployment.
[0056] As a second embodiment of the present invention, based on the core migration architecture, the system further includes the following preferred features: asynchronous access to audit tasks and aggregation of multi-stage results, unified management and consistency assurance of multi-stage audit records, and intelligent identification and reuse of duplicate resources.
[0057] Summary of asynchronous access and multi-stage results for review tasks: The system also includes an audit task access unit and an audit result aggregation unit. The audit task access unit is configured to receive resources submitted by business parties for audit via a Dubbo RPC or HTTP interface, generate a unique resource group identifier (groupId) for each submission, and write the resources to be audited and their associated information (including business party identifier, user identifier, submission time, resource content, etc.) into a resource group table and a resource details table. Subsequently, it sends the submission message to the audit message queue for asynchronous consumption and processing by the audit engine module. The audit result aggregation unit is configured to receive audit results from various stages submitted by the multi-strategy machine audit adaptation unit and the task assignment control module, and aggregate and merge the results according to the resource group dimension. Specifically, a resource group may contain various types of resources such as text, images, audio, and video. Each resource may undergo multiple audit stages such as machine audit, first review, reverse review, and random inspection. The same resource may generate different audit results at different stages (e.g., REVIEW for machine audit, PASS for human first review, and BLOCK for random inspection). The audit result aggregation unit merges the audit results of all resources at all stages within the same resource group according to a preset priority rule (BLOCK has the highest priority, followed by PASS, then REVIEW, and UNKNOWN has the lowest priority), and takes the result with the highest priority as the final audit result for that resource group. After the merging is completed, the audit result aggregation unit triggers the resource migration control module to execute the corresponding bucket migration operation, and sends the final audit result back to the business system via a message queue.
[0058] Regarding the unified management and consistency assurance of multi-stage audit records: The system also includes an audit record management unit, configured to uniformly store and trace the audit details of each resource at each audit stage. Specifically, the audit record management unit maintains an audit record table (pc_approval_resource_check_record) in the system. Each audit record includes at least the resource identifier (resource_id), audit type (check_type, values are THIRD machine audit, FIRST_EMP first audit, SECOND_EMP second audit, MANUAL_REVIEW reverse check, SPOT_REVIEW spot check, ALL_REVIEW full spot check), audit result (result), audit details (detail, stored in JSON format as the original violation tags and confidence scores returned by the supplier), audit supplier (supplier), audit employee ID (emp_id, recorded during manual audit), and audit completion time (complete_time). Whenever the audit engine module completes machine audit or the task assignment control module completes human audit submission, the audit record management unit automatically inserts a new audit record into this table. When the audit result aggregation unit performs multi-stage result merging, it queries all records under the same resource group and the same resource in the audit record table, and merges the results according to the execution order and priority rules of the audit type, thereby ensuring that the audit history of each resource can be completely traced. In addition, the audit record management unit is also configured with a duplicate resource association function: when the current resource to be audited is identified as a duplicate resource, the audit record management unit associates the current resource identifier with the audit record of the historical resource, directly reusing the audit results and audit details of the historical resource, avoiding duplicate machine audit calls, and ensuring the consistency of the audit conclusions of the same MD5 resource under different submission requests.
[0059] Intelligent identification and result reuse of duplicate resources: The system also includes a duplicate resource identification unit, configured to intelligently identify duplicate resources and reuse results based on the content feature values of the resources. Specifically, when a business submits a resource to be reviewed to the system, the duplicate resource identification unit calculates the content feature value (such as the MD5 value) of the resource file to be reviewed during the process of the consumer processing the submission message in the review task access. It then uses this MD5 value as the key to query the resource details table for a historical review record with the same MD5 value. If the query finds a match, it further determines whether the review task corresponding to the historical review record has been completed (i.e., the review result is not UNKNOWN and the completion time is not empty). If it has been completed, the duplicate resource identification unit determines that the current resource is a duplicate resource, directly reuses the final review result and review details from the historical review record, and associates the current resource identifier with the resource identifier of the historical resource (recording the repeat_resource_id field). This triggers the review result aggregation unit to directly call back to the business, without needing to send the current resource to the review engine module for re-machine review. If the review task corresponding to a historical review record has not yet been completed (e.g., it is currently under review), the duplicate resource identification unit marks the current resource as pending. Once the historical resource review is completed, its result will be automatically reused through the association relationship. Through this duplicate resource identification mechanism, the system effectively avoids the problem of repeatedly consuming third-party machine review quotas when the same content resource is repeatedly submitted for review by different users or different business parties, significantly reducing review costs and latency. Simultaneously, in the core migration architecture of this invention, duplicate resources also trigger the resource migration control module to execute corresponding bucket migration operations after reusing historical results, ensuring that the physical storage location of duplicate resources remains consistent with the reused review results, and resources still follow a complete closed loop of three-bucket isolation between different storage areas.
[0060] Specifically, the preferred features described in this second embodiment, such as audit task access, audit result aggregation, audit record management, and duplicate resource identification, are all based on the three-bucket migration core architecture described in the first embodiment. The output of the audit result aggregation unit directly triggers the resource migration control module to execute the corresponding migration path. After reusing historical results, the duplicate resource identification unit also completes the bucket migration operation through the resource migration control module. Thus, while improving the completeness of system functions, it forms a close collaborative relationship with the core migration architecture.
[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A review result driven resource migration content review system, comprising: include: The resource isolation storage module is configured to store the resources to be reviewed in physically isolated public storage areas, intermediate review storage areas, and evidence storage areas according to their review status. The review engine module is configured to receive the resource to be reviewed, perform content review on the resource to be reviewed, and generate review results; The resource migration control module is configured to automatically generate and persist migration task records from the current storage area to the target storage area based on the audit results output by the audit engine module. The migration task records include at least the resource identifier, the source storage area identifier, the target storage area identifier, and the migration status. The migration execution module is configured to asynchronously execute the resource migration operation corresponding to the migration task record, physically move the resource to be reviewed from the current storage area to the target storage area, and update the migration status according to the migration execution result; The resource isolation storage module updates the storage location information of the resource to be reviewed based on the migration execution result of the resource migration control module, so that the physical storage location of the resource to be reviewed is consistent with the review result.
2. The audit result driven resource migration content audit system of claim 1, wherein, The resource migration control module controls the migration of the resource to be reviewed according to at least one of the following migration paths: In response to the submission of the resource to be reviewed, the resource to be reviewed is moved from the public storage area to the review intermediate storage area, which is an isolated storage space that is not accessible to the outside world; In response to the audit result being approved, the resource to be audited is rolled back from the audit intermediate storage area to the public storage area; In response to the audit result being that the audit failed, the resource to be audited is moved from the audit intermediate storage area to the evidence storage area.
3. The audit result driven resource migration content review system of claim 1, wherein, The resource migration control module includes: The migration task creation unit is configured to synchronously create the migration task record in the transaction context used to persist the audit result or the submission record of the resource to be audited, and trigger the sending of the migration message through the callback mechanism after the transaction is committed, so as to ensure the atomicity between the persistence of the audit result and the creation of the migration task record.
4. The audit result driven resource migration content audit system of claim 3, wherein, The resource migration control module also includes: The idempotent control unit is configured to check whether there is an existing migration task record under the same resource ID that is in the pending execution state before creating a new migration task record. If it exists, the existing migration task record is cancelled and a new migration task record is created. And / or, add a distributed lock based on the content feature value of the resource to be reviewed to prevent the same resource from being concurrently executed for migration operations; and / or, set a unique constraint in the database table corresponding to the migration task record to prevent the same resource from being repeatedly created for migration task records under the same migration type.
5. The audit result driven resource migration content review system of claim 1, wherein, The migration execution module includes: The delayed scheduling unit is configured to determine a delayed execution time based on the violation level of the audit result or the business entity identifier to which the resource to be audited belongs, and to set the delayed execution time for the migration task record, so that the migration operation is triggered and executed after the delayed execution time arrives; and The retry unit is configured to automatically re-execute the migration operation according to a preset retry strategy when the migration operation fails, until it succeeds or the maximum number of retries is reached.
6. The audit result driven resource migration content review system of claim 1, wherein, The migration execution module includes: A callback consumption unit is configured to receive a migration completion callback message from the object storage service and update the migration status of the migration task record based on the callback message; and The timed scanning unit is configured to scan migration task records that are in a pending state and have reached their execution time through a timed task, and trigger the migration execution module to perform the corresponding migration operation.
7. The audit result driven resource migration content review system of claim 1, wherein, The audit engine module includes: A multi-strategy machine-based review adaptation unit is configured to define a unified review strategy interface and implement corresponding strategy adapters for different third-party review service providers. Each strategy adapter includes at least adaptation logic for encapsulating differences in request protocols, authentication methods, and response formats among the various third-party review service providers; and... The policy routing unit is configured to dynamically select a serial policy or a parallel policy based on business configuration information. The serial policy calls multiple third-party audit service providers in a preset order until the final audit result is obtained. The parallel policy calls multiple third-party audit service providers concurrently and merges the audit results of each provider according to a preset priority.
8. The audit result driven resource migration content audit system of claim 7, wherein, The audit engine module also includes: The rate limiting and circuit breaking unit is configured to establish a token bucket rate limiter based on the dimensions of third-party audit service providers and resource types. When a call request to any third-party audit service provider triggers rate limiting, the submission message corresponding to the resource to be audited is resubmitted to the delay queue for retry. Additionally, when the failure rate of calls to any third-party audit service provider exceeds a preset threshold, a circuit breaker is triggered, switching subsequent call requests to a backup audit service provider.
9. The audit result driven resource migration content review system of claim 1, wherein, Also includes: The task assignment control module is configured to respond to a reviewer's task assignment request by acquiring a distributed lock on the public task pool according to the review scenario and review type dimensions. Upon successful lock acquisition, it allocates review tasks from the public task pool to the reviewer's personal task pool and updates the status of the review tasks to "assigned." The overdue return module is configured to scan the personal task pool for review tasks whose claim duration exceeds a preset threshold through a scheduled task, reset the status of the review tasks to unclaimed and return them to the public task pool. The sampling inspection scheduling module is configured to extract resources to be inspected from the resources that have been reviewed and approved according to a preset sampling inspection strategy. The sampling inspection strategy includes a periodic execution strategy and a single execution strategy. The sampling inspection scheduling module extracts data from the search engine by business party identifier, review result type and time range through cursor pagination, and generates sampling inspection tasks by evenly distributing the total number of extracted data according to the business party dimension and the review result dimension.
10. The audit result driven resource migration content review system of claim 1, wherein, The resource migration control module also includes: The rollback compatibility unit is configured to support switching between rollback schemes for various object storage services via a configuration switch, so as to select the appropriate rollback scheme when the migration execution module performs the operation of rolling back from the audit intermediate storage area to the public storage area.
Citation Information
Patent Citations
Content auditing system and method, electronic equipment and storage medium
CN122112943A
