A recommendation queue data processing method and device based on multi-dimensional version constraints and a medium

Abstract

The present application relates to the technical field of distributed data processing, and provides a recommendation queue data processing method and device based on multi-dimensional version constraint and medium, comprising: obtaining the object state version and space constraint rule version of a target data object and constructing a recommendation data snapshot; generating a composite deduplication key based on the object identifier, node identifier and double version; in response to meeting the write-in condition, writing the snapshot into the personalized recommendation queue and global monitoring queue; in response to a change event, performing invalidation cleaning on the affected recommendation data snapshot; and in response to a read request, performing secondary verification based on the current double version and determining the display data, wherein the present application solves the problems of dirty data retention of the recommendation queue and inconsistency of multi-entry data in a dynamic environment through the version constraint and double queue isolation mechanism, and improves the queue data consistency and multi-entry data display consistency.

Description

Technical Field

[0001] This invention relates to the field of distributed data processing technology, and in particular to data processing technology in recommendation systems and message push scenarios. Specifically, it relates to a recommendation queue data processing method, apparatus and medium based on multi-dimensional version constraints. Background Technology

[0002] In distributed recommendation systems and message push architectures, it is typically necessary to maintain personalized recommendation queues to display data objects to user terminals. In existing technologies, when the state attributes or spatial constraints of a pushed object change, invalid data in the queue is usually handled by full scan cleanup or broadcast failure notification. However, full scan cleanup requires traversing the personal recommendation queues of all receiving nodes, which can lead to significant computational and I / O overhead in high-concurrency scenarios. Broadcast failure notification requires sending change messages to all relevant nodes, which can easily cause message storms and cache jitter, and in extreme scenarios, messages may be lost or delayed. Furthermore, if different data display entry points use independent filtering logic, the same object can easily exhibit inconsistent display states at different entry points.

[0003] The aforementioned defects prevent recommendation records already written to the queue from promptly detecting changes in source data or rules, resulting in a large amount of dirty data and severely impacting system data consistency and user experience.

[0004] The aforementioned problems are particularly prominent in recommendation scenarios with spatial constraints and dynamic state changes. Taking flexible employment platforms as an example, task status, fence rules, and worker locations may change frequently, and the platform usually has multiple task discovery entry points, making queue consistency control face even more severe challenges.

[0005] Therefore, it is necessary to provide a recommended queue data processing method that can balance queue data consistency, cleaning efficiency, and system resource overhead in a dynamically changing environment. Summary of the Invention

[0006] To address the problems of dirty data retention in recommendation queues under dynamic changing environments, high overhead of full cleanup, and inconsistency of data from multiple entry points in existing technologies, this application proposes a recommendation queue data processing method, apparatus, and medium based on multi-dimensional version constraints to achieve strong consistency of queue data and optimization of system resources in high-concurrency scenarios.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A recommendation queue data processing method based on multi-dimensional version constraints, applied to a distributed recommendation system, includes: responding to the detection that a target data object meets a preset push trigger condition, filtering a set of candidate nodes based on spatial constraint rules to determine a receiving node; obtaining the object state version and spatial constraint rule version of the target data object; constructing a recommendation data snapshot carrying the object state version and the spatial constraint rule version; generating a composite deduplication key based on the object identifier of the target data object, the node identifier of the receiving node, the object state version, and the spatial constraint rule version; responding to the fact that the composite deduplication key does not exist in the caching system, writing the recommendation data snapshot into the personalized recommendation queue corresponding to the receiving node, and writing the recommendation data snapshot into a global monitoring queue; responding to the detection of an attribute change event of the target data object or a rule change event of the spatial constraint rule, performing invalidation cleanup on recommendation data snapshots in the global monitoring queue and the personalized recommendation queue affected by the attribute change event or the rule change event; responding to receiving a read request for the personalized recommendation queue, performing a secondary verification on the recommendation data snapshots in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version, and determining the target display data based on the verification result.

[0008] The above scheme embeds version information of object state and spatial constraint rules into the recommended snapshot and uses it as a factor for generating the composite deduplication key, enabling queue write operations to have state awareness and avoiding resource waste caused by repeated writes under the same state. Utilizing a dual-queue isolation architecture of a global monitoring queue and a personalized recommendation queue, combined with change event-driven failure cleanup and secondary version verification during reads, a closed-loop consistency control mechanism is constructed for writes, cleanup, and reads, effectively solving the technical problems of dirty data retention and excessive overhead of full cleanup in dynamic environments.

[0009] Preferably, obtaining the object status version and spatial constraint rule version of the target data object includes: generating the object status version based on the attribute fields of the target data object, wherein the attribute fields include at least one of object status, capacity threshold, time attribute, or audit status; generating the spatial constraint rule version based on the configuration parameters of the spatial constraint rule, wherein the configuration parameters include at least one of geofence geometric boundary data, rule priority, distance threshold, or effective time; and triggering an update of the corresponding object status version or spatial constraint rule version when any of the attribute fields or configuration parameters changes.

[0010] This preferred scheme clarifies the generation factor and update triggering condition of the version identifier, enabling the version number to accurately represent the substantial changes in data objects and constraint rules, and providing a reliable comparison benchmark for subsequent deduplication judgment and validity verification.

[0011] Preferably, before writing the recommended data snapshot into the personalized recommendation queue corresponding to the receiving node, the method further includes: obtaining a set of candidate nodes, and performing multi-stage filtering on the set of candidate nodes based on the spatial constraint rules corresponding to the version of the spatial constraint rules to generate a set of pushable nodes; the multi-stage filtering includes: performing location validity filtering based on the location timestamp of the candidate node; performing spatial constraint hit filtering and distance threshold filtering based on the current coordinates of the candidate node and the target coordinates of the target data object; performing blacklist filtering based on the blocking or prohibition of interaction between the candidate node and the target data object; and the nodes in the set of pushable nodes serve as the receiving nodes.

[0012] This preferred solution employs a multi-stage shrinking filtering strategy to pre-remove nodes with location failures, spatial mismatches, or abnormal blacklist status before writing them to the queue, thereby reducing the amount of invalid data written to the queue and the amount of subsequent verification calculations.

[0013] Preferably, the step of writing the recommended data snapshot to the personalized recommendation queue corresponding to the receiving node in response to the composite deduplication key not existing in the cache system includes: detecting whether the composite deduplication key exists in the cache system; if the composite deduplication key does not exist, writing the recommended data snapshot to the personalized recommendation queue through an atomic write operation and storing the composite deduplication key; while writing to the personalized recommendation queue, constructing a reverse index from the target data object to the receiving node, the reverse index being used to record the identifier of the receiving node to which the recommended data snapshot of the target data object is written.

[0014] This preferred solution utilizes atomic writes to ensure transactional consistency between deduplication checks and queue writes, preventing duplicate pushes in concurrent scenarios; by constructing a reverse index, it provides a data foundation for subsequent precise location and cleanup based on change events.

[0015] Preferably, in response to detecting an attribute change event of the target data object or a rule change event of the spatial constraint rule, the step of performing invalidation cleanup on the recommendation data snapshots in the global monitoring queue and the personalized recommendation queue affected by the attribute change event or the rule change event includes: determining a cleanup strategy based on the number or urgency of the affected nodes of the attribute change event or the rule change event; when using an active cleanup strategy, locating the affected personalized recommendation queue and deleting the corresponding recommendation data snapshot; when using an invalidation marking strategy, updating the object state version of the target data object or the spatial constraint rule version and writing an invalidation mark in the global monitoring queue; when using a lazy cleanup strategy, retaining the recommendation data snapshots in the personalized recommendation queue for deletion during secondary verification.

[0016] This preferred solution provides three hierarchical strategies: proactive cleanup, failure marking, and lazy cleanup. The system can dynamically schedule cleanup resources based on the scope and urgency of the change, reducing the impact of instantaneous cleanup operations on system stability while ensuring eventual data consistency.

[0017] Preferably, the secondary verification of the recommendation data snapshot in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version includes: comparing the object state version carried in the recommendation data snapshot with the current object state version, and comparing the spatial constraint rule version carried in the recommendation data snapshot with the current spatial constraint rule version; if the version comparison is inconsistent, a re-verification of the state attributes of the target data object is triggered, or a re-verification of the current coordinates of the receiving node with the spatial constraint rule is triggered; if the re-verification fails, or the recommendation data snapshot exceeds a preset validity period, the recommendation data snapshot is marked as invalid or deleted from the personalized recommendation queue.

[0018] This preferred solution uses version comparison as the first line of defense during reading, triggering re-verification only when versions are inconsistent, thus avoiding the overhead of performing a full status query for all read requests. This mechanism serves as a fallback for event-driven cleanup, ensuring that even if cleanup events are lost or delayed, invalid data will not be displayed incorrectly.

[0019] Preferably, the method further includes: in response to a data query request received through multiple data display entry points, invoking a unified validity filtering operator; the unified validity filtering operator performs consistent filtering on candidate data objects corresponding to the multiple data display entry points based on the latest object state version and the latest spatial constraint rule version, so as to intercept candidate data objects that do not meet the display conditions; the multiple data display entry points retain their own data sources and sorting strategies, and only share the filtering logic of the unified validity filtering operator in the display validity judgment.

[0020] This preferred solution decouples the sorting strategy and validity judgment, enabling multiple heterogeneous entry points to share the same set of state filtering logic based on the latest version, fundamentally eliminating the problem of inconsistent data display caused by asynchronous cache refresh or differences in filtering rules among entry points.

[0021] Preferably, before writing the recommended data snapshots into the personalized recommendation queue, the method further includes multi-level throttling control: generating an object-level cooling key based on the object identifier of the target data object to limit the number of push batches of the target data object within a preset time window; generating a sending node-level quota key based on the node identifier of the sending node to limit the total number of pushes by the sending node within a preset period; and generating a receiving node-level throttling key based on the node identifier of the receiving node to limit the number of recommended data snapshots received by the receiving node within a preset time window.

[0022] This preferred solution constructs a three-dimensional traffic protection system encompassing the object, sender, and receiver, effectively preventing queue expansion and message harassment caused by abnormal retries, overlapping scheduled tasks, or malicious order manipulation, thus ensuring the service quality and stability of the recommendation system.

[0023] Furthermore, the present invention also provides a recommendation queue data processing device based on multidimensional version constraints, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the recommendation queue data processing method based on multidimensional version constraints described above.

[0024] The device executes a computer program through a processor to implement any of the methods described above, achieving the beneficial effects of ensuring queue data consistency in a dynamically changing environment, reducing system cleanup overhead, and improving the consistency of multi-entry display.

[0025] In addition, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the recommendation queue data processing method based on multidimensional version constraints as described above.

[0026] Beneficial effects: This invention introduces object state version and spatial constraint rule version as core metadata for recommendation data snapshots and integrates them into the generation logic of composite deduplication keys, achieving fine-grained state awareness and deduplication control for recommendation queue write operations. By constructing a dual-queue isolation architecture of a global monitoring queue and a personalized recommendation queue, combined with change event-driven failure cleanup and version secondary verification during reading, a closed-loop consistency control mechanism is built across the write, cleanup, and read ends, effectively solving the technical problems of dirty data retention and excessive overhead of full cleanup in dynamic environments. Furthermore, by constructing a reverse index, the addressing mode for failure cleanup is transformed from full scanning to precise positioning, reducing the I / O overhead of proactive cleanup; through three hierarchical strategies—proactive cleanup, failure marking, and lazy cleanup—the system can dynamically schedule cleanup resources according to the scope of change impact; by setting a secondary verification fallback mechanism based on version comparison at the read end, it ensures that failed data is not incorrectly displayed; by deploying a unified validity filtering operator, the problem of inconsistent data display states caused by heterogeneous data sources and independent filtering logic at multiple entry points is eliminated; and by constructing multi-level throttling control, queue expansion and message harassment are effectively suppressed. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the structure of the recommendation queue data processing device based on multi-dimensional version constraints according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall process of the recommendation queue data processing method according to an embodiment of the present invention (corresponding to steps S100-S500). Figure 3 This is a schematic diagram of the dual-queue isolated write and reverse index construction process in an embodiment of the present invention (corresponding to step S300 and reverse index construction).

[0028] Among them, 100-Snapshot building module, 200-Deduplication control module, 300-Isolation write module, 400-Failure cleanup module, 500-Secondary verification module, 600-Global monitoring queue, 700-Personalized recommendation queue, and 800-Reverse index storage. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. It should be noted that, without conflict, embodiments and features thereof in this invention can be combined with each other.

[0031] Example 1 like Figure 1 and Figure 2 As shown, this embodiment provides a recommendation queue data processing method based on multi-dimensional version constraints, applied to a distributed recommendation system, aiming to solve the technical problems of recommendation queue data consistency control and system resource optimization under dynamically changing environments. The method described in this embodiment establishes a higher-level technical framework based on version snapshots, using composite deduplication as a control mechanism, a dual-queue storage architecture, and secondary verification as a fallback guarantee.

[0032] In step S100, in response to detecting that the target data object meets the preset push triggering conditions, filtering is performed on the candidate node set based on spatial constraint rules to determine the receiving node. The object state version and spatial constraint rule version of the target data object are obtained, and a recommended data snapshot carrying the object state version and the spatial constraint rule version is constructed. The spatial constraint rule version is generated based on the geofence configuration parameters. Specifically, the preset push triggering conditions refer to the target data object meeting preset push admission conditions, such as the target data object's publisher being a newly registered publisher with a historical completed data volume below a preset threshold, or the target data object passing review and its region having enabled targeted push functionality. The recommended data snapshot is an immutable data structure that records the binding relationship between the target data object and the spatial constraint rules, as well as the version identifier, at a specific moment. The object state version uniquely identifies the attribute combination state of the target data object at the current moment, and the spatial constraint rule version uniquely identifies the currently effective spatial filtering logic configuration. By embedding version information in the snapshot, the recommendation record has timeliness verification capabilities, transforming business validity judgment into a version number comparison operation.

[0033] In step S200, a composite deduplication key is generated based on the object identifier of the target data object, the node identifier of the receiving node, the object state version, and the spatial constraint rule version. This composite deduplication key is a unique index string composed of information from four dimensions, for example, in the format "object ID: node ID: object state version: spatial constraint rule version". The system only determines a push as duplicate and blocks it if and only if none of the four elements—target data object, receiving node, object state, and spatial constraint rule—have changed. However, if a substantial change occurs in the object state or spatial constraint rule, resulting in a version number update, the generated composite deduplication key will change even if the object identifier and node identifier remain unchanged, thus allowing new recommended data snapshots to be written to the queue.

[0034] In step S300, in response to the composite deduplication key not existing in the caching system, the recommended data snapshot is written to the personalized recommendation queue corresponding to the receiving node, and the recommended data snapshot is also written to the global monitoring queue. This embodiment employs a dual-queue isolated write architecture, where the personalized recommendation queue is oriented towards a specific receiving node and carries the data stream to be displayed; the global monitoring queue is independent of the specific receiving node and stores copies or summary information of all recommendation records for global status monitoring and batch failure management. During the write process, the system ensures transactional consistency of the dual-queue write through atomic operations.

[0035] In step S400, in response to the detection of a change event in the attribute field of the target data object or the configuration parameters of the spatial constraint rule, invalidation cleanup is performed on the recommendation data snapshots affected by the change event in the global monitoring queue and the personalized recommendation queue. The change event includes, but is not limited to, modification of object state attributes, adjustment of capacity thresholds, change of spatial geometric boundaries, and adjustment of rule priorities. Upon detecting such an event, the system triggers the invalidation cleanup process, quickly identifying the data range to be processed by comparing the new version carried by the event with the version of the old record in the queue, and flexibly selecting a cleanup strategy based on the scope and urgency of the change.

[0036] In step S500, in response to receiving a read request for the personalized recommendation queue, a secondary verification is performed on the recommendation data snapshot in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version, and the target display data is determined according to the verification result. When the receiving node initiates a read request, the system extracts the recommendation data snapshot from the personalized recommendation queue and obtains the current object state version and spatial constraint rule version in real time. If the version carried in the snapshot is inconsistent with the latest version, the system triggers re-verification logic to re-evaluate whether the object still meets the push conditions. The data is only returned as the target display data if the version is consistent or the re-verification passes. Even if the failure cleanup is not executed in time, the secondary verification can still ensure the accuracy of the final display data.

[0037] Example 2 In this embodiment, the version identifier generation mechanism mentioned in Embodiment 1 is further refined, and the multi-stage pre-filtering process before writing is described in the latter half of this embodiment. Specifically, the process of obtaining the object state version and spatial constraint rule version of the target data object includes: generating the object state version based on the attribute fields of the target data object, wherein the attribute fields include at least one of object state, capacity threshold, time attribute, or review status; generating the spatial constraint rule version based on the configuration parameters of the spatial constraint rule, wherein the configuration parameters include at least one of geometric boundary data, rule priority, distance threshold, or effective time; and triggering an update of the corresponding object state version or spatial constraint rule version when any of the attribute fields or configuration parameters changes.

[0038] In step S101, the system generates an object status version based on the attribute fields of the target data object. These attribute fields include core status parameters that directly determine whether the target data object is eligible for display, such as object status, capacity threshold, time attribute, and review status. This embodiment only uses these key attribute fields as version generation factors; changes to non-key attributes are excluded, thereby avoiding invalid version iterations caused by irrelevant information changes.

[0039] In step S102, the system generates a spatial constraint rule version based on the configuration parameters of the spatial constraint rules. A spatial constraint rule is a dynamic set of policies containing multi-dimensional parameters. The geometric boundary data defines the physical range of the spatial constraint, the rule priority is used for conflict resolution when multiple rules overlap, the distance threshold defines the maximum allowed interaction distance, and the effective time limits the effective period of the rule itself. Adjusting any of the above configuration parameters will result in an update of the spatial constraint rule version.

[0040] As a preferred implementation, a hash digest algorithm is used to generate a version identifier. The selected attribute fields or configuration parameters are concatenated in lexicographical order and the message digest value (such as MD5 or SHA-256) is calculated. The first N bits are taken as the version number. Its mathematical expression is: Ver = Hash(Sort(Field1 || Field2 || ... || FieldN)). Here, `Ver` is the output version identifier string, used to embed in the recommended data snapshot and composite deduplication key as a version comparison benchmark; `Field1` to `FieldN` are the attribute fields or configuration parameters (such as object status, capacity threshold, geometric boundary data, etc.) participating in the version calculation. Each field is serialized as a string and then concatenated. `N` is the total number of fields participating in the calculation; `||` is the string concatenation operator; the `Sort()` function sorts the fields before concatenation according to the lexicographical order of the field names, ensuring that the same set of fields produces a consistent concatenation result regardless of the input order; `Hash()` is the hash digest function, such as MD5 outputting a 128-bit digest or SHA-256 outputting a 256-bit digest; the first N hexadecimal characters are truncated as the final version number. The selection of `N` needs to balance uniqueness and storage overhead. For example, `N=8` provides approximately 4.3 billion combinations, meeting the deduplication requirements of most scenarios. Version identifiers can also be generated using monotonically increasing sequence numbers or timestamp encoding.

[0041] When the system detects a change in any attribute field or configuration parameter, it immediately triggers a recalculation of the version identifier. This process is driven by data change events, such as database binlog listeners or application-layer domain event publications. The system has a built-in field whitelist filtering mechanism in its triggering logic; only changes to fields belonging to the version generation factor set will trigger a version recalculation.

[0042] Through the design of the above-mentioned generation factor, generation algorithm and triggering mechanism, this embodiment provides a definite data foundation and comparison benchmark for composite deduplication key generation, dual-queue write control and secondary verification during reading.

[0043] In this embodiment, after obtaining the version identifier and before writing the recommendation data snapshot to the personalized recommendation queue corresponding to the receiving node, the method further includes: obtaining a candidate node set, and performing multi-stage filtering on the candidate node set based on the spatial constraint rules corresponding to the spatial constraint rule version to generate a pushable node set. This step narrows the candidate range step by step, eliminating nodes that no longer meet the push conditions in the spatiotemporal dimension or correlation, thus preventing invalid data from entering subsequent processes. The rules on which the filtering is based are strictly bound to the aforementioned generated spatial constraint rule version, ensuring strict alignment between the filtering logic and the writing logic in the version dimension.

[0044] The multi-stage filtering includes: performing location validity filtering based on the location timestamp of the candidate node. Specifically, the system reads the timestamp field from the most recently reported location data of the candidate node, calculates the difference between the timestamp and the current system time, and if the difference exceeds a preset location validity duration threshold, the location information is determined to be invalid and removed.

[0045] The multi-stage filtering also includes: performing spatial constraint hit filtering and distance threshold filtering based on the current coordinates of the candidate node and the target coordinates of the target data object. Specifically, spatial constraint hit filtering is used to determine whether a node is located inside the geometric boundary defined by the version of the spatial constraint rules. For polygonal fences, a ray casting algorithm can be used to determine the inclusion relationship, while for circular fences, the Euclidean distance is calculated and compared with the radius. Distance threshold filtering is used to verify whether the physical distance between the node and the target coordinates is less than the maximum allowed interaction distance, preferably using the Haversine formula to calculate the spherical distance.

[0046] The multi-stage filtering also includes: performing blacklist filtering based on the blocking or prohibition of interaction between candidate nodes and target data objects. Specifically, the system queries the pre-stored association index to determine whether there are mutually exclusive, blocking, or prohibition of interaction status markers between the candidate node and the publisher of the target data object. If so, the node is removed from the candidate set.

[0047] The nodes in the pushable node set generated after the above multi-stage filtering are used as the receiving nodes. This pushable node set is the only valid input source for subsequent write operations. Since the filtering process is executed entirely based on the version of the spatial constraint rules, when the rules change, the new version of the rules is automatically applied to the next batch of filtering.

[0048] Example 3 like Figure 3 As shown, in this embodiment, the writing process described in step S300 of embodiment 1 is further refined, focusing on the atomic writing mechanism and reverse index construction logic under the control of the composite deduplication key. Specifically, in response to the composite deduplication key not existing in the cache system, the recommended data snapshot is written to the personalized recommendation queue corresponding to the receiving node, including: detecting whether the composite deduplication key exists in the cache system. The composite deduplication key is formed by connecting the target data object identifier, the receiving node identifier, the object state version, and the spatial constraint rule version through a predefined separator. It is considered a duplicate only when the identity of the object and the node and their version states are completely identical. The system usually detects whether the composite deduplication key has been stored through the key existence query interface provided by the cache system, for example, by using the GET or EXISTS command. If the return is empty, it is determined that the writing condition is met; otherwise, the writing is terminated directly.

[0049] If the composite deduplication key does not exist, the recommended data snapshot is written to the personalized recommendation queue via an atomic write operation, and the composite deduplication key is stored. The atomic write operation encapsulates queue writing and deduplication key storage within a single transaction unit. For example, a Lua script can be used to execute the following steps sequentially: confirm the composite deduplication key does not exist, serialize the snapshot and push it into the queue, write the deduplication key to the cache and set the same expiration time. If any step fails, the entire process is rolled back. In caching systems that do not support scripts, an optimistic locking mechanism combined with a retry strategy can also be used to achieve near-atomicity.

[0050] Simultaneously with writing to the personalized recommendation queue, a reverse index is constructed from the target data object to the receiving node. This reverse index records the identifiers of the receiving nodes to which the recommendation data snapshot of the target data object was written. The construction of this reverse index is also included within the transaction unit of the aforementioned atomic write operation. Its data structure preferably uses a set type, with the key consisting of the target data object identifier and the value being the set of receiving node identifiers that have received recommendations for that object. It is also configured with a TTL attribute consistent with the recommendation data snapshot.

[0051] This embodiment transforms the addressing mode of failure cleanup from full scan to precise location by constructing the aforementioned reverse index, providing crucial data structure support for multi-level failure cleanup strategies. In scenarios where write time needs to be recorded, a hash structure can also be used instead of a set structure.

[0052] In this embodiment, after the recommended data snapshot is written to the queue, when the target data object or spatial constraint rule changes, the recommended data snapshots affected by the change event in the queue need to be cleaned up to become invalid. The invalidation cleanup mechanism described in step S400 of Embodiment 1 is further refined, constructing a multi-level invalidation cleanup strategy driven by change events. In response to detecting a change event in the attribute field of the target data object or the configuration parameters of the spatial constraint rule, the process of cleaning up the recommended data snapshots affected by the change event in the global monitoring queue and the personalized recommendation queue includes: determining a cleanup strategy based on the number of affected nodes or the urgency of the change event. The number of affected nodes is statistically determined by querying the aforementioned inverted index, and the urgency is predefined by the change event type or specified by the business party. The system presets a threshold N1 for the number of affected nodes. When the number of affected nodes is less than N1 and the urgency is high, an active cleanup strategy is selected; otherwise, an invalidation marking strategy or a lazy cleanup strategy is selected.

[0053] When employing a proactive cleanup strategy, the affected personalized recommendation queues are located based on the inverted index, and the corresponding recommendation data snapshots are deleted. The system reads the inverted index set to obtain the identifiers of all affected receiving nodes, iterates through the set, and sends deletion commands to remove the corresponding snapshot entries and composite deduplication keys. Batch pipeline submission can be used to reduce network overhead. After completing the personalized queue cleanup, the global monitoring queue record and the inverted index itself are deleted synchronously. Proactive cleanup has extremely high timeliness, but I / O overhead is linearly related to the number of nodes, therefore its use is strictly limited to small-scale scenarios.

[0054] When an invalidation flag strategy is adopted, the object state version of the target data object or the spatial constraint rule version is updated, and an invalidation flag is written to the global monitoring queue. The system does not directly access the personalized recommendation queue of any receiving node, but operates only at the metadata level: atomically updating the version number so that the old snapshot is identified as expired in the subsequent secondary verification, and simultaneously writing an invalidation flag record to the global monitoring queue. This strategy transforms the O(N) level deletion overhead into an O(1) level version comparison overhead distributed on the reading end, smoothing the system load curve.

[0055] When employing a lazy cleanup strategy, a snapshot of the recommended data in the personalized recommendation queue is retained for deletion during secondary verification. Lazy cleanup relies entirely on the natural eviction mechanism during reading. When a receiving node initiates a read request, the secondary verification module re-executes a complete state attribute verification and spatial constraint verification. If the snapshot is found to be invalid, it is deleted or marked as logically invalid. This strategy shifts the cleanup cost to the actual read behavior, eliminating the need for additional cleanup resources for cold data that has never been accessed again, making it suitable for the gradual governance of long-tail objects.

[0056] The three cleanup strategies described above can be used in combination. For example, for changes with high urgency but a wide impact, the failure marking strategy can be executed first to instantly block the display, followed by proactive cleanup in batches during off-peak hours in the background; for changes with medium to low urgency, failure marking and lazy cleanup can be used simultaneously to ensure data consistency.

[0057] Example 4 In this embodiment, the secondary verification mechanism described in step S500 of Embodiment 1 is further refined. In response to receiving a read request for the personalized recommendation queue, a secondary verification is performed on the snapshot of the recommendation data in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version. This secondary verification mechanism is the final checkpoint before the data is returned to the receiving node, used to compensate for potential timing defects in event-driven failure cleanup. In distributed systems, the propagation of change events and cache updates often have delays, and cleanup events may be lost in extreme scenarios. Therefore, this embodiment forcibly embeds version comparison and re-verification logic into the read path to ensure that the data displayed on the front end is always subject to real-time validity verification.

[0058] The specific execution process of the secondary verification includes: comparing the object state version carried in the recommended data snapshot with the current object state version, and comparing the spatial constraint rule version carried in the recommended data snapshot with the current spatial constraint rule version. The system extracts the historical version number of the recommended data snapshot from the personalized recommendation queue and queries the latest version number in real time. If the two are completely consistent, it means that the state attributes and spatial constraint rules of the target data object have not undergone substantial changes, and the system directly returns it as the target display data without performing a re-verification step.

[0059] If the version comparison is inconsistent, a re-verification of the target data object's state attributes or a re-verification of the current coordinates of the receiving node against the spatial constraint rules is triggered. When the object's state version is inconsistent, the latest state attributes of the target data object are retrieved to verify whether the display conditions are still met; when the spatial constraint rule versions are inconsistent, the latest coordinates of the receiving node are obtained and the spatial constraint hit filtering and distance threshold filtering calculations are re-executed in conjunction with the new version rules. Only when the re-verification clearly indicates that the data is invalid is it determined that it cannot be displayed. This "compare first, then re-verify" strategy significantly reduces invalid refreshes caused by non-critical changes.

[0060] If the revalidation fails, or the recommended data snapshot exceeds its preset validity period, the recommended data snapshot is marked as invalid or deleted from the personalized recommendation queue. When revalidation confirms that the data no longer meets the display conditions or the snapshot TTL has expired, the system processes the data. A "synchronous marking + asynchronous deletion" mode is preferred, meaning that within the current request cycle, the data is only marked as logically invalid and filtered, while the deletion task is executed asynchronously in the background to avoid blocking read responses. Regardless of the method used, the system synchronously updates or deletes the corresponding composite deduplication key.

[0061] Furthermore, this embodiment also introduces a logging and monitoring mechanism for the verification process. The system records each version comparison inconsistency event and re-verification result. Through aggregation analysis, indicators such as version drift rate and re-verification pass rate can be calculated, providing data support for optimizing version generation strategies and investigating and cleaning up defects in the mechanism.

[0062] In this embodiment, the aforementioned secondary verification mechanism is further extended to scenarios with multiple data display entry points, constructing a unified validity filtering architecture. In response to data query requests received through multiple data display entry points, a unified validity filtering operator is invoked. This operator is configured as an independent shared service component in the recommendation system, serving as a mandatory gateway for all entry points before returning the final result. The multiple data display entry points include, but are not limited to, comprehensive recommendation entry points, search entry points, precise push entry points, and distance-based sorting entry points, all of which invoke the same operator through a standardized interface protocol.

[0063] The unified validity filtering operator performs consistency filtering on candidate data objects corresponding to the multiple data display entry points based on the latest object state version and spatial constraint rule version. The operator obtains the latest version number in real time through a multi-level caching strategy, traverses the candidate data object list one by one and compares the historical version with the latest version. If the versions are inconsistent, the re-verification logic is triggered. The object is only blocked and removed if the re-verification fails or the object has exceeded its validity period.

[0064] The multiple data display entry points retain their own data sources and sorting strategies, sharing the filtering logic of the unified validity filtering operator only for display validity judgment. This feature embodies the core architectural idea of ​​"decoupling sorting and filtering." For example, when an object's state version changes due to capacity exhaustion, the operator will intercept it before returning, even if the search entry's index has not been updated. This decoupling design allows new entry points to focus only on sorting and retrieval logic, without having to repeatedly implement state verification code; when adjusting compliance standards, only the operator configuration needs to be updated for all entry points to take effect synchronously, significantly reducing change risks and maintenance costs.

[0065] Example 5 In this embodiment, to further improve the system's robustness under extreme traffic scenarios, the method further includes multi-level throttling control before writing the recommendation data snapshot to the personalized recommendation queue. The execution timing of this multi-level throttling control mechanism is strictly configured before the composite deduplication key detection step described in Embodiment 1, enabling low-cost throttling operations to intercept abnormal traffic in advance and ensuring that the expensive backend deduplication and writing resources only serve legitimate requests.

[0066] The first level of the multi-level throttling control is: generating an object-level cooling key based on the object identifier of the target data object to limit the number of push batches of the target data object within a preset time window. The object-level cooling key is typically composed of "object ID + cooling identifier" and associated with a preset time window parameter. Preferably, a sliding window counter algorithm is used. When it is detected that the number of push batches of a target data object within the current time window has reached the preset cooling threshold, the system will directly reject the write request or place it in a delayed retry queue.

[0067] The second level of the multi-level throttling control is as follows: A sending node-level quota key is generated based on the sending node's node identifier to limit the total push volume of the sending node within a preset period. The sending node-level quota key consists of "sending node ID + quota identifier," and its corresponding preset period is configured according to the system push management strategy. Preferably, a token bucket algorithm is used, and the quota threshold can be dynamically adjusted based on the sending node's reputation level or system load status.

[0068] The third level of the multi-level throttling control is as follows: A receiving node-level throttling key is generated based on the node identifier of the receiving node to limit the number of recommended data snapshots received by the receiving node within a preset time window. The receiving node-level throttling key consists of "receiving node ID + throttling identifier," and the preset time window is configured according to the interaction status of the receiving node. This level can also be adaptively adjusted based on the real-time interaction status of the receiving node; for example, the threshold is relaxed during active browsing and tightened when the background is silent.

[0069] The above three-level flow control constitutes a comprehensive flow protection system. The thresholds at each level can be hot-updated through the configuration center to cope with sudden flow surges. The system also performs embedded monitoring of flow-control trigger events, and automatically triggers alarms or circuit breaker degrade plans when the trigger rate continuously exceeds the safe level.

[0070] Example 6 like Figure 1 As shown, this embodiment provides a recommendation queue data processing device based on multi-dimensional version constraints, including a processor and a memory, and in the latter part of this embodiment, a computer-readable storage medium is provided. This device serves as a physical mapping of the aforementioned method embodiments, and its various modules can be implemented by the processor executing computer program instructions in the memory. The device includes a snapshot construction module 100, a deduplication control module 200, an isolation writing module 300, an expiration cleanup module 400, and a secondary verification module 500.

[0071] The snapshot construction module 100 is used to, in response to the detection that a target data object meets a preset push trigger condition, filter the candidate node set based on spatial constraint rules, determine the receiving node, obtain the object state version and spatial constraint rule version of the target data object, and construct a recommended data snapshot carrying the object state version and the spatial constraint rule version. The snapshot construction module 100 is internally configured with a push trigger detector, a version factor listener, and a serialization encoder. The encoder encapsulates the object information and version number into an immutable recommended data snapshot. The output of this module is connected to the deduplication control module 200.

[0072] The deduplication control module 200 is used to generate a composite deduplication key based on the object identifier of the target data object, the node identifier of the receiving node, the object state version, and the spatial constraint rule version. The deduplication control module 200 receives a snapshot from the snapshot construction module 100, constructs a four-dimensional composite deduplication key through a built-in key generator, and integrates a cache query interface to initiate an existence check to the reverse index storage 800.

[0073] The isolated write module 300, in response to the composite deduplication key not existing in the caching system, writes the recommended data snapshot to the personalized recommendation queue corresponding to the receiving node, and also writes the recommended data snapshot to the global monitoring queue. The isolated write module 300 internally maintains a dual-channel write engine and a transaction coordinator, pushing the snapshot into the personalized recommendation queue 700 and the global monitoring queue 600 respectively. The transaction is only committed after both channels are successful.

[0074] The failure cleanup module 400 is used to perform failure cleanup on the recommended data snapshots in the global monitoring queue and the personalized recommendation queue affected by the attribute change event or the rule change event in response to the detection of an attribute change event of the target data object or a rule change event of the spatial constraint rule. The failure cleanup module 400 includes an event parser, a strategy decision maker, and a cleanup executor. It dynamically selects a cleanup strategy based on a decision matrix and runs through an independent asynchronous processing thread to avoid the cleanup operation blocking the normal write process.

[0075] The secondary verification module 500, in response to a received read request for the personalized recommendation queue, performs a secondary verification on the snapshot of recommendation data in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version. The secondary verification module 500 integrates a version comparator and a re-verification calculator; it only returns data if the re-verification passes; otherwise, it filters the data and triggers asynchronous deletion.

[0076] Regarding the data flow between modules, the snapshot construction module 100 outputs the recommended data snapshot to the deduplication control module 200; after generating a composite deduplication key based on the recommended data snapshot, the deduplication control module 200 outputs the detection results of the recommended data snapshot and the composite deduplication key to the isolated writing module 300; the isolated writing module 300 writes the recommended data snapshot to the global monitoring queue 600 for monitoring by the failure cleanup module 400, and writes the recommended data snapshot to the personalized recommendation queue 700 for reading and verification by the secondary verification module 500.

[0077] This embodiment also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the recommendation queue data processing method based on multi-dimensional version constraints as described in any one of Embodiments 1 to 5 above. The storage medium transforms the data processing logic in the foregoing embodiments into a computer-executable sequence of instructions or a data structure. The present invention does not limit the specific type of the storage medium; it can be any tangible carrier capable of persistently or temporarily storing electronic data, including external storage devices such as semiconductor storage devices, disks, optical disks, and solid-state drives, as well as transmission media for distributing program code in a network environment.

[0078] Example 7 This embodiment uses a flexible employment platform as a specific application scenario to illustrate how the methods described in the preceding embodiments are implemented in actual business. In this embodiment, the mapping relationship between the higher-level concepts in the preceding embodiments and the specific flexible employment scenario is as follows: the target data object corresponds to the new employer task; the sending node corresponds to the employer account; the receiving node corresponds to the worker account; the object status version corresponds to the task status version; the spatial constraint rule version corresponds to the fence rule version; the spatial constraint rule corresponds to the electronic fence rule; the personalized recommendation queue corresponds to the worker personal recommendation queue; and the global monitoring queue corresponds to the global new employer task queue.

[0079] In flexible employment platforms, once an employer posts a new task and it passes the review, the system first determines whether the task qualifies as a new employer task. The criteria for this determination include: the employer's city has enabled the new employer push function; the employer's account registration time does not exceed a preset registration time threshold; and the number of tasks the employer has historically completed does not exceed a preset task quantity threshold. If these conditions are met, a new employer task push event is generated, triggering the subsequent push process.

[0080] In response to a new employer task push event, the system obtains the task status version and fence rule version, and constructs a task recommendation snapshot. The task recommendation snapshot includes fields such as task identifier, employer identifier, city code, task work location coordinates, task status version, fence rule version, task validity period, number of recruits, number of registered applicants, hidden status, start time, and push validity period. This snapshot, as an immutable data structure, records complete state information at the moment the recommendation is generated.

[0081] The system reads a set of candidate workers from the pool of excellent workers, with screening criteria including: the number of completed and settled orders in the past is not less than a preset completion threshold, and the number of negative reviews does not exceed a preset negative review threshold. Subsequently, the system performs multi-stage filtering on the candidate worker set: first, it determines the validity of the location based on the worker's most recently reported location timestamp (if the location timestamp is more than a preset valid location duration, it is considered invalid); then, it performs electronic fence hit detection and order-acceptance distance threshold filtering based on the worker's current coordinates and the coordinates of the task's work location (the order-acceptance distance threshold is configured according to the business scenario); finally, it filters based on the blacklist relationship between the worker and the employer and the worker's current order-acceptance status, generating a set of workers that can be pushed under fence constraints.

[0082] For each worker in the set of workers eligible for recommendations, the system generates a composite deduplication key based on the task identifier, worker identifier, task status version, and fence rule version. If this composite deduplication key does not exist in the cache, an atomic script writes a snapshot of the task recommendation to the worker's personal recommendation queue and global new employer task queue, and stores the composite deduplication key. Both the personal recommendation queue and the global new employer task queue are set with a TTL expiration time consistent with the task's validity period.

[0083] When a worker reads their personal recommendation queue through the precise push entry, the system performs a secondary verification on each task recommendation record in the queue: comparing the task status version in the snapshot with the current task status version, and the fence rule version with the current fence rule version; if the versions are inconsistent, the task status, worker location validity, order acceptance distance, and blacklist relationship are re-verified; if the verification fails or the task has expired, it is marked as invalid and deleted asynchronously.

[0084] When task status, task location, or fence rules change, the system generates a recommendation failure event, updates the corresponding task status version or fence rule version, and locates the affected worker's individual recommendation queue based on the reverse index, performing proactive cleanup, failure marking, or lazy cleanup. Simultaneously, all data display entry points are connected to a unified validity filtering operator to ensure consistent filtering results across all entry points regarding task status, fence rules, blacklist relationships, and recruitment status.

[0085] Through this embodiment, the version constraints, space constraints, dual-queue isolation, deduplication control, secondary verification, and failure cleanup mechanisms described in the preceding embodiments are fully implemented in the new employer task push scenario of the flexible employment platform, effectively solving the technical problems of insufficient exposure of new employer tasks, excessive invalid pushes, dirty data retention in the personal recommendation queue, and inconsistency of data from multiple entry points. The above descriptions are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing recommendation queue data based on multi-dimensional version constraints, applied to a distributed recommendation system, characterized in that, include: Step S100: In response to detecting that the target data object meets the preset push triggering conditions, filter the candidate node set based on the spatial constraint rules, determine the receiving node, obtain the object state version and spatial constraint rule version of the target data object, and construct a recommended data snapshot carrying the object state version and the spatial constraint rule version; Step S200: Generate a composite deduplication key based on the object identifier of the target data object, the node identifier of the receiving node, the object state version, and the spatial constraint rule version. In response to the fact that the composite deduplication key does not exist in the caching system, write the recommended data snapshot to the personalized recommendation queue corresponding to the receiving node, and write the recommended data snapshot to the global monitoring queue. Step S300: In response to detecting an attribute change event of the target data object or a rule change event of the spatial constraint rule, perform invalidation cleanup on the recommendation data snapshots in the global monitoring queue and the personalized recommendation queue that are affected by the attribute change event or the rule change event; Step S400: In response to receiving a read request for the personalized recommendation queue, perform a secondary verification on the recommendation data snapshot in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version, and determine the target display data according to the verification result.

2. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 1, characterized in that, The acquisition of the object state version and spatial constraint rule version of the target data object includes: The object status version is generated based on the attribute fields of the target data object, wherein the attribute fields include at least one of object status, capacity threshold, time attribute, or audit status. The spatial constraint rule version is generated based on the configuration parameters of the spatial constraint rule, wherein the configuration parameters include at least one of the following: geometric boundary data of the geofence, rule priority, distance threshold, or effective time. When any of the attribute fields or configuration parameters changes, an update to the corresponding object state version or space constraint rule version is triggered.

3. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 1, characterized in that, Before writing the recommendation data snapshot to the personalized recommendation queue corresponding to the receiving node, the method further includes: Obtain a set of candidate nodes, and perform multi-stage filtering on the set of candidate nodes based on the spatial constraint rules corresponding to the version of the spatial constraint rules to generate a set of pushable nodes; The multi-stage filtering includes: Filtering based on the location timestamps of candidate nodes to determine location validity; Based on the current coordinates of the candidate nodes and the target coordinates of the target data object, perform spatial constraint hit filtering and distance threshold filtering; Blacklist filtering is performed based on the blocking or prohibition of interaction between candidate nodes and target data objects; The nodes in the set of pushable nodes serve as the receiving nodes.

4. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 1, characterized in that, The step of writing the recommended data snapshot to the personalized recommendation queue corresponding to the receiving node in response to the composite deduplication key not existing in the caching system includes: Detect whether the composite deduplication key exists in the cache system; If the composite deduplication key does not exist, the recommended data snapshot is written to the personalized recommendation queue through an atomic write operation, and the composite deduplication key is stored. While writing to the personalized recommendation queue, a reverse index is constructed from the target data object to the receiving node. The reverse index is used to record the identifier of the receiving node to which the recommendation data snapshot of the target data object is written.

5. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 1, characterized in that, In response to detecting an attribute change event of the target data object or a rule change event of the spatial constraint rule, the method of performing invalidation cleanup on recommendation data snapshots in the global monitoring queue and the personalized recommendation queue that are affected by the attribute change event or the rule change event includes: Based on the number of affected nodes or the urgency of the attribute change event or the rule change event, determine the cleanup strategy; When a proactive cleanup strategy is adopted, the affected personalized recommendation queues are located and the corresponding recommendation data snapshots are deleted. When a failure flag strategy is adopted, the object state version of the target data object or the spatial constraint rule version is updated, and a failure flag is written into the global monitoring queue. When a lazy cleanup strategy is adopted, a snapshot of the recommendation data in the personalized recommendation queue is retained so that deletion can be performed during secondary verification.

6. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 1, characterized in that, The step of performing secondary verification on the recommendation data snapshots in the personalized recommendation queue based on the current object state version and the current spatial constraint rule version includes: Compare the object state version carried in the recommended data snapshot with the current object state version, and compare the spatial constraint rule version carried in the recommended data snapshot with the current spatial constraint rule version; If the versions do not match, a re-verification of the state attributes of the target data object is triggered, or a re-verification of the current coordinates of the receiving node and the spatial constraint rules is triggered. If the re-verification fails, or the recommended data snapshot exceeds the preset validity period, the recommended data snapshot will be marked as invalid or deleted from the personalized recommendation queue.

7. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 6, characterized in that, Also includes: In response to data query requests received through multiple data display entry points, the unified validity filtering operator is invoked; The unified validity filtering operator performs consistency filtering on the candidate data objects corresponding to the multiple data display entry points based on the latest object state version and the latest spatial constraint rule version, so as to intercept candidate data objects that do not meet the display conditions. The multiple data display entry points retain their own data sources and sorting strategies, sharing the filtering logic of the unified validity filtering operator only in determining the validity of the display.

8. The method for processing recommendation queue data based on multi-dimensional version constraints according to claim 1, characterized in that, Before writing the recommendation data snapshot to the personalized recommendation queue, the method further includes multi-level throttling control: An object-level cooldown key is generated based on the object identifier of the target data object to limit the number of push batches of the target data object within a preset time window; A sending node-level quota key is generated based on the sending node's node identifier to limit the total amount of pushes made by the sending node within a preset period; A receiving node-level throttling key is generated based on the node identifier of the receiving node to limit the number of recommended data snapshots received by the receiving node within a preset time window.

9. A recommendation queue data processing device based on multidimensional version constraints, comprising a processor and a memory, characterized in that, When the processor executes the computer program stored in the memory, it implements the recommendation queue data processing method based on multidimensional version constraints as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the recommendation queue data processing method based on multidimensional version constraints as described in any one of claims 1 to 8.

Images