A cache information processing method, device and medium for port platform

Abstract

The present invention provides a cache information processing method, device and medium for a port platform, which belongs to the field of port distributed information processing technology. The method receives an update request, performs authentication and current limiting on the request; obtains a cache key prefix and a data structure type, routes the cache key to a shard node based on a consumption write-back event, performs a write operation in a distributed cache service and sets a lifetime and multi-copy synchronization; delivers a message body to a message middleware and obtains a delivery confirmation; pulls messages from a queue according to the configured concurrency, performs deduplication verification on the idempotence identifier of the message, and performs a data persistence operation after verifying that the message version is higher than the current version of the database; implants an indicator collection point in each module of the entire link, and visualizes the request delay, cache hit rate, and message accumulation depth indicators. The method realizes the rational allocation and optimization of system resources, ensures the consistency and integrity of data, and improves the reliability of message transmission.

Description

Technical Field

[0001] The present invention belongs to the technical field of port distributed information processing, and in particular relates to a cache information processing method, device and medium for a port platform. Background Art

[0002] Currently, the port integration platform has gradually become the data hub that integrates key processes such as ship scheduling, terminal operations, equipment monitoring, and logistics tracking. This platform needs to handle massive amounts of real-time business requests, ensuring efficient data exchange between subsystems while meeting the requirements of sub-second response and high concurrent access.

[0003] To meet port needs, the industry typically uses distributed caching to accelerate access to hotspot data, supplemented by asynchronous message queues to achieve loosely coupled communication between modules. In terms of caching, in-memory data storage servers, such as Redis, are widely used in scenarios such as hotspot data acceleration and distributed locking due to their excellent read and write performance, rich data types supporting strings, lists, hashes, and ordered sets, as well as cluster sharding and high availability.

[0004] The port integration platform's data processing method may concentrate data read and write operations in the database, resulting in slow response speeds and an inability to meet the needs of rapid response in high-concurrency scenarios. Furthermore, the platform's processing capabilities for hotspot data are insufficient, and a large number of requests for hotspot data can overload the storage system, further reducing system performance. In a distributed environment, data is stored and transmitted across multiple nodes, making data inconsistencies prone to occur. Problems such as duplicate message processing and confusing data versions can lead to erroneous or incomplete data in the database. When a system anomaly occurs, the root cause cannot be quickly located, and troubleshooting and repairing the problem takes a long time, impacting the normal operation of the system and user experience. Summary of the Invention

[0005] The present invention provides a cache information processing method for a port platform, which realizes the reasonable allocation and optimization of system resources, ensures the consistency and integrity of data, and improves the reliability of message transmission.

[0006] Methods include:

[0007] S101: Receive an update request containing an order ID, target status, and user credentials, perform authentication and rate limiting on the request, and generate a link ID and initialize a request context for requests that pass rate limiting;

[0008] S102: Parse the link identifier and request body, obtain the cache key prefix and data structure type, perform a write operation in the local memory cache and set a short-term expiration time. After the write is successful, return a response, publish a consumption write-back event, and mark the need to be synchronized to the distributed cache;

[0009] S103: Routing the cache key to the shard node based on the consumer write-back event, performing the write operation in the distributed cache service and setting the expiration time and multi-copy synchronization;

[0010] S104: Construct a message body containing the order ID, status change, version number, idempotence flag, and timestamp based on the cache write result. Select the message exchange type and target queue based on the message partition key, business priority, and queue backlog. Deliver the message body to the message middleware and obtain a delivery confirmation.

[0011] S105: Pull messages from the queue according to the configured concurrency, perform deduplication verification on the idempotence flag of the message, and perform data persistence after verifying that the message version is higher than the current database version;

[0012] S106: Implant indicator collection points in each module of the entire link to visualize the request latency, cache hit rate, and message accumulation depth indicators.

[0013] It should be further explained that step S101 specifically includes:

[0014] When receiving an update request containing an order ID, target status, and user credentials, adjust the authentication and verification strategy based on the business attributes and user role of the request source;

[0015] During the current limiting process, the gateway operation status is collected in real time, and the token generation rate and bucket capacity of the token bucket algorithm are calculated through the rule engine;

[0016] When current limiting is triggered, core business requests are put into a waiting queue and assigned a priority tag;

[0017] When generating a link identifier, the requested service type, user role, and access path are encoded into the prefix of the TraceID to form a structured identifier format;

[0018] When initializing the request context, the TraceID and key metadata are transparently transmitted to the backend SDK and monitoring system; when an exception occurs in a certain link in the link, the TraceID carries the exception information and traces back to the request source.

[0019] It should be further explained that step S102 specifically includes:

[0020] Adjust the short-term lifetime of cache keys based on real-time business load;

[0021] Based on the prefix and data structure type of the cache key, combined with the load status of the distributed cache node, the target shard node of the cache key in the distributed cache is predicted, and the target shard information is marked in the local cache;

[0022] Generate a cache mapping table for each cache key in the local cache, recording its storage location and version number in the local cache and distributed cache;

[0023] Assign priority tags to consumer write-back events based on the business importance of the cache key or the frequency of data updates;

[0024] Step S102 also predicts cache keys that may become hot spots in the future based on the access frequency and historical trends of the current cache keys, and reserves storage space in the distributed cache in advance;

[0025] For predicted hot data, all relevant cache keys are loaded into the distributed cache node, and the loading effect is verified through traffic mirroring technology.

[0026] It should be further explained that step S103 specifically includes:

[0027] When routing cache keys to shard nodes, the routing rules for cache keys are adjusted based on the current load status of the distributed cache nodes. When writing to multiple replicas, each replica is assigned an independent version number. After the write is completed, the version consistency between replicas is verified through the distributed transaction framework.

[0028] Based on historical access logs and current access frequency, predict the cache key set that will become hot in the future.

[0029] It should be further explained that the method is also based on the topological routing label and service type transmitted in step S101. Among them, the core business data adopts cross-region multi-active sharding and is synchronously written to the primary and backup regional nodes; the common business adopts local domain consistent hash sharding. When the target area delay exceeds the threshold in step S106, the shard traffic is switched to the low-latency area.

[0030] Step S103 also activates a backup node in the same region for retrying when the single-node write failure rate exceeds a first preset threshold. When the overall failure rate in the same region exceeds a second preset threshold, cross-layer linkage is triggered: an instruction is sent to step S102 to extend the local cache life cycle; the request is directed to the database read-only replica associated with step S105, and static data is returned; the failed area is marked as a vulnerable area, and non-core business writes are suspended;

[0031] According to the node device implanted in step S106, high-load node sharding is avoided; encrypted snapshots are generated for continuously failed data and stored in the cold backup storage pool, and then multi-region synchronous writing is forced to be enabled; and a credit deduction signal is fed back to S101 to limit the source request.

[0032] It should be further explained that step S104 specifically includes:

[0033] When constructing the message body, a message routing feature vector is generated based on the business entity type associated with the order identifier. The feature vector is matched with the preset routing rule library to adjust the message's Exchange type and target queue priority.

[0034] Layered compression is performed on the message body containing the state change history: dictionary encoding is used to compress common field values, and the compressed message body is delivered to the message middleware through an independent channel;

[0035] Establish a cross-middleware disaster recovery channel for message delivery. When the primary message middleware fails, the message is switched to the corresponding topic of the backup middleware, and the message format is adjusted according to the protocol characteristics of the backup middleware.

[0036] A message aging timestamp field is added to the local Retry queue. For messages that have not been successfully delivered after the preset aging time, the business compensation logic is triggered: the database is queried based on the idempotent identifier in the message. If the corresponding business operation has been completed, the message is marked as processed and discarded.

[0037] It should be further explained that step S105 specifically includes:

[0038] During the message pulling phase, the number of concurrent consumers of the consumer instance is increased according to the peak hours of port operations, and restored to the default value during off-peak hours to match business traffic fluctuations;

[0039] During the idempotence verification phase, the state change path in the message is compared with the previous state recorded in the database to see if it is consistent.

[0040] Before data is persisted, the version number in the message is checked against the current version number in the database. If the difference exceeds the preset threshold, the message is considered outdated and processing is skipped.

[0041] When a message is written to the compensation queue, the cache key and link identifier corresponding to the message are associated to generate a diagnostic data packet containing the operation time, number of failures, and database lock status, so that the compensation task can obtain the lock status information first when retrying.

[0042] It should be further explained that, in this method, based on the real-time message accumulation depth and consumer processing delay indicators issued by S106, the concurrency is linearly increased when the accumulation depth exceeds the threshold, and the batch prefetch mode is enabled when the delay exceeds the threshold; independent consumption channels are allocated to high-credit core businesses to isolate non-core business traffic impacts;

[0043] The core business status change definition verifies the version number continuity; non-core business is set to skip intermediate versions; when a version number gap is detected, query the distributed cache in step S103 to obtain the latest data to complete the version chain;

[0044] When parsing the topology routing tag embedded in the message, the method routes to the database replica in the same region; if the target replica is unavailable, it switches to the nearest low-latency replica based on the defined regional health topology map; the processing result is returned to S106, and the tracking status is updated by associating the full link identifier;

[0045] The method also collects real-time data of the local cache hit rate in step S102, the distributed cache write delay in step S103, the message delivery success rate in step S104, and the consumer processing delay in step S105, and calculates a comprehensive business health score based on (hit rate × 20%) + (1 / delay × 30%) + (success rate × 30%) + (1 / delay × 20%); when the score is lower than the safety threshold, a global warning is triggered.

[0046] According to another embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the cache information processing method of the port platform when executing the program.

[0047] According to another embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cache information processing method of the port platform are implemented.

[0048] It can be seen from the above technical solutions that the present invention has the following advantages:

[0049] The port platform's cache information processing method, provided by the present invention, can verify user identity and permissions, preventing unauthorized users from accessing system resources and ensuring system security. Current limiting can also prevent excessive system load or even system crashes caused by excessive traffic, ensuring stable system operation. Data is first written to the local memory cache and a short expiration time is set. Leveraging the local memory cache's fast access speed, requests can be quickly responded to, reducing user waiting time.

[0050] This method routes cache keys to shard nodes based on consumer writeback events, rationally allocating data storage and improving the efficiency of distributed cache usage. Setting expiration times and multi-copy synchronization enhances data availability and durability. Fully loading hotspot data reduces latency when accessing hotspot data, improving overall system performance.

[0051] The present invention constructs a message body containing multiple key information based on the cache write results to ensure that the message carries complete business information; selects the message exchange type and target queue based on multiple conditions to achieve intelligent message delivery and improve message processing efficiency. The message idempotence identifier is deduplicated to prevent repeated processing of the same message resulting in data inconsistency; after verifying the message version, a persistence operation is performed to ensure that the data in the database is the latest valid data and guarantee data consistency; indicator collection points are implanted in the entire link and displayed visually to facilitate operation and maintenance personnel to monitor the system operation status in real time; resource configuration is automatically adjusted when key indicators exceed the threshold, so that the system can optimize resource utilization according to load conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a schematic diagram of the cache information processing system architecture of the port platform;

[0054] Figure 2 This is a flow chart of the cache information processing method of the port platform;

[0055] Figure 3 Schematic diagram of an electronic device. DETAILED DESCRIPTION

[0056] The port platform's cache information processing method, provided in this paper, deeply integrates and optimizes distributed caching and message queues within the port's integrated platform. Through a unified access layer, asynchronous dual-write consistency assurance, multi-level cache pre-warming, message priority routing, and elastic scaling, it achieves a low-latency, high-throughput cache that can smoothly handle burst traffic.

[0057] like Figure 1 It is the overall system architecture for implementing this method, which includes the request access layer, the middleware collaboration layer, the backend persistence layer and the unified monitoring and control layer.

[0058] Specifically, the request access layer includes a gateway and an embedded SDK, serving as a unified entry point for all client requests into the system. The gateway provides unified authentication, rate limiting, downgrade, and log tagging capabilities, while the SDK is integrated into specific business microservices, providing a standard interface for local caching and message encapsulation.

[0059] The middleware collaboration layer, comprised of a distributed cache subsystem and a message queue subsystem, represents the core innovation of this invention. The distributed cache supports a two-level local and remote cache structure, offering high-performance data read and write capabilities. The message queue subsystem supports asynchronous communication, event-driven processing, and peak-shaving and valley-shifting within the system.

[0060] The backend persistence layer, including databases and file storage systems, is used to ensure the final delivery and consistency of critical business data. This layer uses asynchronous processing mechanisms and idempotence judgment to ensure data correctness even in high-concurrency system conditions.

[0061] The unified monitoring and control layer is responsible for monitoring and managing the entire data flow, supporting real-time analysis of information such as cache hit rates, message accumulation, service response delays, and exception alerts, while cooperating with the container platform to achieve scaling and policy adjustments.

[0062] The system architecture design can use cache and messaging as collaborative middleware capabilities rather than isolated components, forming an intelligent middle platform that can perceive the system's operating status and self-regulate, greatly improving the overall system's resilience and resource utilization.

[0063] The following describes in detail the cache information processing method for the port platform involved in this application. Specific details, such as specific system structures and technologies, are provided for illustrative purposes, not for limitation, to facilitate a thorough understanding of the embodiments of this application. However, it should be apparent to those skilled in the art that this application may also be implemented in other embodiments without these specific details.

[0064] It should be understood that when used in this specification, the term "include" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their combinations. The terms "include", "comprises", "has" and their variations are meant to include but not limited to, unless otherwise specifically emphasized.

[0065] The phrase "one embodiment or some embodiments" in this application means that the specific features, structures or characteristics described in the embodiment are included in one or more embodiments of the application. Therefore, the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in other embodiments" and so on in this application do not necessarily refer to the same embodiment, but mean one or more but not all embodiments, unless otherwise specifically emphasized.

[0066] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0067] See also Figure 2 FIG. 1 is a flow chart of a method for processing cache information of a port platform in a specific embodiment, the method comprising:

[0068] Step S101: receiving an update request including an order identifier, a target state, and user credentials, performing authentication and flow control on the request, and generating a link identifier and initializing a request context for requests that pass flow control.

[0069] In some embodiments, upon receiving an update request containing an order identifier, target status, and user credentials, the system first performs authentication to verify the legitimacy of the user credentials, including token validity and permission checks. Simultaneously, throttling is performed to prevent malicious requests or excessive traffic from impacting the system. For requests that pass throttling, a unique link identifier is generated for full-link tracking, and the request context is initialized, storing request-related metadata. This ensures system security and prevents unauthorized access.

[0070] In some specific embodiments, the ship scheduling system or operation platform sends an update request with the order ID, target status and user credentials through the HTTP / GRPC interface.

[0071] Example request format:

[0072] {"orderId":12345,"newStatus":"LOADED","authToken":"xxx"}.

[0073] The gateway verifies the validity of the authToken and applies token bucket throttling based on the user's role, IP address, and interface level. If throttling is triggered, a predefined downgrade response is returned to non-core services, while core services enter a queue. For requests that successfully pass throttling, the gateway generates a unique TraceID and RequestID for subsequent full-link tracking and log aggregation. Request metadata is then sent to the backend SDK and monitoring system. Request metadata can include user information, request source, and service type.

[0074] Step S102: Parse the link identifier and request body, obtain the cache key prefix and data structure type, perform a write operation in the local memory cache and set a short-term lifetime, return a response after the write is successful, publish a consumption write-back event, and mark that it needs to be synchronized to the distributed cache.

[0075] In some embodiments, indicator collection points are embedded in each module throughout the entire link to collect metrics such as request latency, cache hit rate, and message accumulation depth. These metrics are visualized to facilitate operations and maintenance personnel in monitoring system operational status. When key indicators consistently exceed preset thresholds, system resource configuration is adjusted, such as adding server instances or adjusting cache size. Real-time alerts are issued for indicator and service anomalies, and problem location is supported through link identification, allowing for rapid troubleshooting.

[0076] In some specific embodiments, the SDK parses the TraceID and request body, and obtains the cache key prefix and data structure type from the local configuration. The SDK performs a write operation in the local memory cache and sets a short TTL for the entry, which can be set to 5 seconds or 10 seconds for high-speed response to subsequent short-term repeated requests. After the write is successful, the updated response is immediately returned to the gateway cache to reduce the perceived latency. After the local write, the SDK internally publishes a write-back event to the write-back queue, marking that the entry needs to be synchronized to the L2 distributed cache. At the same time, the local cache hit rate and write latency indicators are reported.

[0077] As an implementation of step S102 of the present application, step S102 specifically includes:

[0078] Adjust the short-term lifetime of the cache key based on the real-time business load; predict the target shard node of the cache key in the distributed cache based on the prefix and data structure type of the cache key, combined with the load status of the distributed cache node, and mark the target shard information in the local cache; generate a cache mapping table for each cache key in the local cache, recording its storage location and version number in the local cache and distributed cache; assign priority tags to consumer write-back events based on the business importance of the cache key or the frequency of data updates.

[0079] Step S102 also predicts cache keys that may become hot spots in the future based on the access frequency and historical trends of the current cache keys, and reserves storage space in the distributed cache in advance; for the predicted hot spot data, the relevant cache keys are fully loaded into the distributed cache node, and the loading effect is verified through traffic mirroring technology.

[0080] As can be seen, by adjusting TTL, the system can intelligently balance data freshness and cache hit rate under varying business loads, improving overall performance. Predicting shard nodes and marking shard information reduces data migration and search overhead in the distributed cache, accelerating data access. The cache mapping table enables more accurate and efficient data synchronization and version control between the local and distributed caches. Traffic mirroring verification ensures the correctness of preloading strategies and reduces the risk of misjudgment.

[0081] Step S103: Based on the consumption write-back event, the cache key is routed to the shard node, and the write operation is performed in the distributed cache service and the lifetime and multi-copy synchronization are set; when cache invalidation or write failure occurs, an alternative node retry is executed or a temporary degradation strategy is enabled, and the hot data is fully loaded based on the access frequency.

[0082] In some embodiments, the cache key is routed to the sharding node based on the consumption write-back event, and the node where the data should be stored is determined according to the sharding rules. Write operations are performed in the distributed cache service, and the lifetime and multi-copy synchronization are set to ensure the availability and persistence of the data. When a cache failure or a write failure occurs, an alternative node is retried. If it still fails, a temporary degradation strategy is activated, such as returning to the default value or degrading the service. Hotspot data is fully loaded based on the access frequency to improve the access performance of hotspot data.

[0083] The sharding rules here are based on algorithms such as consistent hashing or range sharding, mapping cache keys to specific nodes. Distributed caches use cluster modes such as RedisCluster and support multi-replica synchronization. Cache invalidation detection is performed through periodic polling or event notification mechanisms. This improves cache availability and reliability, and multi-replica synchronization ensures data is not lost.

[0084] As an implementation method of step S103, it specifically includes: when routing cache keys to shard nodes, adjusting the routing rules of cache keys based on the current load status of distributed cache nodes, and when writing multiple copies, assigning an independent version number to each copy, and verifying the version consistency between copies through the distributed transaction framework after writing is completed; based on historical access logs and current access frequency, predicting the cache key set that will become a hot spot in the future.

[0085] When routing cache keys to shard nodes, the system collects real-time load data from each node in the distributed cache cluster, including metrics such as CPU usage, memory usage, and network I / O. Based on these metrics, cache key routing rules are adjusted, for example, prioritizing new cache keys to nodes with lower loads to avoid excessive traffic concentration. When writing to multiple replicas, each replica is assigned a unique version number that includes a timestamp and sequence number. After the write is complete, version consistency across all replicas is verified using distributed transaction frameworks such as Seata or TCC mode, ensuring data synchronization across replicas. The system analyzes historical access logs to extract information such as cache key access patterns and frequency trends. Combined with current access frequency data, time series analysis and machine learning algorithms such as the LRU modified algorithm or a popularity prediction model are used to predict cache key sets likely to become popular in the future. This achieves load balancing across the distributed cache cluster, avoids performance bottlenecks caused by overloaded nodes, and improves the throughput and stability of the entire cache system. Independent version numbers and consistency verification mechanisms for multiple replicas ensure strong data consistency across replicas, enhancing data reliability.

[0086] In the example of step S103 of this application, the write-back event is consumed by the cache coordination module, and the key is routed to the corresponding shard node according to the consistent hashing algorithm. The distributed cache service sets a sufficient TTL for the write operation and synchronizes multiple copies. If a short-term failure or node crash occurs during the L2 write process, the module triggers the alternative node retry mechanism. When large-scale cache failure is detected or the write failure rate exceeds the threshold, the cache coordination module will activate a temporary degradation strategy to directly direct the read and write requests to the backend database or return static content.

[0087] This embodiment is based on access frequency monitoring. When the number of accesses to a key exceeds the preheating threshold, the distributed cache subsystem will proactively load the full amount of data for the key to all cache nodes to prevent concurrent pressure during centralized access.

[0088] Step S104: Construct a message body containing the order ID, status change, version number, idempotence ID and timestamp based on the cache write result, select the message exchange type and target queue based on the message partition key, business priority and queue backlog status, deliver the message body to the message middleware and obtain delivery confirmation; if the delivery fails, cache it to the local retry queue and perform a timed retry.

[0089] In some embodiments, a message body containing an order identifier, status change, version number, idempotence flag, and timestamp is constructed based on the cache write result. The message exchange type and target queue are selected based on the message partition key, service priority, and queue backlog. The message body is then delivered to the message middleware and a delivery confirmation is obtained. If delivery fails, the message is cached in a local retry queue and a timed retry is performed to ensure successful delivery. This achieves system decoupling, with different modules communicating asynchronously through the message middleware.

[0090] In some specific embodiments, service message construction and routing are implemented based on the following aspects:

[0091] (1) Message content assembly

[0092] The SDK constructs a message body based on the cache write result, which includes: order ID, status before and after the update, operation version number, idempotence identifier UUID, and timestamp.

[0093] For example:

[0094] {

[0095] "orderId":12345,

[0096] "oldStatus":"LOADING",

[0097] "newStatus":"LOADED",

[0098] "version":42,

[0099] "idempotentKey":"uuid-xyz-123"

[0100] }

[0101] (2) Routing decision

[0102] The routing module selects the appropriate exchange type and target queue based on the partition key in the message content, the business priority, and the current queue backlog. Business priorities can be divided into core and non-core businesses.

[0103] The system supports the issuance of routing rules: when high-priority business breaks out, some non-critical messages can be temporarily redirected to the backup queue to relieve the pressure on the main channel.

[0104] (3) Message delivery and confirmation

[0105] The message is delivered to the middleware via RabbitMQ / KafkaSDK and receives delivery confirmation from the Broker side.

[0106] If message delivery fails, the SDK caches the message in the local Retry queue and performs a scheduled retry. If multiple retries still fail, the message enters the dead letter queue and generates an alarm.

[0107] Step S105: Pull messages from the queue according to the configured concurrency, perform deduplication verification on the idempotence flag of the message, and perform data persistence operation after verifying that the message version is higher than the current database version; if the operation fails, write the message to the compensation queue and record the error log.

[0108] In some embodiments, messages are pulled from the queue according to the configured concurrency, and the message's idempotency flag is verified for deduplication to prevent duplicate processing. After verifying that the message version is higher than the current database version, data persistence is performed to ensure data consistency. If the operation fails, the message is written to a compensation queue and an error log is recorded to facilitate subsequent investigation and resolution. This ensures idempotence of data processing and avoids data inconsistencies caused by duplicate processing. Eventual data consistency is ensured, and version verification ensures that the latest data is persisted.

[0109] In some specific embodiments, the asynchronous consumption and data landing of step S105 are specifically carried out as follows:

[0110] (1) Consumer pull and concurrency control

[0111] The consumer instance pulls messages from the queue based on the configured concurrency. Before consuming, the consumer performs a deduplication check on the message's idempotentKey to avoid repeated processing of the same message.

[0112] (2) Business processing and persistence

[0113] After confirming the message's idempotence, the consumer performs a data version check: the status update is performed only if the message version is higher than the current version in the database; otherwise, the update is skipped. The persistence operation is atomically guaranteed through a database transaction, and the status field and version number of the business table are updated.

[0114] (3) Compensation and error handling

[0115] If a database operation or transaction fails, the consumer writes the message to the compensation queue and records the error log. The compensation task periodically scans the compensation queue and, based on the log and alarm information, attempts to retry or initiate manual intervention.

[0116] As step S105, it also involves increasing the concurrent consumption number of consumer instances according to the peak period of port operations during the message pulling stage, and restoring the default value during non-peak periods to match the business traffic fluctuations; in the idempotence verification link, comparing whether the state change path in the message is consistent with the previous state recorded in the database; before the data is persisted, performing a difference check on the version number in the message and the current version number of the database. If the difference exceeds the preset threshold, it is determined to be an outdated message and skipped for processing; when the message is written to the compensation queue, the cache key and link identifier corresponding to the message are associated to generate a diagnostic data packet containing the operation time, number of failures, and database lock status, so that the compensation task can obtain the lock status information first when retrying.

[0117] As can be seen, during the message pull phase, the system adjusts the number of concurrent consumers per consumer instance based on the pre-set peak port operation hours. By monitoring current traffic data and comparing it with historical peak period data, the system increases the number of consumer instances or the number of concurrent threads per instance during peak periods to accelerate message processing. During off-peak hours, the system restores the default configuration to avoid wasting resources.

[0118] During the idempotency verification phase, the system not only verifies the message's idempotency flag but also compares the state change path carried in the message with the previous state recorded in the database. For example, an order's status can only change from "Created" to "Paid." If the state change path in the message is "Created -> Completed," the change is considered illegal and rejected. Before persisting data, the system extracts the version number in the message and compares it with the version number currently recorded in the database for a difference check. If the difference exceeds a preset threshold, the message is considered outdated, possibly due to a processing delay caused by factors such as network latency. In this case, the system skips processing the message to prevent older data from overwriting newer data. When a message is written to the compensation queue, the system associates and stores the message's corresponding cache key and link identifier, and generates a diagnostic data packet containing information such as the operation time, number of failures, and database lock status.

[0119] When retrying, the compensation task can first obtain the lock status information to determine whether the failure is due to a database lock conflict, and then adopt the corresponding retry strategy, such as delayed retry or priority lock acquisition.

[0120] Step S106: Metric collection points are embedded in all modules throughout the entire link, visualizing metrics such as request latency, cache hit rate, and message backlog depth. System resource allocation is adjusted when key metrics consistently exceed preset thresholds. Real-time alerts are issued for metric and service anomalies, and problem location is supported through link identification.

[0121] In some embodiments, indicator collection points are implanted in each module of the entire link to collect indicators such as request latency, cache hit rate, and message accumulation depth. These indicators are displayed visually to facilitate operation and maintenance personnel to monitor the operating status of the system. When key indicators continue to exceed the preset threshold, adjust the system resource configuration, such as adding server instances or adjusting the cache size. Real-time alarms are issued for indicator anomalies and service anomalies, and problem location is supported through link identification to quickly find the cause of the fault. Achieve system observability and timely detect system anomalies through indicator visualization. Improve system elasticity and adjust resource configuration to cope with traffic changes. Quickly locate and solve problems, link tracking supports full-link problem troubleshooting.

[0122] In some specific embodiments, step S106 may also implement monitoring of early warning and flexible governance processing processes.

[0123] Specifically, Prometheus tracking can be implemented across the entire chain, including the gateway, SDK, cache coordination module, and consumer side, to collect data such as request latency, cache hit rate, message accumulation depth, consumer processing rate, and failure rate. This data is displayed on a visualization platform, and custom dashboards are supported.

[0124] When key metrics, such as message backlog depth > 5000, consumer latency > 200ms, or L2 write latency > 50ms, consistently exceed preset thresholds, the system invokes the container orchestration platform API to add cache nodes or consumer instances. Elastic scaling actions are logged in audit logs to track capacity adjustment history.

[0125] In this embodiment, the monitoring system issues real-time alerts via email, SMS, DingTalk, and other channels when indicators suddenly increase or services become abnormal. Operations and maintenance personnel can view the TraceID in the alert details to quickly locate the specific request link causing the problem and use the link tracking panel to diagnose and repair it.

[0126] In an embodiment of the present invention, based on step S101, a possible embodiment will be given below to illustrate its specific implementation scheme in a non-limiting manner.

[0127] Step S101 specifically includes: when receiving an update request containing an order identifier, target status, and user credentials, adjusting the authentication and verification strategy based on the business attributes and user role of the request source; in the current limiting processing stage, collecting the gateway data processing volume and interface response time in real time, and calculating the token generation rate and bucket capacity of the token bucket algorithm through the rule engine; for example, when the system load is high, reducing the token generation rate to protect the backend service, and increasing the rate to improve throughput when the load is low.

[0128] When rate limiting is triggered, core business requests are queued and assigned a priority tag. When generating a link identifier, the requested business type, user role, and access path are encoded in the TraceID prefix, forming a structured identification format. This identifier is used for rapid classification and exception tracing during subsequent log aggregation. When initializing the request context, the TraceID, user ID, and request source IP address are transparently transmitted to the backend SDK and monitoring system. If an exception occurs at any point in the link, the TraceID, carrying the exception information, is traced back to the request source.

[0129] This ensures both system security and flexibility, protecting core services while providing a differentiated service experience. It effectively balances system stability and throughput, preventing service crashes due to overload. Core service requests are prioritized, ensuring critical services are not impacted by traffic restrictions and improving the user experience. TraceID allows for quick correlation of logs across the entire process, reducing troubleshooting time.

[0130] In an embodiment of the present invention, based on step S104, a possible embodiment will be given below to illustrate its specific implementation scheme in a non-limiting manner.

[0131] Step S104 specifically includes:

[0132] When constructing the message body, a message routing feature vector is generated based on the business entity type associated with the order identifier, and the feature vector is matched with the preset routing rule library to adjust the message's Exchange type and target queue priority.

[0133] Perform layered compression on the message body containing the state change history: dictionary encoding is used to compress common field values, and the compressed message body is delivered to the message middleware through an independent channel.

[0134] Establish a cross-middleware disaster recovery channel for message delivery. When the primary message middleware fails, the message is switched to the corresponding topic of the backup middleware, and the message format is adjusted according to the protocol characteristics of the backup middleware.

[0135] A message aging timestamp field is added to the local Retry queue. For messages that have not been successfully delivered after the preset aging time, the business compensation logic is triggered: the database is queried based on the idempotent identifier in the message. If the corresponding business operation has been completed, the message is marked as processed and discarded.

[0136] It should be noted that feature vector generation is based on the metadata model of business entities. The system resolves the associated entity type from the order identifier, extracts attribute values, and maps them to the feature space. The routing rule library uses a decision tree algorithm to input the feature vector and output the matching exchange type and queue priority. Layered compression employs a two-level process: the first level uses dictionary encoding to create a mapping table between common values ​​and short codes, and the second level uses the GZIP algorithm for further compression. Independent channels establish dedicated connections through frameworks such as Netty to ensure fast message delivery. Cross-middleware disaster recovery relies on a health check mechanism. The system regularly sends heartbeat packets to the primary middleware, triggering a failover if consecutive failures exceed a threshold. Message format conversion is implemented using the adapter pattern, providing serializers and deserializers for different middleware protocols. Aging message processing is implemented using Redis's SortedSet, using the message ID as a member and the aging time as a score. A scheduled task scans for expired members and invokes a compensation service. The layered compression strategy effectively reduces message size, and the independent channels avoid network congestion, improving message transmission efficiency. The cross-middleware disaster recovery channel enhances system resilience, ensuring that messages are not lost in the event of a middleware failure, thereby improving availability. At the same time, the eventual consistency of data is guaranteed through business compensation logic.

[0137] Furthermore, as a refinement and extension of a specific implementation method of the cache information processing method of the above-mentioned port platform, the method is also based on the topological routing label and business type transmitted in step S101, wherein the core business data adopts cross-regional multi-active sharding and is synchronously written to the primary and backup regional nodes; ordinary business adopts local domain consistent hash sharding, and when the target area delay exceeds the threshold when it is detected in step S106, the shard traffic is switched to the low-latency area.

[0138] Step S103 also enables retry of the backup node in the same area when the single-node write failure rate exceeds the first preset threshold; when the overall failure rate in the same area exceeds the second preset threshold, cross-layer linkage is triggered: an instruction is sent to step S102 to extend the local cache life cycle; the request is directed to the database read-only copy associated with step S105, and static data is returned; and the failed area is marked as a vulnerable area, and non-core business writes are suspended.

[0139] According to the node device implanted in step S106, high-load node sharding is avoided; encrypted snapshots are generated for continuously failed data and stored in the cold backup storage pool, and then multi-region synchronous writing is forced to be enabled; and a credit deduction signal is fed back to S101 to limit the source request.

[0140] This embodiment adopts a cross-region multi-active sharding strategy for core business data based on the topology routing label and service type transmitted by S101. The data is written to the primary and backup regional nodes simultaneously to ensure strong consistency.

[0141] When S106 detects that the latency in the target region exceeds the threshold, the system switches the sharded traffic to the low-latency region to ensure service quality. In S103, when the single-node write failure rate exceeds the first preset threshold, the system activates a retry on the backup node in the same region, attempting to send the request to other available nodes in the same region. If the overall failure rate in the same region exceeds the second preset threshold, the cross-layer linkage mechanism is triggered: a command is sent to S102 to extend the local cache lifecycle, reducing reliance on the distributed cache; the request is directed to the database read-only replica associated with S105, returning static data; and the failed region is marked as a vulnerable area, suspending non-core business writes to protect core business. Based on the node load information sent by S106, the system avoids sharding high-load nodes and directs traffic to low-load nodes. For continuously failed data, encrypted snapshots are generated and stored in the cold backup storage pool to ensure data security. At the same time, multi-region synchronous writes are forcibly enabled to enhance data reliability. The system also sends a credit deduction signal to S101 to limit the access frequency of source requests and prevent malicious or abnormal requests from further affecting the system.

[0142] Furthermore, as an embodiment of the cache information processing method of the port platform, a specific implementation method includes the following steps:

[0143] In this method, based on the real-time message accumulation depth and consumer processing delay indicators issued by S106, the concurrency is linearly increased when the accumulation depth exceeds the threshold, and the batch prefetch mode is enabled when the delay exceeds the threshold; independent consumption channels are allocated to high-credit core businesses to isolate non-core business traffic impacts.

[0144] The core business status change definition verifies the version number continuity; non-core business is set to skip intermediate versions; when a version number gap is detected, the distributed cache is queried in step S103 to obtain the latest data to complete the version chain.

[0145] When parsing the topology routing tag embedded in the message, the method routes to the database replica in the same region; if the target replica is unavailable, it switches to the nearest low-latency replica based on the defined regional health topology map; the processing result is returned to S106, and the associated full-link identifier updates the tracking status.

[0146] The method also collects real-time data of the local cache hit rate in step S102, the distributed cache write delay in step S103, the message delivery success rate in step S104, and the consumer processing delay in step S105, and calculates the comprehensive business health score based on (hit rate × 20%) + (1 / delay × 30%) + (success rate × 30%) + (1 / delay × 20%).

[0147] The hit rate in hit rate × 20% is obtained based on step S102. The delay in (1 / delay × 30%) is obtained based on step S103. The hit rate in (success rate × 30%) is obtained based on step S104. The hit rate in (1 / delay × 20%) is obtained based on step S105.

[0148] When the score falls below the safety threshold, a global alert is triggered. The comprehensive business health score provides a quantitative indicator of the overall system operation status, making the early warning and downgrade strategies more scientific.

[0149] For example, if the business health score is less than a certain score, it can trigger the S101 rate limit upgrade and S103 downgrade write. If the business health score is less than a lower score, a global circuit breaker is activated.

[0150] In some specific implementations, when the stack depth exceeds a preset threshold, consumer concurrency is linearly increased. When processing latency exceeds a threshold, batch prefetching is enabled to pull multiple messages at once to reduce network overhead. For state changes in core services, the system strictly verifies version number continuity.

[0151] When a version number gap is detected, the system queries the distributed cache in S103 for the latest data, completes the version chain, and then performs persistence operations. The system parses the topology routing tag embedded in the message and preferentially routes the request to the database replica in the same region. If the target replica is unavailable, it switches to the nearest low-latency replica based on the predefined regional health topology map. The processing results are transmitted back to S106, which updates the full-link tracking status and implements closed-loop monitoring of the request.

[0152] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0153] like Figure 3 As shown, the present application also provides an electronic device, including a display module 103, a memory 102, a processor 101 and a computer program stored in the memory and executable on the processor 101, wherein the processor 101 implements the steps of the cache information processing method of the port platform when executing the program.

[0154] In the embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or claimed herein.

[0155] In the embodiment of the present application, the processor 101 can be implemented by using at least one of a special purpose integrated circuit, a programmable logic device, a field programmable gate array, a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to perform the functions described herein. In some cases, such an embodiment can be implemented in a controller. For software implementation, an embodiment such as a process or function can be implemented with a separate software module that allows the execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any appropriate programming language, and the software code can be stored in a memory and executed by a controller.

[0156] The display module 103 is used to display information input by the user or information provided to the user. The display module 103 may include a display panel, which may be configured in the form of a liquid crystal display, an organic light emitting diode, etc.

[0157] The memory 102 can be used to store software programs and various data. The memory 102 can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0158] The present application also provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the cache information processing method of the port platform are implemented.

[0159] The storage medium can be any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0160] In the context of storage media, a readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0161] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cache information processing method for a port platform, characterized in that: Methods include: S101: Receive an update request containing an order ID, target status, and user credentials, perform authentication and rate limiting on the request, and generate a link ID and initialize a request context for requests that pass rate limiting; S102: Parse the link identifier and request body, obtain the cache key prefix and data structure type, perform a write operation in the local memory cache and set a short-term expiration time. After the write is successful, return a response, publish a consumption write-back event, and mark the need to be synchronized to the distributed cache; S103: Routing the cache key to the shard node based on the consumer write-back event, performing the write operation in the distributed cache service and setting the expiration time and multi-copy synchronization; S104: Construct a message body containing the order ID, status change, version number, idempotence flag, and timestamp based on the cache write result. Select the message exchange type and target queue based on the message partition key, business priority, and queue backlog. Deliver the message body to the message middleware and obtain a delivery confirmation. S105: Pull messages from the queue according to the configured concurrency, perform deduplication verification on the idempotence flag of the message, and perform data persistence after verifying that the message version is higher than the current database version; S106: Prometheus tracking is embedded in the gateway, SDK, cache coordination module, and consumer end based on the entire link to collect information such as request latency, cache hit rate, message accumulation depth, consumer processing rate, and failure rate. Data is displayed through a visualization platform and supports custom dashboards; Based on the real-time message accumulation depth and consumer processing delay indicators issued by S106, the concurrency is linearly increased when the accumulation depth exceeds the threshold, and the batch prefetch mode is enabled when the delay exceeds the threshold; Allocate independent consumption channels for high-credit core businesses to isolate traffic impacts from non-core businesses; The core business status change definition verifies the version number continuity; non-core business is set to skip intermediate versions; when a version number gap is detected, query the distributed cache in step S103 to obtain the latest data to complete the version chain; The method routes to the same-region database replica when parsing the topology routing tag embedded in the message; If the target replica is unavailable, switch to the nearest low-latency replica based on the defined regional health topology.

2. The cache information processing method of the port platform according to claim 1 is characterized in that: Step S101 specifically includes: When receiving an update request containing an order ID, target status, and user credentials, adjust the authentication and verification strategy based on the business attributes and user role of the request source; During the current limiting process, the gateway operation status is collected in real time, and the token generation rate and bucket capacity of the token bucket algorithm are calculated through the rule engine; When current limiting is triggered, core business requests are put into a waiting queue and assigned a priority tag; When generating a link identifier, the requested service type, user role, and access path are encoded into the prefix of the TraceID to form a structured identifier format; When initializing the request context, the TraceID and key metadata are transparently transmitted to the backend SDK and monitoring system; when an exception occurs in a certain link in the link, the TraceID carries the exception information and traces back to the request source.

3. The cache information processing method of the port platform according to claim 1, characterized in that: Step S102 specifically includes: Adjust the short-term lifetime of cache keys based on real-time business load; Based on the prefix and data structure type of the cache key, combined with the load status of the distributed cache node, the target shard node of the cache key in the distributed cache is predicted, and the target shard information is marked in the local cache; Generate a cache mapping table for each cache key in the local cache, recording its storage location and version number in the local cache and distributed cache; Assign priority tags to consumer write-back events based on the business importance of the cache key or the frequency of data updates; Step S102 also predicts cache keys that may become hot spots in the future based on the access frequency and historical trends of the current cache keys, and reserves storage space in the distributed cache in advance; For predicted hot data, all relevant cache keys are loaded into the distributed cache node, and the loading effect is verified through traffic mirroring technology.

4. The cache information processing method of the port platform according to claim 1, characterized in that: Step S103 specifically includes: When routing cache keys to shard nodes, the routing rules for cache keys are adjusted based on the current load status of the distributed cache nodes. When writing to multiple replicas, each replica is assigned an independent version number. After the write is completed, the version consistency between replicas is verified through the distributed transaction framework. Based on historical access logs and current access frequency, predict the cache key set that will become hot in the future.

5. The cache information processing method of the port platform according to claim 1, characterized in that: Step S104 specifically includes: When constructing the message body, a message routing feature vector is generated based on the business entity type associated with the order identifier. The feature vector is matched with the preset routing rule library to adjust the message's Exchange type and target queue priority. Layered compression is performed on the message body containing the state change history: dictionary encoding is used to compress common field values, and the compressed message body is delivered to the message middleware through an independent channel; Establish a cross-middleware disaster recovery channel for message delivery. When the primary message middleware fails, the message is switched to the corresponding topic of the backup middleware, and the message format is adjusted according to the protocol characteristics of the backup middleware. A message aging timestamp field is added to the local Retry queue. For messages that have not been successfully delivered after the preset aging time, the business compensation logic is triggered: the database is queried based on the idempotent identifier in the message. If the corresponding business operation has been completed, the message is marked as processed and discarded.

6. The cache information processing method of the port platform according to claim 1, characterized in that: Step S105 specifically includes: During the message pulling phase, the number of concurrent consumers of the consumer instance is increased according to the peak hours of port operations, and restored to the default value during off-peak hours to match business traffic fluctuations; In the idempotence verification phase, the state change path in the message is compared with the previous state recorded in the database to see if it is consistent; Before data is persisted, the version number in the message is checked against the current version number in the database. If the difference exceeds the preset threshold, the message is considered outdated and processing is skipped. When a message is written to the compensation queue, the cache key and link identifier corresponding to the message are associated to generate a diagnostic data packet containing the operation time, number of failures, and database lock status, so that the compensation task can obtain the lock status information first when retrying.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the cache information processing method of the port platform according to any one of claims 1 to 6 are implemented.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cache information processing method of the port platform according to any one of claims 1 to 6 are implemented.

Images