Cache information processing method and equipment for port platform, and medium

Abstract

The invention provides a cache information processing method and device of a port platform and a medium, and belongs to the technical field of port distributed information processing. An update request is received, and authentication and flow limiting processing are performed on the request; obtaining a cache key prefix and a data structure type, routing a cache key to a fragment node based on a consumption write-back event, executing a write-in operation in a distributed cache service, and setting survival time and multi-copy synchronization; delivering the message body to the message middleware and obtaining delivery confirmation; the messages are pulled from the queue according to the configured concurrency, deduplication verification is carried out on idempotent identifiers of the messages, and data persistence operation is executed after the verification message version is higher than the current version of the database; index collection points are implanted into all modules of the whole link, and the request delay, the cache hit rate and the message accumulation depth index are visually displayed. The reasonable distribution and optimization of system resources are realized; the consistency and integrity of the data are guaranteed, and the reliability of message transmission is improved at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of port distributed information processing, and particularly relates to a method, device and medium for processing cached information of a port platform. Background Technique

[0002] At present, the port integration platform has gradually become a data center integrating key links such as ship scheduling, terminal operation, equipment monitoring, and logistics tracking. This platform needs to process a large number of real-time service requests, not only to ensure efficient data exchange between subsystems, but also to meet the requirements of second-level response and high-concurrency access.

[0003] To meet port requirements, the industry usually uses distributed caching to accelerate access to hot data, and supplements it with an asynchronous message queue to achieve loose coupling communication between modules. In terms of caching, in-memory storage data structure servers represented by Redis are widely used in scenarios such as hot data acceleration and distributed locks due to their excellent read and write performance, rich data type support (strings, lists, hashes, sorted sets, etc.), and cluster sharding and high-availability guarantees.

[0004] In the data processing mode of the port integration platform, data read and write operations may all be concentrated in the database, resulting in slow response speed and inability to meet the requirements of fast response in high-concurrency scenarios. At the same time, the processing ability of hot data is insufficient, and a large number of requests for hot data will cause too high a load on the storage system, further reducing system performance. In a distributed environment, data is stored and transmitted on multiple nodes, and data inconsistency is likely to occur. Problems such as duplicate message processing and data version chaos will cause data errors or incompleteness in the database. When the system has an exception, it is impossible to quickly locate the root cause of the problem, and it takes a long time to troubleshoot and repair the fault, affecting the normal operation of the system and user experience. Summary of the Invention

[0005] The present invention provides a method for processing cached information of a port platform, which realizes reasonable allocation and optimization of system resources; ensures data consistency and integrity, and at the same time improves the reliability of message transmission.

[0006] The method includes: S101: Receive an update request containing an order identifier, a target status, and a user credential, perform authentication and rate limiting processing on the request, and generate a link identifier and initialize a request context for the request that passes the rate limit; S102: Parse the link identifier and the request body, obtain the cache key prefix and the data structure type, perform a write operation in the local memory cache and set a short-term survival time, return a response after successful writing, and at the same time publish a consumption write-back event and mark that it needs to be synchronized to the distributed cache; S103: Route the cache key to the shard node based on the consumption write-back event, perform a write operation in the distributed cache service, and set the survival time and multi-copy synchronization; S104: Construct a message body containing the order identifier, status change, version number, idempotency identifier, and timestamp based on the cache write result. Select the message exchange type and target queue according to the message partition key, business priority, and queue backlog status, and deliver the message body to the message middleware and obtain the delivery confirmation; S105: Pull messages from the queue according to the configured concurrency, perform deduplication verification on the idempotency identifier of the message, and perform data persistence operations after verifying that the message version is higher than the current version of the database; S106: Insert metric collection points into each module of the full link, and visually display metrics such as request latency, cache hit rate, and message backlog depth.

[0007] Furthermore, it should be noted that step S101 specifically includes: When receiving an update request containing the order identifier, target status, and user credentials, adjust the authentication verification policy based on the business attributes and user roles of the request source; During the traffic limiting process, collect the running status of the gateway in real time, and calculate the token generation rate and bucket capacity of the token bucket algorithm through the rule engine; When traffic limiting is triggered, core business requests enter the queuing waiting queue and are assigned priority tags; When generating the link identifier, encode the business type, user role, and access path of the request into the prefix of the TraceID to form a structured identifier format; When initializing the request context, pass the TraceID and key metadata to the backend SDK and monitoring system; when an exception occurs in a certain link of the link, the TraceID carries the exception information back to the request source.

[0008] Furthermore, it should be noted that step S102 specifically includes: Adjust the short-term survival time of the cache key based on the real-time business load; Based on the prefix of the cache key and the data structure type, combined with the load status of the distributed cache node, predict the target shard node of the cache key in the distributed cache, and mark the target shard information in the local cache; Generate a cache mapping table for each cache key in the local cache, and record its storage location and version number in the local cache and the distributed cache; Assign a priority tag to the consumption write-back event according to the business importance or data update frequency of the cache key; Step S102 also predicts cache keys that may become hotspots in the future based on the access frequency and historical trends of the current cache key, and reserves storage space in the distributed cache in advance; For the predicted hot data, all relevant cache keys are fully loaded into the distributed cache nodes, and the loading effect is verified through traffic mirroring technology.

[0009] It should be further noted that step S103 specifically includes: When routing the cache key to the shard node, based on the current load status of the distributed cache node, adjust the routing rule of the cache key, and when writing multiple replicas, assign an independent version number to each replica, and verify the version consistency between replicas through the distributed transaction framework after writing is completed; Predict the set of cache keys that will become hotspots within a certain period of time in the future based on historical access logs and the current access frequency.

[0010] It should be further noted that the method is also based on the topology routing label and business type transmitted in step S101. Among them, core business data adopts cross-region multi-active sharding and is synchronously written to the primary and standby regional nodes; ordinary business adopts local domain consistent hashing sharding, and when the target area delay exceeds the threshold in step S106, the sharding traffic is switched to the low-latency area; When the single-node write failure rate exceeds the first preset threshold in step S103, enable the standby node in the same region to retry; when the overall failure rate in the same region exceeds the second preset threshold, trigger cross-layer linkage: send an instruction to step S102 to extend the local cache survival period; direct the request to the read-only replica of the database associated with step S105 and return the staticized data; and mark the failed area as a vulnerable area and suspend the writing of non-core services; According to the node device implanted in step S106, avoid sharding of high-load nodes; perform encryption snapshot generation on continuously failed data and store it in the cold backup storage pool, and then force multi-region synchronous writing; and feedback a credit deduction signal to S101 to limit the source request.

[0011] It should be further noted that step S104 specifically includes: When constructing the message body, generate a message routing feature vector according to the business entity type associated with the order identifier, match the feature vector with the preset routing rule library, and adjust the Exchange type and target Queue priority of the message; Perform hierarchical compression on the message body containing the status change history: use dictionary encoding to compress the common field values, and the compressed message body is delivered to the message middleware through an independent channel; Establish a cross-middleware disaster recovery channel for message delivery. When the primary message middleware fails, switch the message to the corresponding Topic of the standby middleware, and adjust the message format according to the protocol characteristics of the standby middleware; Add a message aging timestamp field to the local Retry queue. For messages that have not been successfully delivered after exceeding the preset aging time, trigger the business compensation logic: query the database based on the idempotency identifier in the message. If the corresponding business operation has been completed, mark the message as processed and discard it.

[0012] Further, it should be noted that step S105 specifically includes: In the message pulling stage, increase the concurrent consumption quantity of consumer instances according to the peak port operation period, and restore the default value during non-peak periods to match the business traffic fluctuations; In the idempotency verification link, compare whether the status change path in the message is consistent with the previous status recorded in the database; Before data persistence, perform a difference verification on the version number in the message and the current version number of the database. If the difference exceeds the preset threshold, determine the message as an outdated message and skip the processing; When the message is written into the compensation queue, associate the cache key corresponding to the message with the link identifier, and generate a diagnostic data packet containing the operation time, failure times, and database lock status for the compensation task to preferentially obtain the lock status information during retry.

[0013] Further, it should be noted that in the method, based on the real-time message accumulation depth and consumer processing delay metrics issued by S106, when the accumulation depth exceeds the threshold, linearly increase the concurrency, and when the delay exceeds the threshold, enable the batch prefetch mode; allocate an independent consumption channel for high-credit core services to isolate the impact of non-core business traffic; The core business status change defines the continuity of the version number; the non-core business is set to skip intermediate versions; when a version number fault is detected, query the distributed cache in step S103 to obtain the latest data to complete the version chain; When the method parses the topology routing label embedded in the message, route it to the database replica in the same region; if the target replica is unavailable, switch to the nearest low-latency replica according to the defined regional health topology map; the processing result is returned to S106, and the tracking status is updated by associating the full-link identifier. The method also collects real-time data on the local cache hit rate in step S102, the distributed cache write delay in step S103, the message delivery success rate in step S104, and the consumer processing delay in step S105, and calculates the comprehensive business health score based on (hit rate × 20%) + (1 / delay × 30%) + (success rate × 30%) + (1 / delay × 20%); when the score is lower than the safety threshold, trigger a global warning.

[0014] According to another embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the cache information processing method of the port platform are implemented.

[0015] According to still another embodiment of the present application, a storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cache information processing method of the port platform are implemented.

[0016] As can be seen from the above technical solutions, the present invention has the following advantages: The cache information processing method of the port platform provided by the present invention can verify the user identity and permissions, prevent illegal users from accessing system resources, and ensure system security; the traffic limiting process can avoid the system load being too high or even crashing due to excessive traffic, and ensure the stable operation of the system. Writing data into the local memory cache first and setting a short survival time, taking advantage of the fast access speed of the local memory cache, can quickly respond to requests and reduce the user waiting time.

[0017] The present invention routes cache keys to shard nodes based on consumption write-back events, reasonably allocates data storage, and improves the utilization efficiency of distributed caches. Setting the survival time and multi-copy synchronization enhances the availability and persistence of data. Loading hot data in full can reduce the latency when accessing hot data and improve the overall system performance.

[0018] The present invention constructs a message body containing various key information according to the cache write result to ensure that the message carries complete business information; selects the message exchange type and target queue according to various conditions to realize intelligent message delivery and improve message processing efficiency. Duplicate checking is performed on the message idempotency identifier to prevent data inconsistency caused by repeated processing of the same message; after verifying the message version, a persistence operation is executed to ensure that the data in the database is the latest and valid data, guaranteeing data consistency; metric collection points are implanted in the entire link and visually displayed, facilitating operation and maintenance personnel to monitor the system operation status in real time; when key metrics exceed the threshold, the resource configuration is automatically adjusted, enabling the system to optimize resource utilization according to the load situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the present invention, the drawings required to be used in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic diagram of the architecture of the cache information processing system for the port platform; Figure 2 Flowchart of the caching information processing method for the port platform; Figure 3 Schematic diagram of an electronic device. Detailed implementation manners

[0021] The caching information processing method for the port platform provided by the present invention is a method for deeply integrating and optimizing distributed caching and message queues in the port integration platform. Through technical means such as a unified access layer, asynchronous dual-write consistency guarantee, multi-level cache preheating, message priority routing, and elastic scaling, a cache with low latency, high throughput, and the ability to smoothly handle burst traffic is achieved.

[0022] As shown in Figure 1 is the overall system architecture for implementing this method, which are respectively a request access layer, a middleware coordination layer, a backend persistence layer, and a unified monitoring and control layer.

[0023] Specifically, the request access layer includes a gateway and an embedded SDK, serving as the unified entry for all client requests to enter the system. The gateway has functions such as unified authentication, traffic limiting, degradation, and log tagging, while the SDK is integrated into specific business microservices to provide standard interfaces for local caching and message encapsulation.

[0024] The middleware coordination layer consists of a distributed caching subsystem and a message queue subsystem, which is the core innovation area of the present invention. The distributed caching supports a two-level cache structure of local + remote, with high-performance data reading and writing capabilities; the message queue subsystem supports asynchronous communication, event-driven, and peak shaving and valley filling processing within the system.

[0025] The backend persistence layer includes a database and a file storage system, which are used to complete the final landing and consistency guarantee of key business data. This layer ensures data correctness under high concurrency of the system through an asynchronous processing mechanism and idempotency judgment.

[0026] The unified monitoring and control layer is responsible for the monitoring and governance of the full-link data stream, supporting real-time analysis of information such as cache hit rate, message backlog, service response latency, and exception alerts, and at the same time cooperating with the container platform to achieve scaling and policy adjustment.

[0027] The system architecture design can regard caching and messaging as collaborative middleware capabilities rather than isolated components, forming an intelligent middle platform that can perceive the system operation status and self-regulate, greatly improving the resilience and resource utilization rate of the overall system.

[0028] The method for processing cached information of the port platform involved in the present application will be described in detail below. For the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details.

[0029] It should be understood that when used in the specification of the present application, the term "including" indicates the existence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations. The terms "including", "comprising", "having", and their variants all mean including but not limited to, unless otherwise specifically emphasized in other ways.

[0030] Statements such as "in one embodiment described in the present application" or "in some embodiments" mean that the specific features, structures, or characteristics described in the embodiment are included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in the present application do not necessarily all refer to the same embodiment, but mean one or more but not all of the embodiments, unless otherwise specifically emphasized in other ways.

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0032] Please refer to Figure 2 The following is a flowchart of the method for processing cached information of the port platform in a specific embodiment. The method includes: Step S101: Receive an update request containing an order identifier, a target status, and a user credential, perform authentication and rate limiting processing on the request, and generate a link identifier and initialize a request context for the request that passes the rate limiting.

[0033] In some embodiments, after the system receives an update request containing an order identifier, a target status, and a user credential, it first performs authentication processing to verify the legality of the user credential, including token validity, permission check, etc. At the same time, rate limiting processing is performed to prevent malicious requests or excessive traffic from impacting the system. For the requests that pass the rate limiting, a unique link identifier is generated for full-link tracing, and the request context is initialized to store request-related metadata. Ensure system security and prevent unauthorized access.

[0034] In some specific embodiments, the ship scheduling system or operation platform sends an update request with an order ID, a target status, and user credentials through an HTTP / GRPC interface.

[0035] Example of the request format: {"orderId":12345,"newStatus":"LOADED","authToken":"xxx"}.

[0036] The gateway verifies the legitimacy of the authToken and performs token bucket rate limiting according to the user's role, IP address, and interface level. If rate limiting is triggered, a predefined degradation response is returned for non-core services, and core services enter the queue for waiting. For requests that successfully pass the rate limiting, the gateway generates a unique TraceID and RequestID for subsequent full-link tracking and log aggregation. The request metadata is sent to the backend SDK and monitoring system. The request metadata can be user information, request source, and business type.

[0037] Step S102: Parse the link identifier and request body, obtain the cache key prefix and data structure type, perform a write operation in the local memory cache and set a short-term survival time. After the write is successful, return a response, publish a consumption write-back event at the same time, and mark that it needs to be synchronized to the distributed cache.

[0038] In some embodiments, metric collection points are implanted in each module of the full link to collect metrics such as request latency, cache hit rate, and message backlog depth. Visualize these metrics to facilitate system operation status monitoring by operation and maintenance personnel. When key metrics continuously exceed the preset thresholds, adjust the system resource configuration, such as adding server instances or adjusting the cache size. Issue real-time alarms for metric anomalies and service anomalies, and support problem location through the link identifier to quickly troubleshoot the cause of the failure.

[0039] In some specific embodiments, the SDK parses the TraceID and request body, and obtains the prefix of the cache Key and the data structure type from the local configuration. The SDK performs a write operation in the local memory cache and sets a short-term TTL for this entry, which can be set to 5 seconds, 10 seconds, etc. for high-speed response to subsequent short-term repeated requests. After the write is successful, immediately return a cache updated response to the gateway to reduce latency perception. After local writing, the SDK publishes a write-back event to the write-back queue internally, marking that this entry needs to be synchronized to the L2 distributed cache. At the same time, report the local cache hit rate and write latency metrics.

[0040] As an implementation manner of step S102 of the present application, step S102 specifically includes: Adjust the short survival time of the cache key based on the real-time business load; according to the prefix of the cache key and the data structure type, combined with the load status of the distributed cache nodes, predict the target shard node of the cache key in the distributed cache, and mark the target shard information in the local cache; generate a cache mapping table for each cache key in the local cache, recording its storage location and version number in the local cache and the distributed cache; assign a priority label to the consumption write-back event according to the business importance of the cache key or the data update frequency.

[0041] Step S102 also predicts the cache keys that may become hotspots in the future based on the access frequency and historical trend of the current cache key, and reserves storage space in the distributed cache in advance; for the predicted hot data, load all relevant cache keys to the distributed cache nodes, and verify the loading effect through traffic mirroring technology.

[0042] It can be seen that by adjusting the TTL, the system can intelligently balance data freshness and cache hit rate under different business loads, improving the overall performance. Predicting the shard node and marking the shard information reduces the migration and lookup overhead of data in the distributed cache, accelerating data access. The existence of the cache mapping table makes the synchronization and version control of data between the local and distributed caches more accurate and efficient. The traffic mirroring verification ensures the correctness of the preloading strategy and reduces the risk caused by misjudgment.

[0043] Step S103: Route the cache key to the shard node based on the consumption write-back event, perform a write operation in the distributed cache service and set the survival time and multi-copy synchronization; when cache invalidation or write failure occurs, perform alternative node retry or start a temporary degradation strategy, and perform full loading of hot data based on the access frequency.

[0044] In some embodiments, the cache key is routed to the shard node based on the consumption write-back event, and the node where the data should be stored is determined according to the sharding rule. Perform a write operation in the distributed cache service, set the survival time and multi-copy synchronization to ensure the availability and persistence of the data. When cache invalidation or write failure occurs, perform alternative node retry, and if it still fails, start a temporary degradation strategy, such as returning a default value or degrading the service. Perform full loading of hot data based on the access frequency to improve the access performance of hot data.

[0045] The sharding rule here is based on algorithms such as consistent hashing or range sharding, mapping the cache key to a specific node. The distributed cache uses a cluster mode such as RedisCluster, supporting multi-copy synchronization. Cache invalidation detection is through a regular polling or event notification mechanism. Improve cache availability and reliability, and multi-copy synchronization ensures data is not lost.

[0046] As an implementation of step S103, it specifically includes: when routing the cache key to the shard node, adjusting the routing rule of the cache key based on the current load status of the distributed cache node, and when writing multiple replicas, assigning an independent version number to each replica, and verifying the version consistency between replicas through the distributed transaction framework after the writing is completed; predicting the set of cache keys that will become hotspots within a certain period in the future based on the historical access logs and the current access frequency.

[0047] When routing the cache key to the shard node, the system will collect the load status of each node in the distributed cache cluster in real time, including indicators such as CPU usage, memory occupancy, and network I / O. Based on these indicators, adjust the routing rule of the cache key, for example, route the new cache key to the node with lower load first to avoid excessive traffic concentration. When writing multiple replicas, assign an independent version number to each replica, and the version number contains timestamp and sequence number information. After the writing is completed, verify the version consistency between all replicas through a distributed transaction framework such as Seata or TCC mode to ensure the synchronization of data between replicas. The system will analyze the historical access logs, extract information such as the access pattern and frequency change trend of the cache key. Combining the current access frequency data, use time series analysis, machine learning algorithms such as the improved LRU algorithm or a heat prediction model to predict the set of cache keys that may become hotspots within a certain period in the future. It realizes the load balancing of the distributed cache cluster, avoids the performance bottleneck caused by overload of some nodes, and improves the throughput and stability of the entire cache system. The multi-replica independent version number and consistency verification mechanism ensure the strong consistency of data between replicas and improve data reliability.

[0048] Regarding the example of step S103 of this application, the write-back event is consumed by the cache coordination module, and the Key is routed to the corresponding shard node according to the consistent hashing algorithm. The distributed cache service sets a sufficient TTL for the write operation and performs multi-replica synchronization. If a temporary failure or node downtime occurs during the L2 writing process, the module triggers the alternative node retry mechanism. When it is detected that a large-scale cache invalidation or the write failure rate exceeds the threshold, the cache coordination module will enable the temporary degradation strategy, and directly direct the read and write requests to the backend database or return static content.

[0049] Based on the access frequency monitoring in this embodiment, when the access volume of a certain Key exceeds the warm-up threshold, the distributed cache subsystem will actively load all the data of this Key to all cache nodes to prevent the concurrent pressure during concentrated access.

[0050] Step S104: Construct a message body containing the order identifier, status change, version number, idempotency identifier, and timestamp based on the cache write result. Select the message exchange type and target queue according to the message partition key, business priority, and queue backlog status, deliver the message body to the message middleware, and obtain the delivery confirmation; if the delivery fails, cache it to the local retry queue and perform timed retries.

[0051] In some embodiments, a message body containing the order identifier, status change, version number, idempotency identifier, and timestamp is constructed based on the cache write result. Select the message exchange type and target queue according to the message partition key, business priority, and queue backlog status, deliver the message body to the message middleware, and obtain the delivery confirmation. If the delivery fails, cache it to the local retry queue and perform timed retries to ensure that the message is finally delivered successfully. Achieve system decoupling, and different modules communicate asynchronously through the message middleware.

[0052] In some specific embodiments, the construction and routing of business messages are implemented based on the following aspects: (1) Message content assembly The SDK constructs a message body based on the cache write result, including: order ID, pre- and post-update status, operation version number, idempotency identifier UUID, and timestamp.

[0053] For example: { "orderId": 12345, "oldStatus": "LOADING", "newStatus": "LOADED", "version": 42, "idempotentKey": "uuid-xyz-123" } (2) Routing decision The routing module selects the appropriate Exchange type and target Queue based on the partition key, business priority, and current queue backlog status in the message content. Business priorities can be divided into core business and non-core business.

[0054] The system supports the issuance of routing rules: when high-priority services break out, some non-critical messages can be temporarily redirected to the standby Queue to relieve the pressure on the main channel.

[0055] (3) Message delivery and confirmation The message is delivered to the middleware via the RabbitMQ / Kafka SDK, and the delivery confirmation from the Broker side is obtained.

[0056] If the message delivery fails, the SDK caches the message in the local Retry queue and performs timed retries; if it fails after multiple retries, it enters the dead-letter queue and generates an alarm.

[0057] Step S105: Pull messages from the queue according to the configured concurrency, perform deduplication verification on the idempotency identifier of the message, and perform data persistence operations after verifying that the message version is higher than the current version of the database; if the operation fails, write the message to the compensation queue and record the error log.

[0058] In some embodiments, messages are pulled from the queue according to the configured concurrency, and deduplication verification is performed on the idempotency identifier of the message to prevent duplicate processing. After verifying that the message version is higher than the current version of the database, data persistence operations are performed to ensure data consistency. If the operation fails, the message is written to the compensation queue and the error log is recorded, which is convenient for subsequent troubleshooting and processing. Ensure the idempotency of data processing and avoid data inconsistency caused by duplicate processing. Ensure eventual data consistency and ensure that the latest data is persisted through version verification.

[0059] In some specific embodiments, the specific method of asynchronous consumption and data landing in step S105 is as follows: (1) Consumer pulling and concurrency control The consumer instance pulls messages from the Queue according to the configured concurrency. Before consumption, the consumer performs deduplication verification on the idempotentKey of the message to avoid duplicate processing of the same message.

[0060] (2) Business processing and persistence After confirming the idempotency of the message, the consumer performs data version verification: only when the message version is higher than the current version in the database, the status update is performed; otherwise, it is skipped. The persistence operation ensures atomicity through database transactions, and updates the status field and version number of the business table.

[0061] (3) Compensation and error handling If the database operation or transaction submission fails, the consumer writes the message to the compensation queue and records the error log. The compensation task periodically scans the compensation queue and attempts to retry or initiate a manual intervention process based on the log and alarm information.

[0062] Regarding step S105, it also involves adjusting the concurrent consumption quantity of consumer instances according to the peak hours of port operations during the message pulling phase, and restoring the default value during off-peak hours to match the business traffic fluctuations; during the idempotency verification process, comparing whether the status change path in the message is consistent with the previous status recorded in the database; before data persistence, performing a difference verification on the version number in the message and the current version number of the database. If the difference exceeds the preset threshold, the message is determined to be an outdated message and skipped for processing; when the message is written into the compensation queue, associating the cache key corresponding to the message with the link identifier, and generating a diagnostic data packet containing the operation time, failure count, and database lock status for the compensation task to preferentially obtain the lock status information when retrying.

[0063] It can be seen that during the message pulling phase, the system adjusts the concurrent consumption quantity of consumer instances according to the preset peak hours of port operations. By obtaining the current business traffic data through the monitoring system and comparing it with the historical peak hour data, the system increases the number of consumer instances or the concurrent threads of a single instance during peak hours to accelerate message processing; during off-peak hours, it restores the default configuration to avoid resource waste.

[0064] During the idempotency verification process, the system not only verifies the idempotency identifier of the message but also compares the status change path carried in the message with the previous status recorded in the database. For example, the order status can only change from "created" to "paid". If the status change path in the message is "created -> completed", it is determined as an illegal change and rejected for processing. Before data persistence, the system extracts the version number in the message and performs a difference verification with the version number of the current database record. If the difference exceeds the preset threshold, the message is considered an outdated message, which may be caused by network latency or other reasons resulting in a lag in the processing order. At this time, the system skips the processing of this message to avoid overwriting new data with old data. When the message is written into the compensation queue, the system associates and stores the cache key and link identifier corresponding to the message, and generates a diagnostic data packet containing information such as the operation time, failure count, and database lock status.

[0065] When the compensation task retries, it can preferentially obtain the lock status information to determine whether the failure is caused by a database lock conflict, and then adopt corresponding retry strategies, such as delayed retry or preferentially obtaining the lock.

[0066] Step S106: Insert metric collection points into each module of the full link, and visually display metrics such as request latency, cache hit rate, and message backlog depth. When the key metrics continuously exceed the preset threshold, adjust the system resource configuration, give real-time alarms for metric anomalies and service anomalies, and support problem location through the link identifier.

[0067] In some embodiments, metric collection points are implanted in each module of the entire link to collect metrics such as request latency, cache hit rate, message backlog depth, etc. Visualize these metrics to facilitate system operation and maintenance personnel to monitor the system running status. When key metrics continuously exceed the preset thresholds, adjust the system resource configuration, such as increasing server instances or adjusting cache size. Provide real-time alerts for metric anomalies and service anomalies, and support problem location through link identifiers to quickly troubleshoot the root cause of failures. Achieve system observability, and promptly detect system anomalies through metric visualization. Improve system elasticity and adjust resource configuration to cope with traffic changes. Quickly locate and solve problems, and link tracing supports troubleshooting of problems across the entire link.

[0068] In some specific embodiments, step S106 can also implement the monitoring and warning and elastic governance processing processes.

[0069] Specifically, Prometheus tracing points can be implanted in the gateway, SDK, cache coordination module, and consumer side based on the entire link to collect: request latency, cache hit rate, message backlog depth, consumer processing rate, failure rate, etc. The data is displayed through a visualization platform and supports custom dashboards.

[0070] When a certain key metric, such as message backlog depth > 5000, consumer latency > 200ms, L2 write latency > 50ms, continuously exceeds the preset threshold, the system calls the container orchestration platform interface to increase cache nodes or consumer instances. The elastic scaling behavior will be written into the audit log for tracking the history of capacity adjustments.

[0071] In this embodiment, when metrics suddenly increase or service anomalies occur, the monitoring system issues real-time alerts through channels such as email, SMS, and DingTalk. Operation and maintenance personnel can view the TraceID in the alert details, quickly locate the specific request link of the problem, and perform diagnosis and repair in combination with the link tracing panel.

[0072] In an embodiment of the present invention, based on step S101, a possible embodiment will be given below to non-restrictively elaborate on its specific implementation scheme.

[0073] Step S101 specifically includes: when receiving an update request containing an order identifier, a target status, and a user credential, adjust the authentication verification policy based on the business attributes and user roles of the request source; during the traffic limiting processing stage, collect the gateway data processing volume and interface response time in real time, and calculate the token generation rate and bucket capacity of the token bucket algorithm through a rule engine; for example, reduce the token generation rate to protect the backend service when the system load is high, and increase the rate to improve throughput when the load is low.

[0074] When current limiting is triggered, core business requests enter the queuing waiting queue and are assigned priority tags; when generating the link identifier, the business type, user role, and access path of the request are encoded into the prefix of the TraceID to form a structured identifier format. This identifier is used for quick classification and exception tracing during subsequent log aggregation. When initializing the request context, the TraceID, user ID, and request source IP are passed through to the backend SDK and monitoring system; when an exception occurs in a certain link of the link, the TraceID carries the exception information back to the request source.

[0075] In this way, the system combines security and flexibility, can protect core services, and can also provide a differentiated service experience. It effectively balances system stability and throughput, and avoids service crashes caused by overload. Core business requests are queued first to ensure that key services are not affected by current limiting and improve the user experience. Through the TraceID, the full-link logs can be quickly associated, reducing the troubleshooting time.

[0076] In an embodiment of the present invention, based on step S104, a possible embodiment will be given below to non-restrictively elaborate on its specific implementation.

[0077] Step S104 specifically includes: When constructing the message body, generate a message routing feature vector according to the business entity type associated with the order identifier, match the feature vector with a preset routing rule library, and adjust the Exchange type and target Queue priority of the message.

[0078] Hierarchically compress the message body containing the status change history: use dictionary encoding to compress common field values, and the compressed message body is delivered to the message middleware through an independent channel.

[0079] Establish a cross-middleware disaster recovery channel for message delivery. When the primary message middleware fails, switch the message to the corresponding Topic of the standby middleware and adjust the message format according to the protocol characteristics of the standby middleware; Add a message aging timestamp field to the local Retry queue. For messages that have not been successfully delivered after exceeding the preset aging time, trigger the business compensation logic: query the database according to the idempotency identifier in the message. If the corresponding business operation has been completed, mark the message as processed and discard it.

[0080] It should be noted that the generation of feature vectors is based on the metadata model of business entities. The system resolves the associated entity types from the order identifier, extracts the attribute values, and maps them to the feature space. The routing rule library uses the decision tree algorithm. After inputting the feature vectors, it outputs the matching Exchange type and Queue priority. The hierarchical compression adopts two-level processing. The first-level dictionary encoding establishes a mapping table between common values and short codes, and the second-level uses the GZIP algorithm for further compression. The independent channel establishes a dedicated connection through frameworks such as Netty to achieve fast message delivery. The cross-middleware disaster tolerance is based on the health check mechanism. The system periodically sends heartbeat packets to the primary middleware. If the consecutive failures exceed the threshold, a switch is triggered. The message format conversion is implemented through the adapter pattern, providing serializers / deserializers for different middleware protocols. The processing of aging messages is implemented through the SortedSet of Redis, using the message ID as the member and the aging time as the score. The timed task scans the expired members and invokes the compensation service. The hierarchical compression strategy effectively reduces the message volume, and the independent channel avoids network congestion, improving the message transmission efficiency. The cross-middleware disaster tolerance channel enhances the system's resilience, ensuring that messages are not lost in the event of middleware failures and improving availability. At the same time, the data final consistency is ensured through the business compensation logic.

[0081] Further, as a refinement and extension of a specific implementation manner of the cache information processing method of the above port platform, the method is also based on the topology routing label and business type transmitted in step S101. Among them, the core business data adopts cross-region multi-active sharding and is synchronously written to the primary and standby regional nodes; the ordinary business adopts local domain consistent hashing sharding. When it is detected in step S106 that the target area delay exceeds the threshold, the sharding traffic is switched to the low-delay area.

[0082] Step S103 is also based on when the single-node write failure rate exceeds the first preset threshold, enabling the standby node in the same region to retry; when the overall failure rate in the same region exceeds the second preset, triggering cross-layer linkage: sending an instruction to step S102 to extend the local cache survival period; directing the request to the read-only replica of the database associated with step S105 and returning the staticized data; and marking the failed area as a vulnerable area and suspending the writing of non-core services.

[0083] According to the node device implanted in step S106, avoid sharding of high-load nodes; execute the generation of encrypted snapshots for continuously failed data and store them in the cold backup storage pool, and then forcibly enable multi-region synchronous writing; and feedback a credit deduction signal to S101 to limit the source request.

[0084] This embodiment is based on the topology routing label and business type transmitted in S101, adopts the cross-region multi-active sharding strategy for the core business data, and writes the data to the primary and standby regional nodes simultaneously to ensure strong consistency.

[0085] When it is monitored in S106 that the latency in the target area exceeds the threshold, the system switches the sharded traffic to the low-latency area to ensure service quality. In S103, when the write failure rate of a single node exceeds the first preset threshold, the system enables a standby node in the same area to retry and attempts to send the request to other available nodes in the same area. If the overall failure rate in the same area exceeds the second preset threshold, a cross-layer linkage mechanism is triggered: sending an instruction to S102 to extend the survival period of the local cache and reduce the dependence on the distributed cache; directing the request to the read-only replica of the database associated with S105 to return static data; at the same time, marking the failed area as a vulnerable area and suspending the writing of non-core services to ensure core services. The system avoids sharding high-load nodes based on the node load information sent by S106 and directs the traffic to low-load nodes. For continuously failed data, an encrypted snapshot is generated and stored in the cold backup storage pool to ensure data security. At the same time, multi-region synchronous writing is forcibly enabled to enhance data reliability. The system also feeds back a credit deduction signal to S101 to limit the access frequency of source requests and prevent malicious or abnormal requests from further affecting the system.

[0086] Further, as an embodiment of the above cache information processing method for the port platform, the specific implementation manner includes the following steps: In the method, based on the real-time message accumulation depth and consumer processing latency metrics sent by S106, the concurrency is linearly increased when the accumulation depth exceeds the threshold, and the batch prefetch mode is enabled when the latency exceeds the threshold; an independent consumption channel is allocated for high-credit core services to isolate the impact of non-core service traffic.

[0087] The core service status change defines the continuity of the verification version number; the non-core service is set to skip intermediate versions; when a version number break is detected, the distributed cache in step S103 is queried to obtain the latest data to complete the version chain.

[0088] When parsing the topology routing label embedded in the message in the method, it is routed to the database replica in the same area; if the target replica is unavailable, it is switched to the nearest low-latency replica according to the defined regional health topology map; the processing result is sent back to S106, and the tracking status is updated by associating the full-link identifier.

[0089] The method also collects the real-time data of the local cache hit rate in step S102, the distributed cache write latency in step S103, the message delivery success rate in step S104, and the consumer processing latency in step S105, and calculates the comprehensive business health score based on (hit rate × 20%) + (1 / latency × 30%) + (success rate × 30%) + (1 / latency × 20%).

[0090] The hit rate in Hit rate × 20% is obtained based on step S102. The latency in (1 / Latency × 30%) is obtained based on step S103. The success rate in (Success rate × 30%) is obtained based on step S104. The hit rate in (1 / Latency × 20%) is obtained based on step S105.

[0091] When the score is lower than the security threshold, a global warning is triggered. The comprehensive score of business health provides a quantitative indicator of the overall operating state of the system, making the warning and degradation strategies more scientific.

[0092] Exemplarily, for instance, when the comprehensive score of business health is less than a certain score, S101 flow limiting upgrade and S103 degradation writing can be triggered. When the comprehensive score of business health is less than an even lower score, global fusing is initiated.

[0093] In some specific embodiments, when the accumulation depth exceeds a preset threshold, the consumer concurrency is increased linearly; when the processing latency exceeds the threshold, the batch prefetch mode is enabled to pull multiple messages at once to reduce network overhead. For the state change of core services, the system strictly verifies the continuity of the version number.

[0094] When a version number break is detected, the system queries the distributed cache of S103 to obtain the latest data, completes the version chain, and then performs the persistence operation. The system parses the topology routing label embedded in the message and preferentially routes the request to the database replica in the same region. If the target replica is unavailable, according to the predefined regional health topology graph, it switches to the nearest low-latency replica. The processing result is sent back to S106 to update the full-link tracing status and achieve closed-loop monitoring of the request.

[0095] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0096] As Figure 3 shown, the present application also provides an electronic device, including a display module 103, a memory 102, a processor 101, and a computer program stored on the memory and executable on the processor 101. When the processor 101 executes the program, it implements the steps of the cache information processing method of the port platform.

[0097] In embodiments of the present invention, the electronic device includes, but is not limited to, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described herein and / or claimed.

[0098] In embodiments of the present application, the processor 101 may be implemented by using at least one of an application specific integrated circuit, a programmable logic device, a field programmable gate array, a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to execute the functions described herein. In some cases, such an implementation may be implemented in the controller. For a software implementation, an implementation of a process or function may be implemented with a separate software module that allows execution of at least one function or operation. The software code may be implemented by a software application (or program) written in any suitable programming language. The software code may be stored in the memory and executed by the controller.

[0099] The display module 103 is used to display information input by the user or information provided to the user. The display module 103 may include a display panel, and the display panel may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.

[0100] The memory 102 may be used to store software programs and various data. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid state storage devices.

[0101] The present application also provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cache information processing method of the port platform are implemented.

[0102] The storage medium may be any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] In a storage medium, a readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0104] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing cached information of a port platform, characterized in that, The method includes: S101: Receive an update request containing an order identifier, a target status, and user credentials, perform authentication and rate limiting processing on the request, and generate a link identifier and initialize a request context for the request that passes the rate limit; S102: Parse the link identifier and the request body, obtain the cache key prefix and the data structure type, perform a write operation in the local memory cache and set a short-term survival time, return a response after successful writing, and at the same time publish a consumption write-back event and mark that it needs to be synchronized to the distributed cache; S103: Route the cache key to the shard node based on the consumption write-back event, perform a write operation in the distributed cache service and set the survival time and multi-copy synchronization; S104: Construct a message body containing the order identifier, status change, version number, idempotency identifier, and timestamp according to the cache write result, select the message exchange type and the target queue based on the message partition key, business priority, and queue backlog situation, deliver the message body to the message middleware and obtain the delivery confirmation; S105: Pull messages from the queue according to the configured concurrency, perform deduplication verification on the idempotency identifier of the message, and perform data persistence operations after verifying that the message version is higher than the current version of the database; S106: Insert metric collection points into each module of the entire link, and visually display metrics such as request latency, cache hit rate, and message backlog depth.

2. The method for processing cache information of the port platform according to claim 1, wherein Step S101 specifically includes: When receiving an update request containing an order identifier, a target status, and user credentials, adjust the authentication verification policy based on the business attributes and user roles of the request source; In the rate limiting processing stage, collect the running status of the gateway in real time, and calculate the token generation rate and bucket capacity of the token bucket algorithm through the rule engine; When rate limiting is triggered, queue the core business requests into the waiting queue and assign a priority tag; When generating the link identifier, encode the business type, user role, and access path of the request into the prefix of the TraceID to form a structured identifier format; When initializing the request context, pass the TraceID and key metadata transparently to the backend SDK and the monitoring system; when an exception occurs in a certain link of the link, the TraceID carries the exception information and traces back to the request source.

3. The method for processing cached information of a port platform according to claim 1, characterized in that Step S102 specifically includes: Adjust the short-term survival time of the cache key based on the real-time business load; Based on the prefix of the cache key and the data structure type, combined with the load status of the distributed cache node, predict the target shard node of the cache key in the distributed cache, and mark the target shard information in the local cache; Generate a cache mapping table for each cache key in the local cache, and record its storage location and version number in the local cache and the distributed cache; Assign a priority tag to the consumption write-back event according to the business importance or data update frequency of the cache key; Step S102 also predicts the cache keys that may become hotspots in the future based on the access frequency and historical trend of the current cache key, and reserves storage space in the distributed cache in advance; For the predicted hot data, load the relevant cache keys in full to the distributed cache node, and verify the loading effect through traffic mirroring technology.

4. The caching information processing method of the port platform according to claim 1, characterized in that, Step S103 specifically includes: When routing cache keys to shard nodes, adjust the routing rules of cache keys based on the current load status of distributed cache nodes. When writing multiple replicas, assign independent version numbers to each replica, and verify the version consistency between replicas through a distributed transaction framework after writing is completed; Predict the set of cache keys that will become hotspots within a certain period in the future based on historical access logs and current access frequencies.

5. The method for processing cache information of a port platform according to claim 1, wherein The method also depends on the topology routing tags and business types passed in step S101. Among them, core business data uses cross-region multi-active sharding and is synchronously written to the primary and standby regional nodes; ordinary business uses local domain consistent hashing sharding. When it is detected in step S106 that the latency in the target region exceeds the threshold, switch the shard traffic to a region with lower latency; Step S103 also enables retry of standby nodes in the same region when the single-node write failure rate exceeds the first preset threshold; when the overall failure rate in the same region exceeds the second preset threshold, trigger cross-layer linkage: send an instruction to step S102 to extend the survival period of the local cache; direct the request to the read-only replica of the database associated with step S105 and return staticized data; and mark the failed region as a vulnerable area and suspend writing of non-core services; According to the node device implanted in step S106, avoid sharding of high-load nodes; execute the generation of encrypted snapshots for continuously failed data and store them in the cold backup storage pool, then force multi-region synchronous writing; and feedback a credit deduction signal to S101 to limit source requests.

6. The method for processing cached information of the port platform according to claim 1, wherein Step S104 specifically includes: When constructing the message body, generate a message routing feature vector according to the business entity type associated with the order identifier, match the feature vector with the preset routing rule library, and adjust the Exchange type and target Queue priority of the message; Perform hierarchical compression on the message body containing the status change history: use dictionary encoding to compress common field values, and the compressed message body is delivered to the message middleware through an independent channel; Establish a cross-middleware disaster recovery channel for message delivery. When the primary message middleware fails, switch the message to the corresponding Topic of the standby middleware and adjust the message format according to the protocol characteristics of the standby middleware; Add a message aging timestamp field to the local Retry queue. For messages that have not been successfully delivered after exceeding the preset aging time, trigger the business compensation logic: query the database according to the idempotency identifier in the message. If the corresponding business operation has been completed, mark the message as processed and discard it.

7. The method for processing cache information of the port platform according to claim 1, wherein Step S105 specifically includes: During the message pulling phase, increase the concurrent consumption quantity of consumer instances according to the peak hours of port operations, and restore the default value during non-peak hours to match the business traffic fluctuations; In the idempotency verification link, compare whether the status change path in the message is consistent with the previous status recorded in the database; Before data persistence, perform a difference verification on the version number in the message and the current version number of the database. If the difference exceeds the preset threshold, determine it as an outdated message and skip the processing; When the message is written to the compensation queue, associate the cache key corresponding to the message with the link identifier, and generate a diagnostic data packet containing the operation time, failure times, and database lock status for the compensation task to preferentially obtain the lock status information during retry.

8. The method for processing cached information of the port platform according to claim 1, wherein In the method, based on the real-time message accumulation depth and the consumer processing delay metrics sent in S106, when the accumulation depth exceeds the threshold, the concurrency is linearly increased, and when the delay exceeds the threshold, the batch prefetch mode is enabled; Allocate an independent consumption channel for high-credit core services to isolate the impact of non-core service traffic; Define the continuity of the verification version number for core service status changes; non-core services are set to skip intermediate versions; when a version number break is detected, query the distributed cache in step S103 to obtain the latest data to complete the version chain; When the method parses the topology routing tags embedded in the message, route to the database replica in the same region; If the target replica is unavailable, switch to the nearest low-latency replica according to the defined regional health topology map; the processing result is returned to S106, and the tracking status is updated by associating the full-link identifier; The method also collects the real-time data of the local cache hit rate in step S102, the distributed cache write delay in step S103, the message delivery success rate in step S104, and the consumer processing delay in step S105, and calculates the comprehensive business health score based on (hit rate × 20%) + (1 / delay × 30%) + (success rate × 30%) + (1 / delay × 20%); when the score is lower than the safety threshold, a global warning is triggered.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the cache information processing method of the port platform according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the cache information processing method of the port platform according to any one of claims 1 to 8.

Images