E-commerce search service recovery method and device, equipment and medium

Abstract

The invention relates to an e-commerce search service recovery method and device, equipment and a medium, and the method comprises the steps: starting a local search service deployed by a current fault node, and sending a missing data request to a remote search service of any healthy node, the request comprising a latest timestamp of a latest log record in a local transaction log library, the healthy nodes and the fault nodes are disaster recovery nodes; receiving a complete transaction log file which is returned by the remote search service and is generated after the latest timestamp, grouping log records in the complete transaction log file according to the commodity identifier, and determining a target log record with the maximum global version number in each group; constructing the target log record into an incremental transaction log file, writing the incremental transaction log file into a local transaction log library, and replaying the incremental transaction log file into search engine index data; and after the playback is completed, the local search service is registered to the service registration center again. According to the method, the fault node can be quickly recovered, and high consistency of the commodity data among the search service nodes is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of e-commerce, and in particular to an e-commerce search service recovery method and device, equipment and medium. BACKGROUND

[0002] The search service is a high-frequency business in the e-commerce platform, and the e-commerce platform has the characteristics of wide service region, scattered deployment, frequent access, high data security and stability requirements, especially for cross-border e-commerce platforms. In the actual operation scene of the cross-border e-commerce platform, the search service usually adopts single room deployment to support the real-time retrieval demand of global buyers for massive goods.

[0003] Under this architecture, all goods data, index files and transaction logs are stored in the database and storage cluster of the same room. When the room is powered off, the network is interrupted or the hardware fails, the entire search service will be suspended, resulting in global users unable to search for goods, the order link broken, and ultimately causing order loss and damage to brand reputation. In order to avoid single point failure, the industry gradually introduces multi-room deployment solutions, trying to provide redundancy protection by building disaster recovery nodes in different regions. However, the multi-room architecture exposes new technical problems in the cross-border e-commerce scene.

[0004] Firstly, the source of goods data updates between rooms is complex, including inventory adjustment, price modification in the seller's background, and batch price changes in the platform promotion system. These operations are difficult to ensure strict order when transmitted in cross-regional networks, resulting in version conflicts for the same goods record in different rooms, and ultimately leading to the situation that the displayed price or inventory in the search results does not match the actual situation, directly affecting the purchase decision of cross-border buyers.

[0005] Secondly, when a room fails and comes online, the traditional approach usually uses manual library brushing or full data backtracking to rebuild the index. This process requires re-consuming incremental data streams for several hours or even several days, not only time-consuming, but also the search service is in a degraded state during this period, which is likely to cause a sharp drop in user experience and order loss during the peak of the promotion.

[0006] In addition, more importantly, since each room generates different local transaction logs during the failure period, there is a lack of unified version benchmark, and the disaster recovery node cannot quickly identify the latest valid record when merging logs, and can only rely on manual scripts to compare one by one, which is easy to miss or make mistakes, ultimately leading to inconsistency between the search engine index and the real goods data, further exacerbating cross-border transaction disputes.

[0007] Therefore, how to ensure the consistency of goods data versions under the multi-room architecture, and at the same time achieve fast and accurate search service recovery after a single room failure, has become a core technical problem that needs to be solved by cross-border e-commerce platforms. SUMMARY

[0008] The present application aims to solve at least one of the above problems and provide an e-commerce search service recovery method and corresponding device, equipment, non-volatile readable storage medium, and computer program product.

[0009] According to one aspect of the present application, an e-commerce search service recovery method is provided, comprising:

[0010] Starting a local search service deployed by a current fault node, sending a missing data request to a remote search service of any healthy node, the request containing a latest timestamp of the latest log record in a local transaction log library, the healthy node and the fault node being backup nodes for each other;

[0011] Receiving a complete transaction log file returned by the remote search service, the complete transaction log file being generated after the latest timestamp, grouping log records in the complete transaction log file according to commodity identifiers, determining a target log record with a maximum global version number in each group, the global version number being globally monotonically increasing and maintained by a distributed version service cluster independent of the local search service and the remote search service;

[0012] Constructing the target log record of each group into an incremental transaction log file and writing it into the local transaction log library, so as to replay commodity data in the incremental transaction log file into search engine index data of the local search service;

[0013] Re-registering the local search service to a service registration center after the replay is completed, and executing a query based on the search engine index data through the local search service in response to an external access commodity query request.

[0014] According to another aspect of the present application, an e-commerce search service recovery device is provided, comprising:

[0015] A missing request module is configured to start a local search service deployed by a current fault node, and send a missing data request to a remote search service of any healthy node, the request containing a latest timestamp of the latest log record in a local transaction log library, the healthy node and the fault node being backup nodes for each other;

[0016] A target determination module is configured to receive a complete transaction log file returned by the remote search service, the complete transaction log file being generated after the latest timestamp, group log records in the complete transaction log file according to commodity identifiers, and determine a target log record with a maximum global version number in each group, the global version number being globally monotonically increasing and maintained by a distributed version service cluster independent of the local search service and the remote search service;

[0017] a write-back module configured to write target log records of each group into a local transaction log library as an incremental transaction log file for playing back commodity data in the incremental transaction log file into search engine index data of the local search service;

[0018] a service restart module configured to re-register the local search service to a service registry center after the playback is completed, and execute a query based on the search engine index data in response to an externally accessed query request for commodities through the local search service.

[0019] According to another aspect of the present application, there is provided an e-commerce search service recovery device including a central processing unit and a memory, the central processing unit being configured to invoke a computer program stored in the memory to execute the steps of the method described in the present application.

[0020] According to another aspect of the present application, there is provided a non-volatile readable storage medium storing a computer program implemented according to the e-commerce search service recovery method in the form of computer readable instructions, the computer program being invoked by a computer to execute the steps included in the method when run.

[0021] According to another aspect of the present application, there is provided a computer program product including computer programs / instructions, the computer programs / instructions being executed by a processor to implement the steps of the method.

[0022] Compared with the conventional technology, the present application introduces a distributed version service cluster independent of each search node in the disaster recovery process of a search service failure node, assigns a global monotonically increasing version number bound to a commodity identifier to each piece of commodity data, and performs grouping and deduplication by commodity dimension once at the log merging stage to retain the latest valid record and completely eliminate version conflict phenomena caused by network disorder or concurrent updates in different machine rooms. When any machine room fails due to power failure or network interruption, only the complete transaction log file generated after the latest timestamp in the transaction log library of the failure node needs to be pulled to the healthy node, and then an incremental transaction log file is directly generated through the global version number deduplication mechanism and played back into the local search engine index data, the entire process does not require manual comparison of each piece of data or re-consumption of historical data streams, the recovery time is compressed from the conventional several hours or even several days to minutes, and the impact of search service degradation on a big promotion peak is significantly reduced. Since the data consistency check is completed at the log playback stage, the search engine index and the real commodity data are always kept in synchronization, and the commodity data retrieved by cross-border buyers at any time is consistent with the latest status in the background, effectively avoiding order disputes and brand reputation loss. In addition, the unified version benchmark enables all disaster recovery nodes to only focus on the global version number during recovery or normal synchronization, greatly simplifying the operation and maintenance complexity and further improving the stability and disaster recovery capability of the cross-border e-commerce platform under global multi-region deployment. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating one embodiment of the e-commerce search service recovery method of this application;

[0024] Figure 2 This is a schematic diagram of the e-commerce search service recovery device of this application;

[0025] Figure 3 This is a schematic diagram of the structure of an e-commerce search service recovery device used in this application. Detailed Implementation

[0026] To facilitate understanding of the implementation details of this application in a multi-datacenter environment, the following example uses a cross-border e-commerce platform as an illustration of the implementation architecture. This architecture is geographically distributed across three availability zones: North America, Europe, and Southeast Asia. Each availability zone deploys a containerized cluster. Each containerized cluster runs a search service node primarily serving local buyers. These search service nodes are interconnected via high-speed dedicated lines and the internal network, acting as disaster recovery nodes for each other. When a datacenter or its internal search service node fails due to power outage or network interruption, that node becomes the failed node, while the remaining healthy search service nodes become the relatively healthy nodes. For the local search service running on the failed node, the search services running on the other healthy nodes constitute the remote search service.

[0027] All nodes can host local search services, transaction log servers, and disaster recovery synchronization gateways via container instances. The local search service is responsible for responding to product query requests in real time, quickly retrieving relevant product data from the local search engine index data. The transaction log server encapsulates each product update resulting from a product update request into a log record with a global version number and persists it to the local transaction log repository. The disaster recovery synchronization gateway continuously monitors the latest timestamp of the local transaction log repository under the unified scheduling of the container orchestration system. When a failed node restarts, it can drive the disaster recovery synchronization gateway to immediately send a missing data request to the corresponding remote search service on any healthy node. This request carries the timestamp corresponding to the latest log record in the failed node's local transaction log repository. Upon receiving the request, the healthy node compresses and packages the complete transaction log file generated after that timestamp and sends it back to the failed node via an encrypted link. The failed node groups the log records locally by product identifier, deduplicates them according to the global version number, generates incremental transaction log files, and batch replays them into the search engine index data of the local search service, completing minute-level incremental reconstruction.

[0028] In the present application, the data update requirement and online search requirement are decoupled according to the write chain and the read chain. The write chain is mainly used to implement the change of commodity data and make corresponding log records, and the read chain can reconstruct the index required for the search of the commodity data according to the changed log records to form the search engine index data.

[0029] From the perspective of the write chain, the commodity data source is located in the e-commerce database. When all online stores in the e-commerce platform are on the shelf, the price is changed or the inventory is adjusted, the latest field is pushed to the e-commerce database through a commodity update request. The local search service listens to the corresponding commodity change notification message and generates a log record containing the updated commodity data. The commodity data includes but is not limited to commodity identification and other associated information such as commodity price, commodity inventory and effective timestamp, etc. These other information can be determined according to the actual situation. The transaction log server listens to the commodity change notification message of the e-commerce database. After each write is completed, the log record is encapsulated as a log record entry with a global version number. The original timestamp is also retained in the entry. Then it is persisted to the local transaction log library to form a traceable and sequentially unique incremental stream.

[0030] From the perspective of the read chain, the timestamp can be continuously scanned by the disaster recovery synchronization gateway. When the fault node needs to be restored, the missing log file is pulled according to the timestamp. The fault node completes the deduplication locally with commodity identification as the key and global version number as the sequence, and only retains the latest version of each group. The obtained incremental transaction log file can be immediately played back to the search engine index data to reconstruct the inverted and forward indexes, ensuring that the index is consistent with the e-commerce database within seconds. The commodity query request only acts on the search service node. For example, after the buyer inputs the keyword, the local search service directly retrieves and returns the commodity information from the played back search engine index data, without the need to return to the e-commerce database.

[0031] As can be seen, the read and write links of the e-commerce database are completely separated in the present application, which not only guarantees high concurrent query performance, but also avoids the impact of update traffic on the read path.

[0032] On the basis of the above exemplary network architecture, the following continues to develop detailed descriptions of various embodiments of the present application.

[0033] Please refer to Figure 1 According to the e-commerce search service recovery method provided by the present application, in some embodiments, the method comprises the following steps:

[0034] Step S3100, start the local search service deployed on the current fault node, and send a missing data request to the off-site search service of any healthy node. The request contains the latest timestamp of the latest log record in the local transaction log library. The healthy node and the fault node are each other's disaster recovery nodes.

[0035] In the starting phase of the disaster recovery process, the faulty node first restarts the local search service process deployed thereon, and starts the corresponding local search service. The local search service enters a recovery state immediately after being started, and temporarily no longer provides queries to the outside, but focuses on completing the integrity verification and completion of its own data.

[0036] The local search service can first define its own role as a faulty node through a default mechanism or through an environment variable or configuration file automatically injected by a container orchestration system. For example, after the local search service receives a global version number watermark sent by another search service node and compares the global version number watermark with the latest global version number watermark of itself, it finds that the global version number of itself lags behind the global version number of the other search service node, and can trigger this recovery process to regard itself as a faulty node.

[0037] After the local search service starts the recovery process, the timestamp of the last log record in the local transaction log library, that is, the latest log record, can be read as the latest timestamp. The timestamp represents the last write time confirmed by the local node before the fault occurs. Any missing data after that time is necessarily later than the time. To obtain the missing data, the local search service can initiate a missing data request to the search service deployed on any healthy node, that is, the off-site search service, through an intranet private line or a public network encrypted tunnel. The request message contains the latest timestamp and the node identifier. After receiving the request, the healthy node directly scans the local transaction log library of the healthy node using the latest timestamp as the lower bound, batch-pulls all log records later than the latest timestamp to form a complete transaction log file after the latest timestamp, and returns the file in the form of a compressed package. Since the healthy node and the faulty node are mutual backup nodes, the business logic of the respective search services, the log file format, the field definition, and the compression algorithm are completely consistent, and the returned file does not need to be converted for subsequent processing. When generating the returned file, the healthy node can also attach a checksum and a file size to ensure transmission integrity.

[0038] Step S3200, receiving the complete transaction log file generated after the latest timestamp returned by the off-site search service, grouping the log records in the complete transaction log file according to the commodity identifier, determining the target log record with the largest global version number in each group, and the global version number is globally monotonically increasing and maintained by a distributed version service cluster independent of the local search service and the off-site search service.

[0039] After the faulty node receives the complete transaction log file returned by the off-site search service, it first writes the file to a local temporary directory and performs integrity verification. After confirming that the checksum and the file size are completely consistent with the metadata carried in the return message, it enters the subsequent processing process.

[0040] The file content of the complete transaction log file is a sequential sequence composed of a plurality of log records, each corresponding to a commodity data set, including but not limited to commodity identifier, global version number, update event data, timestamp, etc. The field format is consistent with the local transaction log library, and no secondary conversion is required for direct parsing.

[0041] After parsing, the fault node groups the log records using the commodity identifier field, and all log record entries with the same commodity identifier are merged into the same group. After grouping, the fault node sorts each group according to the global version number, and retains the single log record with the largest global version number as the only valid snapshot of the commodity in that group, and the remaining log record entries are discarded.

[0042] In this application, the global version number is generated by a distributed version service cluster independent of the local search service and the remote search service. Each time a commodity update request is received, the corresponding commodity identifier is incremented by one, and the majority write mechanism is used to persist in the cluster, ensuring that the version number of the same commodity identifier is monotonically increasing and globally unique at any time.

[0043] To facilitate implementation by those skilled in the art, it is recommended to use a hash table or skip list to maintain the mapping of commodity identifier to current maximum global version number in memory. Each time a log record is parsed, the commodity identifier is used as the key and the global version number is used as the value for comparison and update. If the global version number of the new record is greater than the current value, update the mapping and retain the record, otherwise discard it. For memory-constrained scenarios, embedded key-value databases such as RocksDB, LevelDB, etc. can be used to implement persistent storage and efficient range queries, with the commodity identifier as the key and the global version number as part of the key.

[0044] For example, assume that the iPhone 15 with commodity identifier SKU-1001 generates three log records in the North American data center, with version numbers "……1201", "……1203", and "……1205" in order. The fault node only retains the log record with version number "……1205" after grouping, and the other two are discarded, ensuring that only the latest state of the commodity is retained in the index during playback, avoiding old version coverage or conflict. The ellipsis in the above example global version number indicates a default prefix, which can be set as needed. For example, in subsequent embodiments of this application, the ellipsis can be a hash value determined by hashing the commodity identifier.

[0045] Step S3300, constructing the target log records of each group into an incremental transaction log file and writing it into the local transaction log library, so as to replay the commodity data in the incremental transaction log file into the search engine index data of the local search service;

[0046] After deduplication, each target log record represents the latest valid state of the corresponding commodity in the current batch. These records can then be written back to an incremental transaction log file for subsequent batch replay. This file can follow the exact physical format as the local transaction log library: commodity identifier, global version number, update event data, timestamp, and checksum fields are written sequentially in fixed byte order, the file header carries the version number and hash checksum, and the tail is appended with an overall CRC32 checksum to ensure integrity after cross-node transmission or disk writing. The incremental transaction log file can be written in sequential append mode, or cached in memory queues first, and then flushed to disk in batches when the queue length or time window reaches the threshold, to reduce disk I / O jitter. For high concurrency scenarios, the incremental transaction log file can be directly mapped to user space buffers using memory-mapped files or zero-copy techniques to avoid additional copy overhead.

[0047] After the incremental transaction log file is generated, it is immediately marked as pending replay and registered in the metadata table of the local transaction log library. The replay process can read this file sequentially in single-threaded or multi-threaded mode, parse each field of the commodity data, and convert it into the inverted index entries and forward storage structure required by the search engine index data. The thread pool size can be dynamically adjusted according to the number of CPU cores and disk throughput, typically, each thread is responsible for a batch of commodity identifiers, and tasks are passed between threads through a lock-free queue to avoid lock contention. After each log record is parsed, it is first located to the corresponding document ID in the search engine index data, and if the document already exists, it is updated in place, otherwise a new document is inserted; during the update process, an optimistic locking mechanism can be used, with the global version number as the conditional version field, if a higher global version number is found in the index data, it is skipped to prevent data rollback caused by concurrent replay.

[0048] To improve replay reliability, the maximum global version number processed is persisted to a local watermark file immediately after successfully writing a batch of index segments; if the replay process exits abnormally, it can continue from this watermark the next time it starts, avoiding repeated replay. The index segment merging strategy can use asynchronous background merging, triggered when the number of index segment files reaches a preset threshold or the deletion ratio in the segment exceeds 20%, new writes can continue to be served through a double buffering mechanism during merging to avoid affecting online queries.

[0049] Step S3400, after the replay is completed, re-register the local search service with the service registration center, and respond to external access commodity query requests through the local search service based on the search engine index data.

[0050] After the incremental transaction log file is completely written into the search engine index data, the local search service enters the last stage of recovery. At this time, the local search service first sends a health check request to the service registry center, and the message carries the current fault node unique identifier, the maximum global version number that has been played back, and the hash check value of the current index segment file list. After the service registry center verifies that the check value and the maximum global version number are up to date, the fault node state is set to “ready”, and the fault node is re-added to the service discovery list. For the scene of using Eureka, Consul or Zookeeper as the service registry center, the node only needs to call the corresponding registration interface to submit the metadata to complete the registration; if Kubernetes is used, the node returns the result to kubelet through the readiness probe, and kubelet automatically modifies the endpoint state to realize automatic online without explicitly calling the registration interface.

[0051] After successful registration, the local search service immediately switches to the external service mode and listens to the commodity query request to access external traffic. The commodity query request sent by any user first reaches the local search service, and the local search service quickly locates the candidate document according to the commodity identifier in the reconstructed inverted index data, and then filters out any obsolete record according to the global version number to ensure that the returned result is the latest valid commodity data. For high-concurrency scenarios, the local search service can enable a multi-threaded query processing pool to combine the index sharding mechanism to balance the query load to multiple shard instances to improve throughput.

[0052] In some embodiments, the query result can be verified again before being returned: if the inventory value is negative, the node immediately triggers an inventory oversell alarm and shields the display of the commodity; if the price field is marked as a high-frequency price commodity, the node enables a price cache lock to return only the latest price within the cache validity period to avoid price fluctuations. All responses can be accompanied by the current global version number as a response header before being returned to the buyer for consistency check by the client.

[0053] At this point, the disaster recovery process is complete, and the local search service has complete service capabilities and can provide low-latency, strongly consistent commodity search experience during the global peak of the big promotion.

[0054] Through the complete execution of the above embodiments, the present application can achieve significant technical progress, which is reflected in multiple aspects, including but not limited to:

[0055] First, by actively pulling the missing log by the fault node and completing the deduplication of commodity dimensions according to the global version number, the risk of unavailability of the search service during the peak of the big promotion is significantly reduced.

[0056] Secondly, in the log merging phase, consistency verification is completed to ensure that the search engine index is strictly synchronized with each commodity record of the e-commerce database, avoiding search result distortion caused by version conflicts or old data residues, thereby improving the shopping experience of cross-border buyers and the platform credibility.

[0057] Furthermore, the unified version benchmark and version number watermark synchronization channel automatically aligns the data between disaster recovery nodes, eliminating the need for complex scripts or manual intervention for comparison, significantly simplifying the operation and maintenance complexity and eliminating human errors.

[0058] In addition, during the post-recovery external service phase, the inventory negative value real-time interception and high-frequency price cache locking mechanism take effect immediately in the query link, further consolidating data consistency and stability, enabling the platform to continue providing low-latency and zero-interruption commodity search services under global multi-region deployment.

[0059] On the basis of any embodiment of the method of the present application, after starting the local search service deployed in the current fault node, the following steps are further included:

[0060] Step S4100, in response to the commodity update notification message, obtaining the corresponding update event data by the local search service, the update event data including the commodity data of the target commodity to be updated, which includes the commodity identifier of the target commodity;

[0061] After the local search service starts and completes the disaster recovery process, it returns to the normal service phase. In the normal service phase, the search service node not only needs to provide commodity query capability externally, but also needs to continuously receive and process commodity update notification messages from the e-commerce database to synchronize the latest commodity status to the local search engine index data.

[0062] The commodity update notification message can be actively pushed by the e-commerce database when the commodity data changes, and its trigger conditions include but are not limited to scenarios such as sellers adjusting inventory quantities, modifying commodity prices, updating commodity descriptions or pictures through the background system, platform operators setting up bulk sale prices, and third-party ERP systems writing back inventory changes through open interfaces. Each commodity update notification message contains corresponding update event data, which at least carries the commodity data of the target commodity to be updated. The commodity data must include a commodity identifier that can uniquely identify the commodity, such as a commodity SKU number, an international standard barcode, a platform internal commodity ID, or other globally unique key values. Taking a cross-border e-commerce platform as an example, when a seller in the North American region adjusts the inventory of SKU-1001 iPhone 15 from 100 to 90 in the background, the e-commerce database will immediately generate an update notification message and push it to the local search service in the North American data center through the message middleware.

[0063] After receiving the message, the local search service first extracts the update event data in the message body through the message parser, and then performs field checking and format conversion on the update event data to ensure that the key fields such as the commodity identifier, inventory value, price information, and effective timestamp are complete and conform to the expected format. If the checking is passed, the local search service encapsulates the update event data into an internal unified data structure for subsequent steps; if the checking fails, an exception log is recorded and an alarm is triggered to avoid dirty data entering the transaction log library.

[0064] In some embodiments, the commodity update notification message can be encapsulated in JSON format; in other embodiments, the message body can be in binary serialization formats such as Protocol Buffers or Avro to reduce network transmission overhead. Regardless of the format, the commodity identifier is always the core field in the parsing process, and all subsequent operations are associated and retrieved based on this field.

[0065] Step S4200, sending a version number acquisition request to the distributed version service cluster to obtain a global version number generated based on the commodity identifier carried in the request;

[0066] After the local search service obtains the commodity data of the target commodity to be updated, it sends a version number acquisition request to the distributed version service cluster that is independent of the local search service and the remote search service. The request is a lightweight remote procedure call, and the request body can only contain the commodity identifier field, such as {"sku_id":"SKU-1001"}. The distributed version service cluster can be implemented according to the majority write mechanism, consisting of an odd number of nodes, typically three or five nodes. Each node runs a version number maintenance service process, and internally stores a mapping table of commodity identifiers to global incremental serial numbers in RocksDB. After receiving the version number acquisition request, the version number maintenance service generates a globally monotonic and globally unique global version number and returns it.

[0067] At this point, the local search service obtains the global version number corresponding to the commodity identifier. The version number is globally monotonic and strongly bound to the commodity identifier, and version number conflicts will never occur between any two different commodities. After the global version number is generated, the version number maintenance service immediately writes it into the local RocksDB and synchronizes it to other nodes in the version service cluster through Raft log replication, ensuring data consistency in disaster recovery switching.

[0068] Step S4300, constructing a log record by associating the global version number with the update event data and the timestamp of the current time, and writing it into the local transaction log library of the local search service.

[0069] After receiving the global version number, the local search service assembles the global version number, the update event data, and the millisecond-level timestamp of the current time into a log record, and then writes the log record into the local transaction log library through a sequential append mode. The transaction log library can be implemented by using a WAL mechanism. Each log record is first written into a memory buffer. When the buffer is full or a timer expires, the log records are batched and flushed to a local SSD. After the flushing is completed, the log records are synchronized to a backup file descriptor through mmap, so that even if the process crashes, the written log records can be completely recovered when the process is restarted.

[0070] As can be seen, the local search service can continuously and reliably synchronize the latest commodity status to the local transaction log library in the normal service stage, thereby providing an accurate and ordered data source for the real-time update of the search engine index data. Since the search services deployed by other search service nodes are also the same business logic, each search service node can implement the mechanism.

[0071] The above embodiment deeply couples the commodity-level real-time update with the global version number atomic increment mechanism, so that the search service node can continuously and conflict-free absorb all incremental changes of the e-commerce database in the normal running stage, thereby fundamentally eliminating the version conflict risks caused by network disorder or concurrent writing in the traditional multi-machine room deployment. With the majority persistence of the distributed version service cluster and the Raft replication, the global version number remains monotonically increasing and never repeated in the case of any node failure or network partition, thereby ensuring that each commodity record has a unique and consistent version reference between cross-regional nodes. The sequential append process of writing the update event data, the global version number, and the timestamp into the local transaction log library, in combination with the WAL flushing and mmap double insurance, not only guarantees the high throughput and low latency of persistence, but also provides an accurate and traceable data source for subsequent minute-level incremental playback. Finally, all search nodes operate in the same mechanism, realizing the real-time data consistency at the cluster level, significantly reducing the operation and maintenance complexity and shortening the fault recovery window to the minute level, thereby improving the high availability and strong consistency capability of the cross-border e-commerce platform in the global range.

[0072] On the basis of any embodiment of the method of the application, a version number acquisition request is sent to the distributed version service cluster, and a global version number generated based on the commodity identifier carried in the request is obtained, including the following steps executed by a version number maintenance service in the version service cluster:

[0073] In step S5100, a version number acquisition request from the local search service or the remote search service is received, and the commodity identifier in the request is parsed and obtained.

[0074] The version number maintenance service continuously waits for a version number acquisition request from any search service node on a network listening port. The listening port can be a fixed TCP port or dynamically assigned by the configuration center when the node starts; the transport layer protocol can reuse the existing gRPC channel between the search node and the version service cluster, or use a RESTful endpoint based on HTTP / 2. The request message can only carry the product identifier field, and the field name remains the same as that used in the update event data, for example, sku_id, so that the version number maintenance service can directly parse without additional mapping. The parsing action is completed by a lightweight deserialization library inside the version number maintenance service: if the request body is JSON, the byte stream is mapped to a memory object; if the request body is Protocol Buffers, the generated stub code is directly called for decoding. The parsed result is temporarily stored in a thread-local variable in the form of a string for subsequent steps.

[0075] To be compatible with different message formats, the parser has a set of field mapping rules: when the request body contains multiple layers of nesting, the version number maintenance service extracts the product identifier according to the predefined path; when the upstream system uses different naming styles, the parser completes the conversion through case-insensitive matching or an alias table. For example, the upstream may use SKU, skuId, or sku_id in three ways, and the parser will convert them all to sku_id before passing them to the subsequent logic. If the parsing fails, the version number maintenance service immediately returns an error code with a readable exception description, preventing the search node from being blocked due to waiting. If the parsing is successful, the next step of atomic increment is triggered.

[0076] Step S5200, atomically increment the global auto-increment sequence number corresponding to the product identifier through the majority write mechanism to obtain the current sequence number segment;

[0077] In order to atomically generate a segment of incrementing global auto-increment sequence numbers for each product identifier within the version service cluster, the version number maintenance service can maintain a counter table locally, which uses the product identifier as the primary key and each key corresponds to a monotonically increasing integer. Whenever a product identifier is successfully parsed, the version number maintenance service immediately locates the corresponding counter slot in the table. If the counter slot does not exist when it is first referenced, it is automatically initialized to 0, ensuring that any product identifier can immediately obtain a sequence number when it first appears.

[0078] After confirming the counter slot, the current global auto-increment sequence number in the counter slot is obtained as the current sequence number segment, and then the current value in the slot is incremented by 1 and persisted to the entire cluster, so that all nodes agree on the "incremented value" to make the global auto-increment sequence number of the counter slot complete increment and be used as the current global auto-increment sequence number for the next request. The increment process of the global auto-increment sequence number can be ensured to be atomically incremented by a majority write mechanism.

[0079] Step S5300, splice the product identifier and the current sequence number segment and then perform hash calculation to generate a fixed-length product hash segment.

[0080] After obtaining the current sequence number segment, the version number maintenance service combines the current sequence number segment and the corresponding product identifier into an original spliced string according to a pre-agreed splicing rule. The splicing rule is uniformly published by the configuration center at system initialization, which can be in the form of underscore separation "product identifier_sequence number segment", colon separation "product identifier: current sequence number segment", or even directly use a continuous byte sequence without separator, as long as the product identifier is in front and the sequence number segment is in back. After splicing, the version number maintenance service performs a one-way hash calculation on the original spliced string by calling a hash algorithm such as SHA-256; the algorithm outputs a fixed length of 32 bytes, has high dispersion and anti-collision ability, and can generate almost unique hash values in the product identifier and sequence number segment combination space.

[0081] In order to balance storage and transmission efficiency, only the first 64 bits, i.e. 8 bytes, of the hash value can be extracted as a fixed-length product hash segment. If there is a higher requirement for space, the extraction length can be reduced to 48 bits or expanded to 96 bits, which is specified by the cluster-level configuration item hash_bits, and all nodes pull and keep consistent at startup. The introduction of the product hash segment makes the global version number spliced subsequently have high entropy characteristics in the global range, avoiding predictability due to similar product identifiers or consecutive sequence number segments, thereby enhancing the anti-collision ability of the version number.

[0082] Step S5400, splice the product hash segment and the current sequence number segment to form a global version number, and respond to the version number acquisition request, while persisting the corresponding relationship data of the global version number and the product identifier.

[0083] The version number maintenance service combines the commodity hash segment and the current serial number segment into a global version number according to the splicing rule declared in advance in the configuration center after obtaining the commodity hash segment and the current serial number segment. The splicing rule can be the same as the splicing rule used in the previous step, which is loaded into the memory constant area at system startup. Common forms include fixed order direct splicing of the commodity hash segment in front and the current serial number segment in back, or inserting a predefined separator such as a hyphen “-” or an underscore “_” between the two, thereby forming a string that is easy for humans to identify. In the no-separator mode, the byte sequence of the commodity hash segment and the current serial number segment is continuously written into the same buffer, and the final output is a fixed-length result. Regardless of which splicing form is used, the rule is completely consistent within the entire version service cluster, ensuring that the global version number generated by different nodes for the same commodity identifier is absolutely consistent. Taking the commodity identifier SKU-1001 as an example, if the commodity hash segment is 4a3c…e7f2 and the current serial number segment is 1205, then the global version number is 4a3c…e7f2-1205 in the hyphen mode, and 4a3ce7f21205 in the no-separator mode. All implementations follow the fixed length convention to avoid ambiguity.

[0084] Once the global version number is generated, the version number maintenance service immediately encapsulates it into a response packet and returns it to the local search service or the remote search service that initiated the request through the original data channel. The packet format is symmetrical to the request format, and the field name is also version, so that the calling end can be directly deserialized.

[0085] The above embodiments encapsulate the five links of request parsing, majority atomic increment, hash segmentation, splicing and encapsulation, and double-write persistence into an idempotent remote call, so that the version service cluster can still output a globally monotonic and non-repeating version number for the same commodity identifier in the case of node failure, network partition or high-concurrency write scenarios, completely eliminating the version conflict risks caused by cross-machine room concurrent updates; the joint use of the majority write mechanism and the Raft replication ensures the atomicity and persistence of the increment operation, and any node downtime will not lose the generated version number; the introduction of the commodity hash segment gives the version number a high-entropy feature, eliminating predictability and collision risks, while maintaining a fixed-length format for easy network transmission and storage indexing; the bidirectional mapping relationship between the global version number and the commodity identifier is persisted and synchronized to the entire cluster at the moment of generation, providing auditable metadata support for subsequent disaster recovery switching, log playback and consistency checking, overall compressing the version number generation delay to milliseconds and significantly reducing the operation and maintenance complexity, ensuring that the cross-border e-commerce platform can continuously provide strong consistency and low-latency commodity data version benchmarks in the global multi-machine room deployment.

[0086] On the basis of any embodiment of the method of the application, the global self-incrementing serial number corresponding to the commodity identifier is atomically incremented by the majority write mechanism to obtain a current serial number segment, comprising:

[0087] Step S5210, determine the counter slot corresponding to the commodity identifier, and write the current global auto-increment sequence number in the counter slot to more than half of the nodes in the version service cluster through the majority to submit a write request of the current global auto-increment sequence number plus 1 to more than half of the nodes in the version service cluster;

[0088] After receiving the version number acquisition request and parsing the commodity identifier, the version number maintenance service immediately enters the positioning process of the counter slot in the counter table. The counter table takes the commodity identifier as the unique primary key, and the underlying implementation can be a skip list in memory, a prefix key in RocksDB, or a hash slot in Redis. Accordingly, the corresponding counter slot can be located according to the commodity identifier; each record in the counter table saves an integer as the current global auto-increment sequence number. When the commodity identifier appears for the first time, the corresponding counter slot has not been established, and it can return the default value 0 when it is read and be created atomically when it is written for the first time.

[0089] After positioning, the version number maintenance service reads the current value in the corresponding counter slot, which is the current global auto-increment sequence number, as the atomic basis for this increment, and then constructs an internal write command as a write request, which contains the commodity identifier, the current global auto-increment sequence number, and the target value after adding 1. The write request is sent in parallel to all nodes in the version service cluster through a pre-established long connection or connection pool, but only requires more than half of the nodes to complete local persistence to be considered successful; the number of nodes is fixed to an odd number, typically three or five nodes, so the half threshold is two or three votes.

[0090] Step S5220, only when more than half of the nodes respond to the write request and return confirmation information that the current global auto-increment sequence number plus 1 has been written successfully, the current global auto-increment sequence number that has been committed is taken as the current sequence segment;

[0091] The version number maintenance service enters a synchronization waiting phase corresponding to a preset timeout window after issuing a write request, and the synchronization waiting phase takes majority confirmation as the only success condition. After each node receives the write request, it first checks whether the commodity identifier carried by the request and the current global auto-increment sequence number are still consistent with the local counter slot: if consistent, it immediately performs local atomic increment 1 and disk flushing, and after successful disk flushing, returns a confirmation frame containing the node ID, commodity identifier, and new value to the coordination node; if the sequence number has changed due to concurrent conflict, a failure frame is returned and the latest value is carried to facilitate retry. The coordination node maintains a confirmation table when collecting confirmation frames, the key is the node ID, and the value is a Boolean state. When the number of success markers in the confirmation table first exceeds the half threshold of the cluster, the waiting is stopped and the current global auto-increment sequence number that has been committed is marked as the current sequence number segment, and then all blocked threads in the waiting queue are released to ensure that the calling end obtains the result within milliseconds.

[0092] Step S5230, if the confirmation information of more than half of the nodes is not received, a preset failure retry process is triggered.

[0093] When the coordination node of the version number maintenance service fails to collect more than half of the confirmation frames within the preset timeout window, it immediately determines that the current round of majority write fails, and enters the failure retry process. The process first extracts the failure reason: if the node is lost, the available node list is refreshed according to the current cluster topology; if the sequence number drifts due to concurrent conflict, the latest global auto-increment sequence number is extracted as the reference for the next round. Then, the version number maintenance service updates the current global auto-increment sequence number field in the write request to the latest value, re-encapsulates the commodity identifier, the updated sequence number, and the target value, and sends them to the refreshed majority nodes through the long connection in parallel again. The retry interval can use an exponential backoff strategy, with a first retry waiting for 1 millisecond, a second retry waiting for 2 milliseconds, a third retry waiting for 4 milliseconds, and a maximum of 64 milliseconds. During the backoff period, the CPU is still kept in light polling to avoid thread starvation. Before each retry, the version number maintenance service first detects whether the local counter slot has been updated by other successful write, and if it has been updated, the new value is directly returned to avoid unnecessary repeated write; if it has not been updated, the majority write is continued. The upper limit of the number of retries is controlled by a cluster-level configuration item, which can be set to 8 by default, for example. If the upper limit is exceeded, a system busy error code is returned to the calling end, triggering upstream throttling.

[0094] The entire retry process is implemented in the form of a state machine inside the version number maintenance service, and the states include waiting for confirmation, retry interval, refreshing topology, and direct return. The state transition is driven by events, ensuring that any abnormal path can converge to success or explicit failure, and ensuring that the atomicity and monotonicity of the global auto-increment sequence number are not destroyed during the retry process.

[0095] The above embodiment puts the mapping of the commodity identifier to the global self-incrementing sequence number under the control of distributed majority, so that each increment operation remains atomic, monotonic and persistent in the cross-node network, node failure or concurrent competition scenario, thereby completely eliminating the sequence number rollback or repetition problem that may occur when the traditional single-machine counter is switched over in disaster recovery; by deploying fixed odd nodes and a half-confirmation threshold, service can still be provided when any version service node is down or network partitioned, and availability is not degraded by failure; the exponential backoff retry mechanism is superimposed on the state machine-driven event loop to ensure that a failed write automatically converges to success or an explicit exception within milliseconds, avoiding long blocking of the calling end; the global self-incrementing sequence number is immediately locked and returned as the current sequence number segment, so that the subsequent hashing, splicing and persistent link obtain an immutable reference, ensuring that the global version number is unique and sequentially incremented in the global scope; the entire process does not require human intervention, and the node topology and failure cause are automatically sensed and refreshed within the cluster, significantly reducing the complexity of operation and maintenance, and supporting the cross-border e-commerce platform to achieve second-level version number issuance, minute-level failure recovery and cross-machine room strongly consistent commodity data view in a high-concurrency commodity update scenario.

[0096] On the basis of any embodiment of the method of the application, the target log records of each group are constructed as incremental transaction log files written into the local transaction log library, so that after the commodity data in the incremental transaction log files is played back into the search engine index data of the local search service, the method comprises:

[0097] Step S6100, in response to the playback end event, broadcasting the maximum global version number in the log record that has completed playback as a watermark to the corresponding off-site search service of the respective healthy node based on the version number watermark synchronization channel established between the local search service and the off-site search service;

[0098] After the local search service completes the playback of the incremental transaction log file, a playback end event is published on the event bus, and the event carries the maximum global version number corresponding to the commodity identifier in the last log record that has been played back as the latest data water level of the current node.

[0099] The event bus can be an in-process memory queue, or a distributed message middleware such as Kafka or Redis Stream; in high-availability deployment, a message middleware with persistence capability is preferred to ensure that events are not lost due to node restart.

[0100] The publishing action is triggered synchronously by the playback thread, and the event body is serialized into a JSON or Protobuf byte stream, and the fields can include the search service node ID, the commodity identifier and its corresponding maximum global version number, the timestamp and the CRC32 checksum, which are used for the receiving end to quickly verify the integrity.

[0101] Upon the event is published, the local disaster recovery synchronization gateway encapsulates it into a watermark broadcast message immediately and sends it to all online search service nodes, such as the health node disclosed above, through a pre-established version number watermark synchronization channel. This channel can reuse the gRPC bidirectional stream or the HTTP / 2-based SSE connection between the local search service and the remote search service, supporting continuous pushing of multiple watermarks on the same TCP connection to reduce connection overhead and maintain low latency.

[0102] The format of the watermark broadcast message can be the same as that of the missing data request, with only a watermark field added. The content can be a hexadecimal string of the maximum global version number, such as 4a3c…e7f2-1205.

[0103] The sending strategy can be full broadcast or point-to-point pushing based on the cluster member list. If network jitter occurs, the watermark synchronization channel has a built-in exponential backoff retransmission mechanism. The initial backoff window is 1 millisecond, the upper limit is 32 milliseconds, the number of retransmissions is 3 by default, and if the upper limit is exceeded, a failure log is recorded and an alarm is triggered.

[0104] Step S6200, when any of the health nodes detects that its watermark falls behind its disaster recovery node by more than a preset threshold, it automatically sends the missing data request to the corresponding disaster recovery node to pull the corresponding complete transaction log file for the playback.

[0105] For the watermarks broadcast by the corresponding version number watermark synchronization channel, each search service node maintains a watermark table locally. The table takes the search service node ID as the primary key and the maximum global version number as the value. The watermarks received by each search service node can be stored in this watermark table.

[0106] Taking the search service node as an example, it continuously scans the watermark table through a background timing task or an event-driven listener, calculates the difference between the current digit number segment of the global version number under the same commodity identifier of the local node and any search service node (which can also be understood as a disaster recovery node) in the watermark table. The difference calculation adopts a per-commodity comparison strategy: for each commodity identifier in the watermark table, the current digit number segment recorded by the local node is extracted and subtracted from the corresponding current digit number segment recorded by the disaster recovery node. If the absolute value obtained is greater than the preset threshold defined by the configuration item, that is, the trigger condition is met, the automatic compensation process is triggered. The preset threshold is expressed in integer steps, and a typical setting is 1000 or 10000. It can also be adjusted to 100 or 500 according to the business sensitivity, and the modification takes effect immediately without the need to restart the node.

[0107] After the trigger condition is met, the health node embeds the latest timestamp of itself under the commodity identifier into the missing data request, and sends a pulling instruction to the backup node through the existing gRPC long connection or HTTP / 2 channel; the request message uses the previous contract, and a specific field can be added to describe the lag range. After receiving the request, the backup node locates the starting offset of the local transaction log library according to the latest timestamp, reads the subsequent complete transaction log files in batches, and returns them after compression. If the backup node detects that the latest timestamp in the request has exceeded the local log range, it returns an empty file identifier. The health node marks the commodity identifier as synchronization completed and stops subsequent pulling. After the complete transaction log files arrive, the health node still follows the business process disclosed in the foregoing of the present application, groups them according to the commodity identifier, removes the global version number, and constructs the incremental transaction log files. Then, the playback thread is started to write the commodity data into the local search engine index data. After the playback is completed, the playback end event is published again, and the watermark table is updated to form a closed loop. The entire compensation process does not require manual intervention, and the node continuously self-checks and self-repairs during operation to ensure that the version number difference of all commodity identifiers under the cross-machine room view is always below the preset threshold.

[0108] The above embodiments, through the automatic compensation closed loop triggered by watermark broadcast and threshold, completely embed the detection and repair of version number difference in the node running period, eliminating the operation and maintenance cost of manual inspection and script comparison; the event bus, long connection and exponential backoff retransmission jointly guarantee that the watermark can be reliably delivered within milliseconds, ensuring real-time consistency of the cross-machine room view; the per-commodity difference calculation and the preset threshold that can be hot updated make the lag determination have business-configurable sensitivity, avoiding excessive pulling and eliminating potential data drift; after detecting the lag, the node automatically initiates a missing data request and completes incremental playback, the entire process does not require restart or manual intervention, achieving second-level self-checking and minute-level self-healing; finally, the version number difference of all commodity identifiers is continuously compressed to within the threshold, ensuring that the search service always presents a strongly consistent and low-latency commodity data view under global multi-machine room deployment, and reducing the potential data inconsistency risk to zero during the peak of large promotions.

[0109] On the basis of any embodiment of the method of the present application, the target log records of each group are constructed into incremental transaction log files and written into the local transaction log library, for playing back the commodity data in the incremental transaction log files into the search engine index data of the local search service, comprising:

[0110] Step S7100, according to the log records in the incremental transaction log files, playing back the commodity data in the update event data of each log record into the search engine index data of the local search service;

[0111] The playback thread sequentially reads the incremental transaction log files after starting, as disclosed above, the file format is completely consistent with the local transaction log library, and each record contains the product identifier, global version number, update event data and timestamp in turn. Based on this, the corresponding log record is first mapped to the user mode buffer in zero copy mode, and then the product data field in the update event data is extracted, including but not limited to inventory value, price value, on-off shelf status and other platform defined indexable attributes. After parsing, the playback thread locates the document ID in the search engine index data according to the product identifier: if the document exists, perform in-place update, write inventory value, price value and other fields into the inverted index and forward storage; if the document does not exist, create a new document and insert the index. During the update process, the global version number is used as the optimistic lock condition field to participate in writing, and the index layer compares the to-be-written version number with the existing version number first, and only allows update when the to-be-written version number is larger, preventing data rollback caused by concurrent playback.

[0112] Step S7200, during the playback process, if it is detected that the inventory value in the product data corresponding to the same product identifier is less than zero, an inventory oversold alarm message is triggered, the visibility of the product data of the product identifier in the search engine index is suspended until the inventory correction log arrives and the playback is successful again to restore the display.

[0113] During the process of writing product data into the search engine index data, the playback thread extracts the inventory value field from the update event data whenever it encounters a log record, and parses it into a signed long integer; if the parsing result is negative, it is immediately determined that the product is in an oversold state, then a inventory oversold alarm message is constructed, the message body can contain the product identifier, the current inventory value, the global version number and the detection timestamp, then it is sent through the preset alarm channel; the alarm channel can be an in-process event bus, or a Kafka topic or an HTTP callback endpoint, and the sending strategy supports synchronous blocking and asynchronous buffering two modes, determined by the corresponding configuration item. At the same time of sending the alarm, the search engine index layer immediately takes effect on the visibility shielding of the corresponding product identifier: in the inverted index, the visibility flag of the product document is set to false, and the replica of the document in the cache is refreshed synchronously; at this time, even if the external query hits the product, it will be intercepted by the filtering logic and not displayed to the buyer.

[0114] During the visibility shielding period, the playback thread continues to listen to subsequent log records; when the inventory correction log arrives and is successfully played back, the corrected inventory value is read again, and if the value is greater than or equal to zero, the shielding is immediately lifted: the visibility flag in the inverted index is set to true, and a visibility recovery event is sent to refresh the cache. The entire inventory oversold detection and visibility control process can be completed within a single log record processing period, with a delay of less than one millisecond, and the impact on playback throughput is negligible.

[0115] The above embodiment uses the global version number of the present application as a monotonic increasing and cross-node unique optimistic lock reference, so that the replay thread can complete atomic update with single record granularity when writing the incremental transaction log file into the search engine index data, ensuring that any inventory correction log covers the old value as soon as it is replayed, completely eliminating dirty writes and version rollback in concurrent scenarios; when the inventory value is parsed as negative, overselling judgment, alarm sending and index visibility shielding are completed within the same processing cycle, and the validity of the subsequent correction log is verified again through the global version number, ensuring that only the high version record with non-negative inventory that is truly updated can trigger visibility recovery, thereby realizing closed-loop control from detection, interception to automatic release within milliseconds of delay, significantly reducing overselling risk and improving inventory consistency and buyer experience of cross-border e-commerce platforms during peak periods.

[0116] On the basis of any embodiment of the method of the present application, the target log records of each group are constructed into an incremental transaction log file and written into a local transaction log library, so as to replay the commodity data in the incremental transaction log file into the search engine index data of the local search service, comprising:

[0117] Step S8100, according to the log records in the incremental transaction log file, replay the commodity data in the update event data in each log record into the search engine index data of the local search service;

[0118] As in the previous embodiment, after the playback thread is started, the incremental transaction log file is read in sequence, which uses the binary format of the local transaction log library, and each record fixedly contains the commodity identifier, global version number, update event data and timestamp. The reading uses a zero-copy memory mapping method to directly map the file content to the user buffer, avoiding additional copying between the kernel and user space; then each record is parsed in reverse order to deserialize the update event data into a structured object. For example, the update event data at least contains the price value, price timestamp, inventory value, on- / off-shelf status and commodity identifier. After parsing, the playback thread performs a point search in the search engine index data using the commodity identifier as the key: if the document exists, the corresponding document ID is located, and the global version number is used as the optimistic lock condition, and only when the to-be-written version number is greater than the existing version number in the index, the update is performed; if the document does not exist, a new document is created, and the inverted and forward fields are initialized. The update action is divided into two steps: the first step is to write the forward storage, and the price value, inventory value and other fields are appended to the row storage segment; the second step is to update the inverted index, and the price interval, inventory interval and other searchable fields are refreshed to the corresponding dictionary. The two steps are committed in the same transaction, and the changes are written to the memory buffer before the commit. When the buffer is full or the timer reaches, the disk is flushed to generate a new index segment. After the disk flushing is completed, the playback thread persists the maximum global version number processed to the local watermark file, ensuring that the process can be restarted for breakpoint continuation; if the number of index segments reaches the background merging threshold, the segment merging is automatically triggered by the merging thread, and the new write continues to be served through double buffering during the merging period, and the query is not affected. The whole process runs in a single-thread or multi-thread pool mode, the size of the thread pool is dynamically adjusted according to the number of CPU cores, and the tasks are transmitted between threads through a lock-free queue to avoid lock contention.

[0119] In step S8200, during the playback process, if the difference between the current price timestamp and the previous price timestamp in the commodity data in the log record is less than a preset time threshold, the corresponding commodity identifier is marked as a high-frequency price commodity.

[0120] When processing the incremental transaction log file, the playback thread calculates the difference between the current price timestamp in the commodity data in each log record and the last price timestamp retained in the memory; the difference is in milliseconds, and a fixed threshold is given by a corresponding configuration item, and a typical value can be 5000 milliseconds, or can be adjusted to 1000 milliseconds or 30000 milliseconds according to business needs. If the difference is less than the threshold, the corresponding commodity is immediately considered as a high-frequency price commodity, and the corresponding commodity identifier is written to the high-frequency price commodity set, which is implemented as a thread-safe map table with the commodity identifier as the key and the first detection timestamp as the value. To prevent misjudgment, the same commodity identifier can be confirmed only when the two consecutive detections meet the conditions, and after confirmation, the playback thread calls the cache layer interface to enable the price cache lock.

[0121] Step S8300, for the high-frequency price commodity, enable price cache lock in the cache layer, and return the latest price to the commodity query request within the cache lock validity period;

[0122] After confirming that a certain commodity identifier is marked as a high-frequency price commodity, the playback thread immediately calls the cache layer interface to add a write price cache lock for the commodity identifier. The price cache lock is a key-level lock, with the key name being the commodity identifier and the value structure containing the latest price value, lock creation timestamp, and validity period length. The cache layer can be implemented based on local Caffeine, Redis, or Memcached; taking Redis as an example, the lock record can be written in STRING type, with the key format being price_lock:{sku_id} and the value format being "{price}:{lock_ts}:{ttl}", and the expiration time is set immediately after writing through PEXPIRE. The lock validity period is controlled by the configuration item price_lock_ttl, which defaults to 5 seconds and can be updated to 1 second or 30 seconds through the configuration center at runtime without the need to restart the node.

[0123] Within the validity period, the local search service is responsible for controlling all commodity query requests for the commodity identifier to first access the cache layer: if the price cache lock is hit, the latest price value is directly returned, and the search engine index data is no longer retrieved; if it is not hit, the index is queried according to the normal link and the cache is written back. To prevent price jitter caused by concurrent writing, the cache layer uses atomic commands when writing locks to ensure that only one thread successfully locks at the same time, and the remaining threads wait for the lock to expire and then immediately retry.

[0124] Step S8400, when the cache lock reaches the validity period or receives the price stabilization log, the price cache lock is released.

[0125] The release of the price cache lock can be triggered by two parallel control paths: the first path is a timing invalidation, and the cache layer carries a valid period in milliseconds when writing the lock record, which is uniformly managed by the configuration item price_lock_ttl, which supports runtime hot update; after reaching the valid period, the cache layer automatically deletes the price_lock:{sku_id} key, making the lock record invalid immediately, and subsequent query requests are no longer subject to lock restrictions. The second path can be event-driven, and when the playback thread parses the price stable log of the corresponding commodity, that is, the interval between the price timestamp of the log record and the previous record is greater than the preset threshold, the cache layer interface is called to actively delete the price cache lock of the corresponding commodity identifier; the delete action uses atomic commands to ensure idempotency in high-concurrency scenarios and avoid race conditions caused by repeated deletions. The two paths can share the same unlock hook, which first checks whether the lock key exists, and if it does, it executes the deletion and broadcasts the price unlock event, removing the commodity identifier from the high-frequency price commodity set; if the lock key has been automatically cleaned up due to expiration, the hook directly returns success to prevent accidental deletion.

[0126] After the unlock is completed, the local search service restores the real-time price reading link for the commodity identifier, and the query request penetrates the search engine index data again, ensuring that the buyer-side price display is synchronized with the latest backend state; the entire release process is completed in microseconds, with no perceptible query delay.

[0127] In large-scale promotion scenarios, the simultaneous price changes of a large number of sellers result in intensive updates of commodity price timestamps, and the search engine index faces high-frequency write impact; if the index is directly played back, the inverted dictionary and the forward row need to be frequently rebuilt, the query delay increases dramatically, and buyers may see continuous price fluctuations in a short period of time, resulting in a damaged experience. To address this issue, in the above embodiment, the playback thread uses the global version number as an optimistic lock condition and only updates when the version number to be written is higher than the existing version number, completely eliminating the coverage of old data; after detecting that the current price timestamp is less than the preset threshold from the previous one, the commodity identifier is automatically marked as a high-frequency price commodity, and a write price cache lock is immediately added to the cache layer; all queries directly return the latest price within the lock validity period, avoiding index retrieval; after the lock expires or is explicitly triggered by the price stable log, the cache lock is atomically released, and the query link is restored to real-time penetration. The global version number ensures the idempotency and data consistency of playback, and the cache lock shields high-frequency fluctuations, providing stable price display with millisecond-level delay even at the peak of promotions, significantly reducing index write pressure and improving buyer experience.

[0128] Please refer to Figure 2According to one aspect of the present application, an e-commerce search service recovery device is provided, which comprises a missing request module 3100, a target determination module 3200, a write playback module 3300, and a service restart module 3400. The missing request module 3100 is configured to start a local search service deployed on a current fault node, and send a missing data request to an off-site search service of any healthy node, wherein the request contains the latest timestamp of the latest log record in a local transaction log library, and the healthy node and the fault node are backup nodes of each other. The target determination module 3200 is configured to receive a complete transaction log file returned by the off-site search service, which is generated after the latest timestamp, group the log records in the complete transaction log file according to the commodity identifier, determine a target log record with the largest global version number in each group, and maintain the global version number by a distributed version service cluster independent of the local search service and the off-site search service. The write playback module 3300 is configured to construct the target log record of each group into an incremental transaction log file and write it into the local transaction log library, so as to play back the commodity data in the incremental transaction log file into the search engine index data of the local search service. The service restart module 3400 is configured to re-register the local search service to a service registration center after playback is completed, and execute a query based on the search engine index data through the local search service in response to an externally accessed commodity query request.

[0129] On the basis of any embodiment of the device of the present application, after the missing request module 3100, the present application further comprises an update response module configured to obtain corresponding update event data from the local search service in response to a commodity update notification message, wherein the update event data comprises commodity data of a target commodity to be updated, and the commodity data comprises a commodity identifier of the target commodity; a version determination module configured to send a version number acquisition request to the distributed version service cluster to obtain a global version number generated based on the commodity identifier carried in the request; and a log construction module configured to construct a log record by associating the global version number with the update event data and a timestamp of the current time, and write the log record into a local transaction log library of the local search service.

[0130] On the basis of any embodiment of the device of the present application, the device further comprises the following modules run by the version number maintenance service in the version service cluster: an identification determination module, configured to receive a version number acquisition request from the local search service or the off-site search service, and parse a commodity identification in the request; a number segment determination module, configured to atomically increment a global auto-increment sequence number corresponding to the commodity identification by a majority write mechanism to obtain a current sequence number segment; a hash determination module, configured to splice the commodity identification and the current sequence number segment and perform hash calculation to generate a fixed-length commodity hash segment; and a version number generation module, configured to splice the commodity hash segment and the current sequence number segment to form a global version number, and respond to the version number acquisition request, and persist the correspondence between the global version number and the commodity identification.

[0131] On the basis of any embodiment of the device of the present application, the number segment determination module comprises: a request group sending module, configured to determine a counter slot corresponding to the commodity identification, and submit a write request of the current global auto-increment sequence number plus 1 to more than half of the nodes in the version service cluster through a majority write flow; a confirmation verification module, configured to, when and only when more than half of the nodes return confirmation information that the current global auto-increment sequence number plus 1 has been successfully written in response to the write request, take the current global auto-increment sequence number that has been submitted as the current sequence number segment; and an exception retry module, configured to, if confirmation information from more than half of the nodes is not received, trigger a preset failure retry process.

[0132] On the basis of any embodiment of the device of the present application, after the write playback module 3300, the device further comprises: a watermark broadcast module, configured to, in response to a playback end event, broadcast a maximum global version number in the log record that has completed playback as a watermark to the off-site search service corresponding to each healthy node based on a version number watermark synchronization channel established between the local search service and the off-site search service; and an automatic tracking module, configured to, when any of the healthy nodes detects that its own watermark lags behind its disaster recovery node by more than a preset threshold, automatically send the missing data request to the corresponding disaster recovery node to pull a corresponding complete transaction log file for the playback.

[0133] On the basis of any embodiment of the device of the application, the write playback module 3300 comprises: a playback execution module configured to play back commodity data in update event data in each log record into search engine index data of a local search service according to log records in the incremental transaction log file; a sell-out processing module configured to, in a playback process, if it is detected that a stock value in commodity data corresponding to a same commodity identifier is less than zero, trigger a stock sell-out alarm message, suspend visibility of commodity data of the commodity identifier in the search engine index, and restore display after a stock correction log arrives and playback is successful again.

[0134] On the basis of any embodiment of the device of the application, the write playback module 3300 comprises: a playback execution module configured to play back commodity data in update event data in each log record into search engine index data of a local search service according to log records in the incremental transaction log file; a high-frequency identification module configured to, in a playback process, if a difference between a current price timestamp and a previous price timestamp in commodity data in the log record is less than a preset time threshold, mark a corresponding commodity identifier as a high-frequency price commodity; a cache locking module configured to, for the high-frequency price commodity, enable a price cache lock at a cache layer, and return a most recent price to a commodity query request within a valid period of the cache lock; and a cache unlocking module configured to, when the cache lock reaches the valid period or a price stabilization log is received, release the price cache lock.

[0135] Another embodiment of the application also provides an e-commerce search service recovery device. As shown in Figure 3 The e-commerce search service recovery device includes a processor, a computer readable storage medium, a memory, and a network interface connected through a system bus. The computer readable nonvolatile storage medium of the e-commerce search service recovery device stores an operating system, a database, and computer readable instructions. The database can store information sequences, and the computer readable instructions, when executed by the processor, can enable the processor to implement an e-commerce search service recovery method.

[0136] The processor of the e-commerce search service recovery device is used to provide computing and control capabilities to support the operation of the entire e-commerce search service recovery device. The memory of the e-commerce search service recovery device can store computer readable instructions, which, when executed by the processor, can enable the processor to execute the e-commerce search service recovery method of the application. The network interface of the e-commerce search service recovery device is used to connect and communicate with a terminal.

[0137] Those skilled in the art can understand, Figure 3The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the e-commerce search service recovery device to which the scheme of the present application is applied. The specific e-commerce search service recovery device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0138] The processor in the embodiment is configured to execute the specific functions of each module in Figure 2 The memory stores the program codes and various data required for executing the above-mentioned modules or sub-modules. The network interface is configured to realize data transmission between the user terminal or the server. The non-volatile readable storage medium in the embodiment of the present application stores the program codes and data required for executing all modules in the e-commerce search service recovery device of the present application, and the server can call the program codes and data of the server to execute the functions of all modules.

[0139] The present application also provides a non-volatile readable storage medium storing computer readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the e-commerce search service recovery method of any embodiment of the present application.

[0140] The present application also provides a computer program product, including computer programs / instructions, which, when executed by one or more processors, implement the steps of the method described in any embodiment of the present application.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of each method. The storage medium can be a computer readable storage medium such as a magnetic disc, an optical disc, a read-only memory (ROM), or a random access memory (RAM).

[0142] In summary, in the multi-machine room deployment scenario of the e-commerce platform, the present application assigns a globally monotonically increasing product-level version number to each product record, and uses timestamp-based log pulling between disaster recovery nodes, product dimension deduplication, minute-level incremental playback, and real-time inventory / price consistency verification to achieve the comprehensive effect of zero manual disaster recovery, minute-level recovery, synchronization of search results with the latest product data, and high availability, strong consistency, and zero interruption of user experience in global peak promotions.

Claims

1. An e-commerce search service recovery method, characterized by, Comprise: Start the local search service of the current fault node deployment, send a missing data request to the remote search service of any healthy node, the request contains the latest timestamp of the latest log record in the local transaction log library, and the healthy node and the fault node are each backup nodes; Receive the complete transaction log file returned by the remote search service generated after the latest timestamp, group the log records in the complete transaction log file by commodity identifier, determine the target log record with the largest global version number in each group, and the global version number is globally monotonically increasing and maintained by a distributed version service cluster independent of the local search service and the remote search service; The target log record of each group is constructed as an incremental transaction log file and written into the local transaction log library, so that the commodity data in the incremental transaction log file is replayed into the search engine index data of the local search service; After the replay is completed, the local search service is re-registered with the service registration center, and the local search service responds to external access commodity query requests based on the search engine index data to execute queries.

2. The method of claim 1, wherein, After starting the local search service of the current fault node deployment, it further comprises: In response to a commodity update notification message, obtain the corresponding update event data by the local search service, the update event data including the commodity data of the target commodity to be updated, wherein the commodity data includes the commodity identifier of the target commodity; Send a version number acquisition request to the distributed version service cluster to obtain a global version number generated based on the commodity identifier carried in the request; The global version number is associated with the update event data and the timestamp of the current time to construct a log record, which is written into the local transaction log library of the local search service.

3. The method of claim 2, wherein, Send a version number acquisition request to the distributed version service cluster to obtain a global version number generated based on the commodity identifier carried in the request, comprising the following steps executed by the version number maintenance service in the version service cluster: Receive the version number acquisition request from the local search service or the remote search service, and parse the commodity identifier obtained therein; Atomically increment the global auto-increment sequence number corresponding to the commodity identifier through the majority write mechanism to obtain the current sequence segment; Hash calculation is performed after splicing the commodity identifier and the current sequence segment to generate a commodity hash segment of fixed length; Splice the commodity hash segment and the current sequence segment to form a global version number to respond to the version number acquisition request, and persist the correspondence data of the global version number and the commodity identifier.

4. The method of claim 3, wherein, Atomically increment the global auto-increment sequence number corresponding to the commodity identifier through the majority write mechanism to obtain the current sequence segment, comprising: Determine the counter slot corresponding to the commodity identifier, and through the majority write flow, submit a write request of the current global auto-increment sequence number plus 1 to more than half of the nodes in the version service cluster; When and only when more than half of the nodes return the confirmation information that the current global self-incrementing sequence number plus 1 has been written successfully, the current sequence number segment is committed as the current sequence number segment; If the confirmation information of more than half of the nodes is not received, a preset failure retry process is triggered.

5. The e-commerce search service recovery method of any one of claims 1 to 4, wherein, The target log records of each group are constructed as incremental transaction log files and written into the local transaction log library, so that the commodity data in the incremental transaction log files is replayed into the search engine index data of the local search service, including: In response to a replay end event, a maximum global version number in the log record that has completed replay is broadcast as a watermark to the corresponding off-site search service of each healthy node based on a version number watermark synchronization channel established between the local search service and the off-site search service; When any of the healthy nodes detects that its own watermark falls behind its standby node by more than a preset threshold, the missing data request is automatically sent to the corresponding standby node to pull the corresponding complete transaction log file for the replay.

6. The e-commerce search service recovery method of any one of claims 1 to 4, wherein, The target log records of each group are constructed as incremental transaction log files and written into the local transaction log library, so that the commodity data in the incremental transaction log files is replayed into the search engine index data of the local search service, including: According to the log records in the incremental transaction log files, the commodity data in the update event data in each log record is replayed into the search engine index data of the local search service; During the replay process, if it is detected that the inventory value in the commodity data corresponding to the same commodity identifier is less than zero, an inventory oversold alarm message is triggered, and the visibility of the commodity data of the commodity identifier in the search engine index is suspended until the inventory correction log arrives and the replay is successful again.

7. The e-commerce search service recovery method of any one of claims 1 to 4, wherein, The target log records of each group are constructed as incremental transaction log files and written into the local transaction log library, so that the commodity data in the incremental transaction log files is replayed into the search engine index data of the local search service, including: According to the log records in the incremental transaction log files, the commodity data in the update event data in each log record is replayed into the search engine index data of the local search service; During the replay process, if it is detected that the difference between the current price timestamp and the previous price timestamp in the commodity data in the log record is less than a preset time threshold, the corresponding commodity identifier is marked as a high-frequency price commodity; For the high-frequency price commodity, a price cache lock is enabled at the cache layer, and only the latest price is returned to the commodity query request within the validity period of the cache lock; When the cache lock reaches the validity period or a price stabilization log is received, the price cache lock is released.

8. An e-commerce search service recovery apparatus characterized by comprising: The missing request module is configured to start the local search service deployed by the current fault node, and send a missing data request to the off-site search service of any healthy node, the request containing the latest timestamp of the latest log record in the local transaction log library, the healthy node and the fault node being standby nodes of each other; The missing request module is configured to start the local search service deployed by the current fault node, and send a missing data request to the off-site search service of any healthy node, the request containing the latest timestamp of the latest log record in the local transaction log library, the healthy node and the fault node being standby nodes of each other; The target determining module is configured to receive the complete transaction log file generated after the latest time stamp returned by the off-site search service, group the log records in the complete transaction log file according to the commodity identifier, determine the target log record with the largest global version number in each group, and the global version number is globally monotonically increasing and maintained by a distributed version service cluster independent of the local search service and the off-site search service. The write-back module is configured to construct the target log record of each group into an incremental transaction log file and write the incremental transaction log file into the local transaction log library, so as to replay the commodity data in the incremental transaction log file into the search engine index data of the local search service. The service restart module is configured to re-register the local search service to a service registry center after the replay is completed, and execute a query based on the search engine index data through the local search service in response to an externally accessed commodity query request.

9. An e-commerce search service recovery apparatus comprising a central processing unit and a memory, characterized by, The central processing unit is configured to call a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized by The computer readable instructions store a computer program implemented according to the method of any one of claims 1 to 7, and when the computer program is called and run by a computer, the steps included in the corresponding method are executed.

Images