Method and system for high throughput inventory update based on semaphores and read-write locks

Abstract

The application discloses a kind of based on signal amount and read-write lock implementation high throughput's inventory update method and system, above-mentioned method includes: receiving the inventory deduction request initiated by business system, obtains commodity id list and the required deduction quantity of each commodity according to inventory deduction request;According to commodity id list, the inventory amount recorded in cache of commodity and the required deduction quantity of commodity, the inventory in cache of commodity is deducted by semaphore API one by one;According to commodity id list, the inventory amount in database of commodity and the required deduction quantity of commodity, the inventory in database of commodity is deducted one by one.The application uses two kinds of data structures supporting concurrent operation, read-write lock and semaphore, to realize the update and query of inventory, supports safe multithreading high concurrency update when updating, effectively improves the inventory updating capability of system.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for achieving high-throughput inventory updates based on semaphores and read-write locks. Background Technology

[0002] In e-commerce systems or inventory management systems, concurrent updates to product inventory are often required. Considering the high access volumes in e-commerce systems during certain scenarios (such as flash sales), concurrent inventory updates often need to meet high throughput requirements while ensuring inventory accuracy (overselling). Currently, there are two commonly used technical solutions:

[0003] 1. Simply use a database for storage; queries and updates directly request the database.

[0004] 2. Use a database + cache for storage. Queries request the cache to improve performance, while updates simultaneously update both the database and the cache. Data consistency between the two is achieved using mutex locks.

[0005] The current technical solution has the following problems:

[0006] 1. If you simply use a database as storage, you can avoid data consistency issues, but the performance bottleneck will be limited by the database system. When the system's concurrency is too high (such as in a flash sale scenario), the database will have difficulty supporting the massive request load, which may cause the database to freeze or crash.

[0007] 2. Using mutex locks and caching can effectively improve the system's query performance, but it will create new performance bottlenecks for update operations. This is because, in order to ensure data consistency during updates, a mutex lock needs to be added to the product, which means that only one request can update the inventory for the same product at a time, thus limiting the system's throughput. Summary of the Invention

[0008] This invention addresses the performance bottlenecks of existing storage methods by proposing a high-throughput inventory update method and system based on semaphores and read-write locks.

[0009] In a first aspect, embodiments of this application provide a high-throughput inventory update method based on semaphores and read-write locks, including:

[0010] Deduction request receiving steps: Receive inventory deduction requests initiated by the business system, and obtain a list of product IDs and the required deduction quantity for each product based on the inventory deduction request;

[0011] Cache deduction steps: Based on the product ID list, the inventory of the product recorded in the cache, and the required deduction quantity for the product, the inventory of each product in the cache is deducted one by one through the semaphore API;

[0012] Database deduction steps: Based on the product ID list, the product's inventory in the database, and the required deduction quantity, deduct the product's inventory in the database one by one.

[0013] The above-described inventory update method includes the following step:

[0014] Inventory quantity query steps: Query the inventory quantity of the products in the product ID list recorded in the cache;

[0015] Inventory deduction steps: If the inventory is greater than 0, the quantity of the product in the cache is deducted according to the product requirements through the semaphore API; if the inventory in the cache is successfully deducted, the mapping of the successfully deducted product ID and the corresponding deducted quantity is recorded in the thread context; if the inventory in the cache is not successfully deducted, the product is marked as having inconsistent inventory data according to the adjustment probability threshold, the mapping of the successfully deducted product ID and the corresponding deducted quantity recorded in the thread context is released, and the information of insufficient product inventory is returned.

[0016] The above-described inventory update method further includes, in the step of deducting the cache:

[0017] First mapping release step: If the inventory quantity is 0, mark the product as having inconsistent inventory data according to the adjustment probability threshold, and release the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context through the semaphore API, and return the information that the product inventory is insufficient.

[0018] The second mapping release step: If the inventory quantity is less than 0, the mapping of the successfully deducted product ID and the corresponding deducted quantity recorded in the thread context is released through the semaphore API, and information about abnormal product inventory data is returned.

[0019] The aforementioned inventory update method, wherein the deduction request receiving step further includes:

[0020] The first inspection step is to sort the product ID list and then repeatedly check whether the products in the sorted product ID list already exist in the cache and whether there is a flag indicating an inconsistency in inventory data.

[0021] First initialization step: If the product ID does not exist in the cache, and / or there is a marker indicating an inconsistency in inventory data, then the product ID list is initialized.

[0022] The above-described inventory update method, wherein the first initialization step includes:

[0023] First write lock acquisition step: Add a write lock from the read-write lock to the product ID list;

[0024] First inventory information update step: Query the inventory information of the products in the product ID list from the database, and update the cache one by one with the retrieved inventory information;

[0025] First marker deletion step: If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data.

[0026] First write lock release step: Release the acquired write lock.

[0027] The above-described inventory update method further includes the step of receiving the deduction request: adding a read lock to the sorted list of product IDs and recording the held read lock in the thread context.

[0028] The above-described inventory update method, wherein the database deduction step includes:

[0029] Information return steps: If the product's inventory in the database is successfully deducted, a message indicating successful deduction will be returned.

[0030] Rollback steps: If the inventory of the product in the database is not successfully deducted, the product that failed to be deducted is marked as having inconsistent inventory data, and the semaphore in the database and the cache is rolled back through the transaction rollback hook.

[0031] The aforementioned inventory update method, wherein the rollback step further includes:

[0032] The third mapping release step: After the rollback is completed, the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context is released through the semaphore API;

[0033] Read-write lock release steps: Release all currently held read and write locks recorded in the thread context.

[0034] The aforementioned inventory update method also includes:

[0035] Query request receiving steps: Receive the inventory query request initiated by the business system, and obtain the product ID list and the expected inventory quantity of each product based on the inventory query request.

[0036] The above-described inventory update method, wherein the query request receiving step includes:

[0037] The second inspection step is to repeatedly check whether the products in the product ID list already exist in the cache, and whether there is a flag indicating an inconsistency in inventory data.

[0038] Second initialization step: If the product ID does not exist in the cache, and / or there is a flag indicating an inconsistent inventory data state, then the product ID list is initialized;

[0039] Inventory information query steps: Query the inventory information of the products in the cache in a loop and record it. If the inventory of the products in the cache is less than the expected inventory quantity, mark the products as having an inventory quantity that does not meet the requirements.

[0040] The steps to obtain the query results include: returning the inventory information of the records and marking the status of inventory quantity not meeting the requirements.

[0041] The above-described inventory update method, wherein the second initialization step includes:

[0042] The second write lock acquisition step: Add a write lock from the read-write lock to the product ID list;

[0043] The second inventory information update step is to query the inventory information of the products in the product ID list from the database and update the cache one by one with the retrieved inventory information;

[0044] Second marker deletion step: If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data.

[0045] Second write lock release step: Release the acquired write lock.

[0046] Secondly, embodiments of this application provide a high-throughput inventory update system based on semaphores and read-write locks, used to implement the inventory update method of the first aspect described above, including:

[0047] Deduction Request Receiving Unit: Receives inventory deduction requests initiated by the business system, and obtains a list of product IDs and the required deduction quantity for each product based on the inventory deduction request;

[0048] Cache deduction unit: Based on the product ID list, the inventory quantity of the product recorded in the cache, and the required deduction quantity of the product, the unit deducts the inventory of the product in the cache one by one through the semaphore API. If the inventory of the product in the cache is successfully deducted, the mapping of the successfully deducted product ID and the corresponding deduction quantity is recorded in the thread context. If the inventory of the product in the cache is not successfully deducted, the product that failed to deduct is marked as having inconsistent inventory data according to the adjustment probability threshold, and the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context is released.

[0049] Database deduction unit: Based on the product ID list, the product's inventory in the database, and the required deduction quantity, deduct the product's inventory in the database one by one; if the product's inventory in the database is successfully deducted, a successful deduction message is returned; if the product's inventory in the database is not successfully deducted, the product with the failed deduction is marked as having inconsistent inventory data and the semaphores in the database and cache are rolled back.

[0050] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0051] This application employs two data structures that support concurrent operations: read-write locks and semaphores, to implement inventory updates and queries. During updates, it supports safe multi-threaded high-concurrency updates, effectively improving the system's inventory update capabilities. Furthermore, when the inventory level meets the requirements, it supports multiple requests to simultaneously perform inventory update operations on the same one or more products, thereby increasing the system's throughput. Attached Figure Description

[0052] Figure 1 A schematic diagram illustrating the steps of a high-throughput inventory update method based on semaphores and read-write locks provided by the present invention;

[0053] Figure 2 An initialization flowchart provided for this invention;

[0054] Figure 3 The flowchart for deducting inventory provided by this invention;

[0055] Figure 4 The flowchart for checking inconsistencies in inventory data provided by this invention;

[0056] Figure 5 A flowchart for the transaction rollback hook processing provided by this invention;

[0057] Figure 6 This invention provides a framework diagram of a high-throughput inventory update system based on semaphores and read-write locks. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0059] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0060] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0061] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0062] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, it should be noted that these embodiments are not intended to limit the present invention. All equivalent changes or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are within the scope of protection of the present invention.

[0063] Example 1:

[0064] Figure 1 This invention provides a schematic diagram illustrating the steps of a high-throughput inventory update method based on semaphores and read-write locks, as shown below. Figure 1 As shown in the figure, this embodiment discloses a specific implementation of a high-throughput inventory update method (hereinafter referred to as the "method") based on semaphores and read-write locks.

[0065] Specifically, the method disclosed in this embodiment mainly includes the following steps:

[0066] Step S1: Receive the inventory deduction request initiated by the business system, and obtain the product ID list and the required deduction quantity for each product based on the inventory deduction request;

[0067] Specifically, the business system initiates a set of inventory deduction requests for products. The parameters of this inventory deduction request include at least a list of product IDs and the required deduction quantity for each product, where the required deduction quantity refers to the quantity of inventory to be deducted for each product. Based on the received inventory deduction requests, the required deduction quantities for each product are obtained. The purpose is to deduct the required deduction quantities from the product inventory in the database. For example, if the required deduction quantity for product A is 6, then the business system requests to deduct 6 units of product A inventory.

[0068] In this context, "business system" refers to the upper-level system using this invention, such as an e-commerce system or a WMS system. "Product ID" refers to a key-value pair that uniquely identifies a product; this value must be unique and immutable within the business system.

[0069] Furthermore, step S1 also includes:

[0070] Step S11: Sort the product ID list and iteratively check whether the products in the sorted product ID list already exist in the cache, and whether there is a flag indicating an inconsistency in inventory data.

[0071] Step S12: If the product ID does not exist in the cache, and / or there is a flag indicating an inconsistency in inventory data, then the product ID list is initialized.

[0072] The product ID list can be sorted in any order, such as ascending byte order, but it must be ensured that a fixed sorting method is used each time. The purpose of sorting the product ID list is to prevent deadlocks when locking the product sequence.

[0073] It's important to note that deadlock refers to a situation where multiple threads compete for multiple locks during the locking process, resulting in all threads waiting for a lock. For example, if thread 1 and thread 2 both want to acquire locks A and B, and the locking order is that thread 1 acquires lock A first and then lock B, while thread 2 acquires lock B first and then lock A, after thread 1 acquires lock A and thread 2 acquires lock B, if thread 1 then tries to acquire lock B and thread 2 tries to acquire lock A, neither thread will be able to acquire the lock they want. This situation is called deadlock.

[0074] It should be noted that the inventory data inconsistency flag proposed in this invention refers to the flag added when the inventory data in the cache and the database are inconsistent, and it is divided into two cases:

[0075] (1) The situation where there is sufficient inventory in the cache but insufficient inventory in the database. For example, the business system requests to deduct 6 units of product A from the inventory. The cache records that there are 8 units of product A in stock (greater than or equal to 6), but the database records that there are 3 units of product A in stock (less than 6).

[0076] When the above situation (1) occurs, the present invention will mark this product as having inconsistent inventory data.

[0077] (2) The cached inventory is 0, but the database actually has inventory. For example, the business system requests to deduct 6 units of product A from the inventory. The cache records that product A has 0 units in stock, but the database records that product A has 6 units (greater than or equal to 6 units).

[0078] When the above situation (2) occurs, the present invention has a certain probability of marking this product as having inconsistent inventory data. The marking method is to record a key-value pair in the cache. The key is "not_consistency:" + product ID, and the value can be any value. The specific marking process will be described in detail in the following embodiments.

[0079] In the above embodiment, each product in the product ID list is extracted sequentially. For each product extracted, it is checked whether the product already exists in the cache system. If it does not exist, the product ID is recorded and the next product is extracted. If it exists, it means that the product has been initialized and is skipped. The next product is extracted until all products in the product ID list have been extracted. After all products have been extracted, the product ID list that does not exist in the cache system is initialized.

[0080] Next, extract each product from the product ID list in sequence. For each product extracted, check if there is a marker indicating inconsistent inventory data. If not, it means the inventory data for that product is correct, so skip it and continue extracting the next product. If there is a marker, record the product ID and continue extracting the next product until all products in the product ID list have been extracted. After all products have been extracted, initialize the list of product IDs marked with inconsistent inventory data.

[0081] The process of initializing the product ID list is as follows:

[0082] Specifically, step S12 includes:

[0083] Step S121: Add a write lock to the product ID list; if the lock is successfully added, proceed to step S122; otherwise, it indicates that the initialization failed, the initialization process is terminated, and the subsequent inventory deduction process cannot be performed.

[0084] It's important to note that a read-write lock is a pair of locks, consisting of a read lock and a write lock. The write lock is a mutex, while the read lock is a shared lock. Acquiring a write lock requires that neither the read lock nor the write lock is held by any other thread. Acquiring a read lock requires that the write lock is not held by any other thread; whether or not the read lock is held by other threads is irrelevant. Furthermore, "read-write lock-read lock" refers to the read lock within a read-write lock, and "read-write lock-write lock" refers to the write lock within a read-write lock. A mutex is used to control that only one thread can execute a specific block of logic (function or code snippet).

[0085] Step S122: Query the inventory information of the products in the product ID list from the database, and update the cache one by one with the retrieved inventory information;

[0086] The process involves updating the retrieved inventory information to the cache system one by one, recording it in the form of key (product ID) - value (inventory quantity). If any failure occurs during step S122, it indicates that the initialization has failed, and the acquired write lock is released, terminating the initialization process and preventing subsequent inventory deduction. If the retrieved inventory information is successfully updated to the cache one by one, step S123 is executed.

[0087] Step S123: If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data. It should be noted that step S123 is only required during the initialization process of the product ID list that has a marker indicating inconsistent inventory data.

[0088] Step S124: Release the acquired write lock. Initialization ends.

[0089] Step S1 also includes registering a database transaction rollback hook, which can realize the unified release of read-write locks and the rollback of cached inventory data.

[0090] Specifically, the transaction rollback hook is used when the modification operation of the database system fails for some reason. In this case, all modifications of the database system will be rolled back to the state before the transaction started. When this happens, the database access framework (such as Mybatis) will notify the caller, so that the caller (referring to this invention) can perform some personalized operations (in this invention, the cached semaphore data will be rolled back).

[0091] Step S13: Acquire a read lock from the read-write lock on the sorted list of product IDs, and record the held read lock in the thread context. If locking fails, the inventory deduction process ends, and the business system is notified of the locking failure. Optionally, step S13 can be re-executed after a period of time.

[0092] Step S2: Based on the product ID list, the inventory quantity of the product recorded in the cache, and the required deduction quantity of the product, deduct the inventory of the product in the cache one by one through the semaphore API;

[0093] Furthermore, step S2 specifically includes:

[0094] Step S21: Query the inventory quantity of the products in the product ID list recorded in the cache;

[0095] Step S22: If the inventory quantity is greater than 0, the product's inventory in the cache is reduced by deducting the quantity according to the product's requirements through the semaphore API; if the product's inventory in the cache is successfully reduced, the mapping of the successfully reduced product ID and the corresponding reduced quantity is recorded in the thread context; if the product's inventory in the cache is not successfully reduced, the product is marked as having inconsistent inventory data according to the adjustment probability threshold, and the mapping of the successfully reduced product ID and the corresponding reduced quantity recorded in the thread context is released, and the product inventory is returned as insufficient.

[0096] In this context, the thread context refers to a memory space that stores information for a thread to access throughout its execution cycle. This memory space is cleared after the thread terminates. A mapping is established in the thread context to record the successfully deducted product IDs and their corresponding deduction quantities. This mapping establishes a correspondence between the successfully deducted product IDs and the deduction quantities for each product ID.

[0097] The probability threshold is adjusted to determine whether to mark a product as having inconsistent inventory data when there is insufficient inventory in the cache. Since it is impossible to know whether the actual inventory is truly insufficient or due to inconsistencies between the cache and database data (e.g., the inventory data in the cache has been manually modified), this invention uses this probability to determine whether to mark the product as having inconsistent inventory data in order to improve fault tolerance (preventing the system from failing to deduct inventory due to insufficient actual inventory in the database but inaccurate cache data).

[0098] Specifically, the system determines whether to mark a product as having inconsistent inventory data based on the set adjustment probability threshold. The determination method is to generate a random number between 0 and 10, and determine whether the random number is less than the adjustment probability threshold. If it is less, the product is marked as having inconsistent inventory data; otherwise, it is not marked.

[0099] Step S23: If the inventory quantity is 0, mark the product as having inconsistent inventory data according to the adjustment probability threshold, and release the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context through the semaphore API, and return the information that the product inventory is insufficient.

[0100] Step S24: If the inventory quantity is less than 0, release the mapping of the successfully deducted product ID and the corresponding deducted quantity recorded in the thread context through the semaphore API, and return information about abnormal product inventory data.

[0101] Step S3: Based on the product ID list, the product's inventory in the database, and the required deduction quantity, deduct the product's inventory in the database one by one.

[0102] Furthermore, step S3 includes:

[0103] Step S31: If the inventory of the product in the database is successfully deducted, return a message indicating successful deduction;

[0104] Step S32: If the inventory of the product in the database is not successfully deducted, the product that failed to be deducted is marked as having inconsistent inventory data, and the semaphore in the database and the cache is rolled back through the transaction rollback hook.

[0105] Semaphores are data structures in computer systems that control access to a resource with a fixed quantity (such as inventory) by multiple threads. If an error occurs during transaction execution, i.e., the inventory of a product in the database is not successfully deducted, the executed statements must be rolled back to avoid impacting the database. The database will revert to its state before the transaction, i.e., the state before the inventory deduction. Therefore, even if the product's inventory in the database is not successfully deducted, it has no impact on the database itself.

[0106] Specifically, step S32 also includes:

[0107] Step S321: After the rollback is completed, release the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context through the semaphore API;

[0108] Step S322: Release all currently held read locks and write locks recorded in the thread context.

[0109] The inventory update method proposed in this embodiment includes an inventory query process and an inventory deduction process. In the above embodiment, steps S1-S3 are the inventory deduction process. The steps of the inventory query process are described in detail below with reference to step S1'.

[0110] On the other hand, in some embodiments, it also includes:

[0111] Step S1': Receive an inventory query request initiated by the business system, and obtain a list of product IDs and the expected inventory quantity of each product based on the inventory query request.

[0112] The purpose of obtaining the expected inventory quantity of a product based on the received inventory query request is to check whether the quantity of the product in the database has reached the expected inventory quantity. For example, if the expected inventory quantity of product B is 5, the business system requests to query whether there are 5 units of product B in the database.

[0113] Specifically, step S1' includes:

[0114] Step S11': Loop through and check whether the products in the product ID list already exist in the cache, and whether there is a flag indicating an inconsistency in inventory data;

[0115] Step S12': If the product ID does not exist in the cache, and / or there is a flag indicating an inconsistency in inventory data, then the product ID list is initialized;

[0116] Furthermore, step S12' specifically includes:

[0117] Step S121': Add a write lock to the product ID list; if the lock is successfully added, proceed to step S122'; otherwise, step S121' has failed and the inventory query process is terminated.

[0118] Step S122': Query the inventory information of the products in the product ID list from the database, and update the cache with the retrieved inventory information one by one;

[0119] The process involves updating the retrieved inventory information to the cache system one by one, recording it in the form of key (product ID) - value (inventory quantity). If the inventory information update is successful, step S123' is executed. If any failure occurs during the execution of step S122', the acquired write lock is released, and subsequent inventory query processes cannot proceed.

[0120] Step S123': If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data.

[0121] Step S124': Release the acquired write lock.

[0122] Step S13': Loop through and query the inventory information of the goods in the cache and record it. If the inventory of the goods in the cache is less than the expected inventory quantity, mark the goods as having an inventory quantity that does not meet the requirements.

[0123] Specifically, for each product, the process iterates through the cache, queries the cache for the product's key-value information, records the key-value information, and if the value (inventory quantity) in the cache is less than the expected inventory quantity (acquire), then the product is marked as not filling. This process continues until the product loop ends.

[0124] Step S14': Return the recorded inventory information and the flag indicating that the inventory quantity does not meet the requirements; that is, return the key-value information and not-fill information recorded in the business system.

[0125] This embodiment not only enables inventory updates and queries, but also supports secure multi-threaded high-concurrency updates during updates. Furthermore, when the inventory level meets the requirements, it supports multiple requests to simultaneously perform inventory update operations on the same or one or more products, thereby improving the system's throughput.

[0126] Please refer to the following. Figures 2-5 The application process of this method is illustrated below with reference to specific embodiments:

[0127] The application process is mainly divided into five processes: initialization process, inventory query process, inventory deduction process, inventory data inconsistency check process, and transaction rollback hook handling process. The following is a brief overview of the relationships between these five processes. From the perspective of the business system, the inventory query process and the inventory deduction process are the entry points, corresponding to two APIs, queryAvailable and trySubtractAvailable, respectively.

[0128] The upper-layer APIs provided by the business system need to be simple and easy to use, facilitating integration with upper-layer business systems. Therefore, this embodiment mainly provides the above two APIs: one is trySubtractAvailable, used to deduct inventory; the other is queryAvailable, used to query inventory levels.

[0129] (I) Initialization process: such as Figure 2 As shown,

[0130] 1. Query the cache system using the product ID list given by the parameters (generally, this process occurs when it is found that the product has not been initialized in the inventory deduction process, so the product ID list is usually the same as the product ID list in the inventory deduction process) to determine whether the product list already exists in the cache system.

[0131] 2. If all elements already exist, it means initialization is complete, and the initialization process ends. However, during the inventory deduction process, an internal initialization process will be executed as needed. If the initialization process ends, the next step will proceed based on the specific circumstances.

[0132] 3. Otherwise, check if each product already exists in the cache system (execute step 4 during the loop). Continue until the list of product IDs is completed. Then execute step 5. Otherwise, execute the following steps sequentially.

[0133] 4. If the product exists, it means the product has been initialized and no further processing is needed. If the product does not exist, record the product ID. Proceed to step 3.

[0134] 5. After the loop check is complete, generate a list of product IDs based on the product IDs recorded in step 4 that do not exist in the cache system. Acquire a read-write lock (or write lock) on the list of product IDs recorded in step 4 that do not exist in the cache system (the implementation of the read-write lock is not restricted, as long as it implements read-write lock semantics in a distributed environment, such as Redisson). If the locking is successful, proceed to step 6; otherwise, the initialization process ends in failure. During the inventory deduction process, the initialization process will be executed internally as needed. If the initialization process ends in failure, the corresponding inventory deduction process will also end.

[0135] 6. Query the inventory information of the product list recorded in step 4 from the database in batches. If this fails, i.e., if the inventory information corresponding to the product ID list recorded in step 4 cannot be found, release all read-write locks acquired in step 5.

[0136] 7. If the inventory information of the product list recorded in step 4 is found in the database, the retrieved inventory information is updated to the cache system one by one, recorded in the form of key (product ID)-value (inventory quantity). If any failure occurs, all read-write locks acquired in step 5 are released.

[0137] 8. Delete the inventory data inconsistency markers corresponding to the product list recorded in step 4.

[0138] 9. Release all read-write locks acquired in step 5. Initialization complete.

[0139] (II) Inventory Inquiry Process:

[0140] 1. The business system initiates an inventory query request for a group of products. The request parameters include a list of product IDs and the expected inventory quantity for each product (acquire).

[0141] 2. Check if the product list in the parameters has been initialized (i.e., step 1 in the initialization process). If not, execute the initialization process.

[0142] 3. Perform an inventory data inconsistency check process on the product list in the parameters. If there is a marker for inconsistent inventory data, perform an initialization process on the marked products.

[0143] 4. Loop through each product, query the cache for the product's key-value information, and record the key-value information. If the value (inventory quantity) in the cache is less than the expected inventory quantity (acquire), mark the product as not filling. Continue until the product loop ends.

[0144] 5. Return the key-value information and not-fill information recorded in step 4 of the business system.

[0145] (III) Inventory deduction process: such as Figure 3 As shown,

[0146] 1. The business system initiates an inventory deduction request for a group of products. The request parameters include a list of product IDs and the quantity of inventory to be deducted for each product.

[0147] 2. Sort the product ID list in the parameters. The sorting can be done in any order (e.g., ascending byte order), but the sorting method must be consistent each time. This step aims to prevent deadlocks when locking the product sequence. All operations on the product ID list in the following steps are based on the sorted result from this step.

[0148] 3. Check if the product list in the parameters has been initialized (i.e., step 1 in the initialization process). If not, execute the initialization process.

[0149] 4. Perform an inventory data inconsistency check process on the product list in the parameters. If there is a marker for inconsistent inventory data, perform an initialization process on the marked products.

[0150] 5. Register a database transaction rollback hook, which can realize the unified release of read-write locks and the rollback of cached inventory data.

[0151] 6. Acquire a read-write lock-read lock operation on the sorted list of product IDs (the implementation of the read lock is not restricted, as long as the semantics of the read lock are guaranteed in a distributed environment). If the lock acquisition is successful, record the held read-write lock-read lock in the thread context and execute subsequent operations; otherwise, the process ends and the business system is notified that the lock acquisition failed.

[0152] 7. Loop through all the parameter items and deduct the item inventory recorded in the cache one by one (steps 8-9). Continue until all items in the parameters have been deducted, then proceed to step 10.

[0153] 8. Query the inventory quantity of the product recorded in the cache. If the inventory quantity is 0, determine whether to mark the product as having inconsistent inventory data according to the set adjustment probability threshold (the determination method is to generate a random number between 0 and 10, and determine whether the random number is less than the adjustment probability threshold. If it is less, mark the product as having inconsistent inventory data; otherwise, do not mark it). Then, release the cached inventory that has been successfully deducted (the mapping between product ID and the quantity already deducted) recorded in the thread context through the semaphore API. The process ends, and the system is returned that the product's inventory quantity is insufficient. If the inventory quantity is greater than 0, proceed to step 9. If the inventory quantity is less than 0, the cached inventory that has been successfully deducted recorded in the context is also released. The process ends, and the system is returned that the product's inventory data is abnormal.

[0154] 9. Using the semaphore API (the implementation of the semaphore is not restricted, as long as the semantics of the semaphore are implemented in a distributed environment – ​​safe multi-threaded acquisition of semaphores), deduct the required amount (the quantity specified in the request parameters) from the cached inventory. If the deduction is successful, record the product ID and the deducted quantity mapping (Map) in the thread context and execute step 7; if it fails (insufficient cached inventory), follow the method in step 8 to determine whether to mark the product as having inconsistent inventory data, release the successfully deducted cached inventory recorded in the context, end the process, and return a message to the business system indicating insufficient inventory for the product.

[0155] 10. Deduct the inventory of each item in the parameterized product list from the database one by one (using SQL: "UPDATE table_name SET inventory_column = inventory_column - quantity to be deducted WHERE product_id_column = product_id AND (inventory_column - quantity to be deducted) >= 0"). If the deduction fails for any item due to insufficient inventory in the database, the corresponding item will be marked as having inconsistent inventory data, and the semaphores in the database and cache will be rolled back (using a transaction rollback hook). The process ends, and the system is returned that the product is out of stock. If all deductions are successful, the process ends, and the system is returned that the process is successful.

[0156] (iv) Inventory data inconsistency inspection process: such as Figure 4 As shown,

[0157] 1. Query the cache system using the product ID list given by the parameters (generally, this process occurs during the inventory deduction process when checking for inconsistencies in inventory data, so this product ID list is usually the same as the product ID list in the inventory deduction process) to determine whether there are any inventory data inconsistency flags in the product list.

[0158] 2. If the product does not exist, it means the product's inventory data is correct, and the process ends.

[0159] 3. Otherwise, repeatedly check each product for any discrepancies in inventory data (execute step 4 during the loop). Continue until the list of product IDs has been looped through. Then proceed to step 5. Otherwise, execute the following steps sequentially.

[0160] 4. If the product does not exist, it means the product inventory data is correct, skip this step; if the product exists, record the product; proceed to step 3.

[0161] 5. Return to the product list recorded in step 4 (indicating inconsistencies among these products). The process ends.

[0162] (V) Transaction rollback hook processing flow: such as Figure 5 As shown,

[0163] 1. This process is triggered after the database transaction is completed (including normal commit and rollback).

[0164] 2. Determine whether the database transaction has been rolled back.

[0165] 3. If a rollback has already been performed, release the cached inventory that has been successfully deducted, which is recorded in the context (the mapping between the product ID and the quantity already deducted), through the semaphore API.

[0166] 4. Release all currently held read locks recorded in the thread context.

[0167] 5. Release all currently held write locks recorded in the thread context.

[0168] In a specific implementation, the functionality of the present invention can be achieved through the following operations, taking Java language as an example:

[0169] 1. Redisson is introduced into the project to implement read-write locks and semaphore semantics APIs.

[0170] 2. MyBatis and Spring-TX are introduced into the project to implement the notification function of transaction rollback hooks.

[0171] 3. Implement the inventory deduction function according to the above inventory deduction process. If using a Redis cache database + Redisson client, then use RedissonSemaphore as the semaphore structure and RReadWriteLock as the read-write lock.

[0172] 4. Implement the inventory initialization function according to the above initialization process.

[0173] 5. Implement the inventory data inconsistency check function according to the above inventory data inconsistency check process.

[0174] 6. Implement the transaction commit and rollback processing functions according to the above transaction rollback hook process.

[0175] By using the method provided in this embodiment, the system's inventory update capability can be effectively improved, because both read-write locks and semaphores are data structures that support concurrent operations. Furthermore, testing revealed that for concurrent inventory updates of six products, with 80 threads and each thread performing 1000 updates, the update efficiency of the method described in this invention is seven times that of the mutex lock method.

[0176] Example 2:

[0177] In conjunction with the high-throughput inventory update method based on semaphores and read-write locks disclosed in Embodiment 1, this embodiment discloses a specific implementation example of a high-throughput inventory update system based on semaphores and read-write locks (hereinafter referred to as "the system").

[0178] Reference Figure 6 As shown, the system includes:

[0179] Deduction Request Receiving Unit 1: Receives inventory deduction requests initiated by the business system, and obtains a list of product IDs and the required deduction quantity for each product based on the inventory deduction request;

[0180] Specifically, the deduction request receiving unit 1 further includes:

[0181] First inspection module 11: Sort the product ID list, and cyclically check whether the products in the sorted product ID list already exist in the cache, and whether there is a marker indicating an inconsistency in inventory data;

[0182] First initialization module 12: If the product ID does not exist in the cache, and / or there is a marker indicating an inconsistency in inventory data, then the product ID list is initialized.

[0183] Furthermore, the first initialization module 12 includes:

[0184] First write lock acquisition module 121: Adds a write lock from the read-write lock to the product ID list;

[0185] First inventory information update module 122: Query the inventory information of the products in the product ID list from the database, and update the cache one by one with the queried inventory information;

[0186] First marker deletion module 123: If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data.

[0187] First write lock release module 124: Releases the acquired write lock.

[0188] The deduction request receiving unit 1 further includes: adding a read lock to the sorted list of product IDs, and recording the held read lock in the thread context.

[0189] Cache deduction unit 2: Based on the product ID list, the inventory quantity of the product recorded in the cache, and the required deduction quantity of the product, the inventory of the product in the cache is deducted one by one through the semaphore API;

[0190] Specifically, the cache deduction unit 2 includes:

[0191] Inventory Query Module 21: Query the inventory of the products in the product ID list recorded in the cache;

[0192] Inventory deduction module 22: If the inventory is greater than 0, the inventory of the product in the cache is deducted according to the product requirements through the semaphore API; if the inventory of the product in the cache is successfully deducted, the mapping of the successfully deducted product ID and the corresponding deduction quantity is recorded in the thread context; if the inventory of the product in the cache is not successfully deducted, the product is marked as having inconsistent inventory data according to the adjustment probability threshold, and the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context is released, and the information of insufficient product inventory is returned.

[0193] First mapping release module 23: If the inventory quantity is 0, mark the product as having inconsistent inventory data according to the adjustment probability threshold, and release the mapping of the successfully deducted product ID and the corresponding deducted quantity recorded in the thread context through the semaphore API, and return the information that the product inventory is insufficient;

[0194] Second mapping release module 24: If the inventory quantity is less than 0, release the mapping of the successfully deducted product ID and the corresponding deducted quantity recorded in the thread context through the semaphore API, and return information about abnormal product inventory data.

[0195] Database deduction unit 3: Deduct the inventory of each product in the database one by one according to the product ID list, the product's inventory in the database, and the required deduction quantity.

[0196] Specifically, the database deduction unit 3 includes:

[0197] Information return module 31: If the inventory of the product in the database is successfully deducted, a message indicating successful deduction will be returned.

[0198] Rollback module 32: If the inventory of a product in the database is not successfully deducted, the product that failed to be deducted is marked as having inconsistent inventory data, and the semaphores in the database and the cache are rolled back through the transaction rollback hook.

[0199] Furthermore, the rollback module 32 also includes:

[0200] Third mapping release module 321: After rollback is completed, release the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context through the semaphore API;

[0201] Read-write lock release module 322: Releases all currently held read and write locks recorded in the thread context.

[0202] On the other hand, the system also includes:

[0203] Query Request Receiving Unit 1': Receives the inventory query request initiated by the business system, and obtains a list of product IDs and the expected inventory quantity of each product based on the inventory query request.

[0204] Specifically, the query request receiving unit 1' includes:

[0205] Second inspection module 11': Iteratively checks whether the products in the product ID list already exist in the cache, and whether there is a marker indicating an inconsistency in inventory data;

[0206] Second initialization module 12': If the product ID does not exist in the cache, and / or there is a flag indicating an inconsistent inventory data state, then the product ID list is initialized;

[0207] Furthermore, the second initialization module 12' includes:

[0208] Second write lock acquisition module 121': Adds a write lock from the read-write lock to the product ID list;

[0209] Second inventory information update module 122': Query the inventory information of the products in the product ID list from the database, and update the cache one by one with the queried inventory information;

[0210] Second marker deletion module 123': If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data;

[0211] Second write lock release module 124': Releases the acquired write lock.

[0212] Inventory information query module 13': Iteratively queries and records the inventory information of the goods in the cache. If the inventory quantity of the goods in the cache is less than the expected inventory quantity, the goods are marked as having an inventory quantity that does not meet the requirements.

[0213] The query results are obtained in module 14'; it returns the inventory information of the records and a flag indicating that the inventory quantity does not meet the requirements.

[0214] The high-throughput inventory update system based on semaphores and read-write locks disclosed in this embodiment and the high-throughput inventory update method based on semaphores and read-write locks disclosed in Embodiment 1 are similar in other aspects. Please refer to Embodiment 1 for the technical solutions, which will not be repeated here.

[0215] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0216] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for achieving high-throughput inventory updates based on semaphores and read-write locks, characterized in that, include: Deduction request receiving steps: Receive inventory deduction requests initiated by the business system, and obtain a list of product IDs and the required deduction quantity for each product based on the inventory deduction request; Sort the product ID list, and then add a read lock to the sorted product ID list. Cache deduction steps: Based on the product ID list, the inventory quantity of the product recorded in the cache, and the required deduction quantity for the product, the inventory of each product in the cache is deducted one by one through the semaphore API; wherein, the semaphore API can control multiple threads to access the inventory together; Database deduction steps: Based on the product ID list, the product's inventory in the database, and the required deduction quantity, deduct the product's inventory in the database one by one; The cache deduction step includes: Inventory quantity query steps: Query the inventory quantity of the products in the product ID list recorded in the cache; Inventory deduction steps: If the inventory is greater than 0, the quantity of the product in the cache is deducted according to the product requirements through the semaphore API; if the inventory in the cache is successfully deducted, the mapping of the successfully deducted product ID and the corresponding deduction quantity is recorded in the thread context; if the inventory in the cache is not successfully deducted, the product is marked as having inconsistent inventory data according to the adjustment probability threshold, and the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context is released, and the information of insufficient product inventory is returned. The adjustment probability threshold is used to compare the product with a generated random number when the product's inventory in the cache is not successfully deducted. If the random number is less than the adjustment probability threshold, the product is marked as having inconsistent inventory data; otherwise, no marking is performed.

2. The inventory update method according to claim 1, characterized in that, The cache deduction step also includes: First mapping release step: If the inventory quantity is 0, mark the product as having inconsistent inventory data according to the adjustment probability threshold, and release the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context through the semaphore API, and return the information that the product inventory is insufficient. The second mapping release step: If the inventory quantity is less than 0, the mapping of the successfully deducted product ID and the corresponding deducted quantity recorded in the thread context is released through the semaphore API, and information about abnormal product inventory data is returned.

3. The inventory update method according to claim 1, characterized in that, The deduction request receiving step also includes: The first inspection step is to sort the product ID list and then repeatedly check whether the products in the sorted product ID list already exist in the cache and whether there is a flag indicating an inconsistency in inventory data. First initialization step: If the product ID does not exist in the cache, and / or there is a marker indicating an inconsistency in inventory data, then the product ID list is initialized.

4. The inventory update method according to claim 3, characterized in that, The first initialization step includes: First write lock acquisition step: Add a write lock from the read-write lock to the product ID list; First inventory information update step: Query the inventory information of the products in the product ID list from the database, and update the cache one by one with the retrieved inventory information; First marker deletion step: If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data. First write lock release step: Release the acquired write lock.

5. The inventory update method according to claim 4, characterized in that, The deduction request receiving step also includes: Add a read lock to the sorted list of product IDs and record the held read lock in the thread context.

6. The inventory update method according to claim 5, characterized in that, The database deduction steps include: Information return steps: If the product's inventory in the database is successfully deducted, a message indicating successful deduction will be returned. Rollback steps: If the inventory of the product in the database is not successfully deducted, the product that failed to be deducted is marked as having inconsistent inventory data, and the semaphore in the database and the cache is rolled back through the transaction rollback hook.

7. The inventory update method according to claim 6, characterized in that, The rollback step also includes: The third mapping release step: After the rollback is completed, the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context is released through the semaphore API; Read-write lock release steps: Release all currently held read and write locks recorded in the thread context.

8. The inventory update method according to claim 1, characterized in that, Also includes: Query request receiving steps: Receive the inventory query request initiated by the business system, and obtain the product ID list and the expected inventory quantity of each product based on the inventory query request.

9. The inventory update method according to claim 8, characterized in that, The query request receiving step includes: The second inspection step is to repeatedly check whether the products in the product ID list already exist in the cache, and whether there is a flag indicating an inconsistency in inventory data. Second initialization step: If the product ID does not exist in the cache, and / or there is a flag indicating an inconsistent inventory data state, then the product ID list is initialized; Inventory information query steps: Query the inventory information of the products in the cache in a loop and record it. If the inventory of the products in the cache is less than the expected inventory quantity, mark the products as having an inventory quantity that does not meet the requirements. The steps to obtain the query results include: returning the inventory information of the records and marking the status of inventory quantity not meeting the requirements.

10. The inventory update method according to claim 9, characterized in that, The second initialization step includes: The second write lock acquisition step: Add a write lock from the read-write lock to the product ID list; The second inventory information update step is to query the inventory information of the products in the product ID list from the database and update the cache one by one with the retrieved inventory information; Second marker deletion step: If there is a marker indicating inconsistent inventory data in the product ID list, then delete the marker indicating inconsistent inventory data. Second write lock release step: Release the acquired write lock.

11. A high-throughput inventory update system based on semaphores and read-write locks, used to implement the inventory update method according to any one of claims 1-10, characterized in that, include: Deduction Request Receiving Unit: Receives inventory deduction requests initiated by the business system, and obtains a list of product IDs and the required deduction quantity for each product based on the inventory deduction request; Sort the product ID list, and then add a read lock to the sorted product ID list. Cache deduction unit: Based on the product ID list, the inventory quantity of the product recorded in the cache, and the required deduction quantity of the product, the unit deducts the inventory of each product in the cache one by one through a semaphore API. The semaphore API can control multiple threads to access the inventory simultaneously. If the inventory deduction in the cache is successful, the mapping of the successfully deducted product ID and the corresponding deduction quantity is recorded in the thread context. If the inventory deduction in the cache is unsuccessful, the product that failed to deduct is marked as having inconsistent inventory data according to an adjusted probability threshold, and the mapping of the successfully deducted product ID and the corresponding deduction quantity recorded in the thread context is released. The adjustment probability threshold is used to compare the product with a generated random number when the product's inventory in the cache is not successfully deducted. If the random number is less than the adjustment probability threshold, the product is marked as having inconsistent inventory data; otherwise, no marking is performed. Database deduction unit: Based on the product ID list, the product's inventory in the database, and the required deduction quantity, deduct the product's inventory in the database one by one; if the product's inventory in the database is successfully deducted, a successful deduction message is returned; if the product's inventory in the database is not successfully deducted, the product with the failed deduction is marked as having inconsistent inventory data and the semaphores in the database and cache are rolled back.

Images