Method and device for solving high concurrency scene of hot commodities and storage medium
By using a dynamic, multiplying, segmented inventory approach, the performance bottleneck of e-commerce inventory management systems under high-concurrency scenarios for popular products was resolved, achieving efficient inventory management and system stability, and improving the system's dynamic scalability and operational efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-03
AI Technical Summary
Existing e-commerce inventory management systems are prone to request queuing, backlog, and timeouts in high-concurrency scenarios involving popular products, leading to system paralysis. Furthermore, existing pre-segmentation methods suffer from high maintenance costs, limited scalability, high locking failure rates, and resource waste.
The system adopts a dynamic doubling segmented inventory approach. Through random sorting and dynamic segmentation logic, it dynamically adapts to the amount of concurrent requests, avoids pre-splitting, reduces locking conflicts by using random sorting, and ensures data consistency during the dynamic segmentation process.
It enables operation and maintenance without manual intervention in high-concurrency scenarios, has strong dynamic scalability, reduces locking conflict rate, avoids resource waste, and improves system performance and operation and maintenance efficiency.
Smart Images
Figure CN121788036A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of e-commerce inventory information management, and in particular to a method, apparatus and storage medium for solving high-concurrency scenarios of popular products. Background Technology
[0002] When an e-commerce system receives a customer order, it deducts inventory from the inventory management system. To ensure data consistency, the inventory management system uses a locking mechanism to isolate requests for orders of the same product during the inventory deduction process. During the locking period, order requests for the same product can only wait until the operation is completed and unlocked. When the request volume is small or the order products are scattered, this method can maintain the normal operation of the system. However, once faced with a large number of concurrent orders for popular products, the inventory management system is prone to request queuing, backlog, and timeouts, which may eventually paralyze the entire system. This is especially true when dealing with surges in orders for popular products during flash sales, new product launches, promotions, and other events, which can lead to system performance bottlenecks. This not only causes order processing delays and a poor customer experience but may also cause businesses to miss business opportunities and suffer huge economic losses.
[0003] Existing technologies improve concurrency by pre-segmenting inventory data. Inventory management systems pre-divide product inventory data into a fixed number of records, segments, or blocks, and generate and retrieve these segments through polling. This method can alleviate lock wait issues, but it has the following drawbacks: 1. High manual operation and maintenance costs: The inventory system has a large number of products, the hot spots are uncertain, and it is difficult to maintain the pre-splitting operation of products, resulting in high overall operation and maintenance costs.
[0004] 2. Limited expansion capability due to pre-segment quantity: The system's concurrent capacity for popular products is related to the number of segments, and the number of pre-segmented segments is fixed. When the number of accesses to popular product orders exceeds the capacity of this number of segments, lock waiting issues will still occur.
[0005] 3. Difficulty in adjusting after splitting: After the splitting is completed, further splitting to deal with sudden traffic or merging to save storage requires consideration of issues such as segment balancing, which is difficult to implement due to the high algorithm complexity.
[0006] 4. High failure rate of locking inventory: The algorithm for multiple concurrent order requests to poll and read segmented inventory data is the same, which can easily cause multiple requests to lock the same inventory data at the same time, resulting in a high probability of failure in locking inventory data.
[0007] 5. May lead to resource waste: There are objective errors in the prediction of product concurrency, which may result in too many pre-split products and too many segments, resulting in waste of storage resources. Summary of the Invention
[0008] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of the invention, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.
[0009] Therefore, the purpose of this invention is to provide a method, apparatus, and storage medium for solving high-concurrency scenarios of popular products. Based on a dynamic doubling segmented inventory approach, it solves the inventory locking and waiting problem in high-concurrency access of popular products, which not only improves the performance of the inventory management system, but also improves the system operation and maintenance efficiency.
[0010] To address the aforementioned technical problems, this invention provides a method for resolving high-concurrency scenarios involving popular products, employing the following technical solution: including: Step S1: Initialize the inventory storage structure: Establish the basic data structure for dynamic inventory segmentation; Step S2: Inventory Scan and Sort Step S2.1: Data Scan: Query the inventory dataset in the data storage medium for all products whose product ID is equal to the requested product ID and whose bound inventory quantity is 0, by means of database SQL search, memory scan or index scan. Step S2.2: Random sorting algorithm: Randomly sort the inventory dataset based on the Fisher-Yates Shuffle and Knuth Shuffle algorithms; Step S2.3: Return result: Select the first record after random sorting as the return result. If there is no available record, return "Insufficient inventory". Step S3: Lock in inventory: Step S3.1: Row-level lock contention: Attempt to lock the inventory query results; Step S3.2: Atomicity check: After the lock is successfully locked, verify again whether the remaining inventory is sufficient; Step S3.3: Locking failure handling: If locking fails, return to step 2 to rescan and randomly select other inventory data; Step S4: Segmentation determination algorithm: Segmentation stops when the remaining inventory quantity in a segment is lower than the preset threshold; otherwise, the dynamic inventory segmentation process is started. Step S5: Dynamic Inventory Segmentation Logic: Step S5.1: Order Binding Inventory Segment: Generate an order-associated bound inventory segment and complete physical storage. The bound segment data inherits the product ID of the original segment, the available inventory quantity is zero, the bound inventory quantity is the quantity requested by the order, and the external ID is the order ID requested accordingly. Step S5.2: Available Inventory Splitting: The available inventory after deducting the order quantity is dynamically divided into two new segments, and physical storage is completed; Step S5.3: Unlock and delete the locked inventory segment; Step S5.4: Data consistency guarantee: The atomicity of binding inventory segments and generating new available inventory segments is guaranteed through transactions. If an exception occurs during the splitting process, the entire splitting process will be rolled back. Step S6: The final result is returned.
[0011] Optionally, the inventory storage structure includes product ID, total available inventory, bound inventory quantity, binding reason, and external ID.
[0012] Optionally, in step S3.1, the inventory query results may be locked using a database SELECT, FORUPDATE, or Redis distributed lock.
[0013] Optionally, in step S4, the threshold calculation formula is: Threshold = (Order request transaction timeout duration / average order request transaction duration) * (1 – order request duration error rate).
[0014] Optionally, in step S5.2, the split ratio is adjusted by rounding down based on the number of odd and even numbers, and the two new segments inherit the product IDs of the original segments, only adjusting the available inventory quantity.
[0015] Optionally, step S6 specifically includes: if the inventory segmentation is successful, return the already bound inventory segment; if it fails, return to step 2 to rescan and randomly select other inventory data; if the retries fail 3 times, return an algorithm exception.
[0016] A device for solving high-concurrency scenarios involving popular products, comprising: The memory is configured to store instructions; The processor is configured to retrieve the instructions from the memory and to implement the method described above when executing the instructions.
[0017] A machine-readable storage medium having instructions stored thereon for causing a machine to perform the method described above.
[0018] In summary, the present invention has at least one of the following beneficial effects: 1. By dynamically segmenting the data, the concurrent request volume of each product can be dynamically adapted without pre-splitting, thus achieving zero manual operation and maintenance workload.
[0019] 2. The dynamic segmentation method completely breaks the capacity limit of the pre-segmentation scheme. When the number of concurrent order requests surges beyond the capacity of the number of segments, it can be dynamically expanded without upper limit. In actual project testing, a single computer and a single product can handle 2,000 concurrent requests per second using the pre-allocation method; after using the dynamic inventory segmentation method, the request volume increased to over 50,000 requests per second.
[0020] 3. The dynamic splitting algorithm is simple, adopting a dynamic binary search mechanism, and does not require the implementation of complex segment quantity balancing logic.
[0021] 4. When reading and locking inventory data, replace the static polling method with a random sorting algorithm to reduce lock contention between requests. In actual project testing, the lock contention rate of the polling algorithm was 45%, which was reduced to less than 2% after using the random sorting algorithm.
[0022] 5. The initial state of the dynamic segmentation method does not decompose the product inventory, thus avoiding unnecessary waste of storage resources. Attached Figure Description
[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0026] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," "top / bottom," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0027] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "sleeved / connected," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0028] Example 1 Reference Figure 1 This invention discloses a method for solving high-concurrency scenarios involving popular products, comprising: Step S1: Initialize the inventory storage structure: Objective: To establish a basic data structure for dynamic inventory segmentation, ensuring that inventory can be dynamically split and merged; The inventory storage structure includes product ID, total available inventory, bound inventory quantity, binding reason, and external ID.
[0029] Example structure:
[0030] Step S2: Inventory Scan and Sort Objective: To efficiently locate deductible inventory segments.
[0031] Implementation steps: Step S2.1: Data Scan: Query the inventory dataset in the data storage medium for all products whose product ID is equal to the requested product ID and whose bound inventory quantity is 0, through methods such as database SQL search, memory scan or index scan; Step S2.2: Random sorting algorithm: Randomly sort the inventory dataset based on shuffling algorithms such as Fisher-Yates Shuffle and Knuth Shuffle to reduce the probability of conflicts caused by multiple requests to lock inventory in subsequent steps due to hot spot concentration; Step S2.3: Return Result: Select the first record after random sorting as the return result. If no record is available, directly return "Insufficient Stock". Example of a random sorting algorithm (based on Python): block_ids = [1, 2, 3, ...] for i in range(len(block_ids)-1, 0, -1): j = random.randint(0, i) block_ids[i], block_ids[j] = block_ids[j], block_ids[i].
[0032] Step S3: Lock in inventory: Objective: To ensure data consistency in concurrent scenarios.
[0033] Implementation steps: Step S3.1: Attempt to lock the inventory query results (e.g., using a database SELECT ... FOR UPDATE or a Redis distributed lock); Step S3.2: Atomicity check: After the lock is successfully locked, verify again whether the remaining inventory is sufficient (to prevent data from being modified before the lock is locked); Step S3.3: Locking failure handling: If locking fails (e.g., it is already occupied by another thread), return to step 2 to rescan and randomly select other inventory data; Step S4: Segmentation determination algorithm: Segmentation stops when the remaining inventory quantity in a segment is lower than the preset threshold; otherwise, the dynamic inventory segmentation process is started.
[0034] Objective: To dynamically determine whether it is necessary to split the currently locked inventory data.
[0035] The threshold calculation formula is: Threshold = (Order request transaction timeout duration / average order request transaction duration) * (1 – order request duration error rate).
[0036] Actual vertical test example: Order request transaction timeout duration is 3000ms, average order request transaction duration is 100ms, and order request duration error rate is 20%. Threshold = (3000 / 100) * (1 - 0.2) = 24 That is, segmentation stops when the segmented inventory quantity is less than 24; otherwise, the dynamic inventory segmentation process is started.
[0037] Step S5: Dynamic Inventory Segmentation Logic: Objective: To split the original block into three segments (order-bound inventory segment + two new available inventory segments) while ensuring atomicity. Step S5.1: Order Binding Inventory Segment: Generate an order-associated bound inventory segment (e.g., Order ID=O001, bound inventory quantity=5) and complete physical storage. The bound segment data inherits the product ID of the original segment, the available inventory quantity is zero, the bound inventory quantity is the quantity requested by the order, and the external ID is the corresponding requested order ID; Step S5.2: Available Inventory Splitting: The available inventory after deducting order quantities is dynamically divided into two new segments, and physical storage is completed. The splitting ratio is adjusted by rounding down based on whether the remaining inventory is odd or even. For example: if the remaining inventory is odd (e.g., 95), it is split into 47 and 48 (to avoid decimals after equal division). If the remaining inventory is even (e.g., 96), it is split into 48 and 48. The two new segments inherit the product IDs from the original segments; only the available inventory quantity is adjusted.
[0038] Step S5.3: Unlock and delete the locked inventory segment; Step S5.4: Data Consistency Guarantee: Transactions ensure the atomicity of binding inventory segments and generating new available inventory segments. If an anomaly occurs during the splitting process, the entire splitting process is rolled back to maintain data consistency. Example:
[0039] Step S6: Final result return: If the inventory segmentation is successful, return the already bound inventory segment; if it fails, return to step 2 to rescan and randomly select other inventory data; if it fails to retrace 3 times, return an algorithm exception.
[0040] Example 2 Based on the same concept as Embodiment 1 above, an apparatus for solving high-concurrency scenarios involving popular products is also included. This apparatus for solving high-concurrency scenarios involving popular products includes: The memory is configured to store instructions; The processor is configured to retrieve instructions from memory and to execute instructions in accordance with the methods described above.
[0041] Example 3 Based on the same concept as Embodiment 1 above, a machine-readable storage medium is also included, on which instructions are stored to cause a machine to perform the method described above.
[0042] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.
Claims
1. A method for solving high-concurrency scenarios involving popular products, characterized in that: include: Step S1: Initialize the inventory storage structure: Establish the basic data structure for dynamic inventory segmentation; Step S2: Inventory Scan and Sort Step S2.1: Data Scan: Query the inventory dataset in the data storage medium for all products whose product ID is equal to the requested product ID and whose bound inventory quantity is 0, by means of database SQL search, memory scan or index scan. Step S2.2: Random sorting algorithm: Randomly sort the inventory dataset based on the Fisher-Yates Shuffle and Knuth Shuffle algorithms; Step S2.3: Return result: Select the first record after random sorting as the return result. If there is no available record, return "Insufficient inventory". Step S3: Lock in inventory: Step S3.1: Row-level lock contention: Attempt to lock the inventory query results; Step S3.2: Atomicity check: After the lock is successfully locked, verify again whether the remaining inventory is sufficient; Step S3.3: Locking failure handling: If locking fails, return to step 2 to rescan and randomly select other inventory data; Step S4: Segmentation determination algorithm: Segmentation stops when the remaining inventory quantity in a segment is lower than the preset threshold; otherwise, the dynamic inventory segmentation process is started. Step S5: Dynamic Inventory Segmentation Logic: Step S5.1: Order Binding Inventory Segment: Generate an order-associated bound inventory segment and complete physical storage. The bound segment data inherits the product ID of the original segment, the available inventory quantity is zero, the bound inventory quantity is the quantity requested by the order, and the external ID is the order ID requested accordingly. Step S5.2: Available Inventory Splitting: The available inventory after deducting the order quantity is dynamically divided into two new segments, and physical storage is completed; Step S5.3: Unlock and delete the locked inventory segment; Step S5.4: Data consistency guarantee: The atomicity of binding inventory segments and generating new available inventory segments is guaranteed through transactions. If an exception occurs during the splitting process, the entire splitting process will be rolled back. Step S6: The final result is returned.
2. The method for solving high-concurrency scenarios of popular products according to claim 1, characterized in that: The inventory storage structure includes product ID, total available inventory, bound inventory quantity, binding reason, and external ID.
3. The method for solving high-concurrency scenarios of popular products according to claim 1, characterized in that: In step S3.1, the inventory query results are tested by attempting to lock the database using SELECT, FOR UPDATE, or a Redis distributed lock.
4. The method for solving high-concurrency scenarios of popular products according to claim 1, characterized in that: In step S4, the threshold calculation formula is as follows: Threshold = (Order request transaction timeout duration / average order request transaction duration) * (1 – order request duration error rate).
5. The method for solving high-concurrency scenarios of popular products according to claim 1, characterized in that: In step S5.2, the split ratio is adjusted by rounding down the number of odd and even numbers. The two new segments inherit the product IDs of the original segments, and only the available inventory quantity is adjusted.
6. The method for solving high-concurrency scenarios of popular products according to claim 1, characterized in that: Step S6 specifically includes: if the inventory segmentation is successful, return the already bound inventory segment; if it fails, return to step 2 to rescan and randomly select other inventory data; if it fails to retries 3 times, return an algorithm exception.
7. A device for solving high-concurrency scenarios involving popular products, characterized in that: include: The memory is configured to store instructions; A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the method according to any one of claims 1 to 6.
8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform the method according to any one of claims 1 to 8.