Method and apparatus for processing resource data, storage medium, and electronic device
By constructing a dependency graph to identify independent resource transfer requests in SSDs, the method allows for parallel processing, addressing the inefficiencies in sequential SSD operations and enhancing processing efficiency.
Patent Information
- Application Number
- CN202510314487.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-17
- Publication Date
- 2025-07-15
- Estimated Expiration
- 2045-03-17
AI Technical Summary
Since the resource data processing process of solid-state memory is sequential, the processing efficiency is low and the hardware resources of the multiprocessor core cannot be effectively utilized.
By constructing a dependency graph between resource transfer requests, resource transfer requests that can be processed in parallel are determined, and multithreads are called to process these requests in parallel to store resource data to the first storage area of solid state memory.
It improves the processing efficiency of resource data, optimizes the execution process of resource transfer requests, avoids dependency conflicts, and improves the overall processing performance of the system.
Smart Images

Figure CN119847451B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computers, and more particularly, to a method and apparatus for processing resource data, a storage medium, and an electronic device. Background Art
[0002] In the field of storage systems, a solid-state memory (SSD, Solid State Drive) is a storage device with semiconductor flash memory as the core storage medium, and has been widely used in recent years. It has many differences from traditional hard disk drives (HHD, Hard Disk Drive) in terms of performance, power consumption, and reliability.
[0003] During the actual read and write processes of an SSD, there are various operations, such as writing data to the SSD, reading data from the SSD, formatting the SSD, erasing the SSD, etc. For each operation, there are multiple steps, and the description of the implementation process of each step is sequential (serial), which cannot be implemented by the hardware of multi-processor cores, thereby resulting in low processing efficiency of resource data.
[0004] Therefore, aiming at the current situation where the implementation process of solid-state memories is sequential, resulting in low processing efficiency of resource data, has become an urgent problem to be solved. Summary of the Invention
[0005] The embodiments of the present application provide a method and apparatus for processing resource data, a storage medium, and an electronic device, so as to at least solve the problem that the implementation process of solid-state memories is sequential, resulting in low processing efficiency of resource data.
[0006] According to an embodiment of the present application, there is provided a method for processing resource data, which is applied to a solid-state memory, and a first storage area for storing resource data is provided in the solid-state memory, including: receiving M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored in the first storage area; obtaining a first connection graph between the M resource transfer requests, and determining N resource transfer requests from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship between the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph; calling N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel.
[0007] In an exemplary embodiment, the steps of constructing the first connection graph include: based on the M resource transfer requests, respectively extracting the corresponding transfer times and transfer identifiers from the M resource transfer requests; determining a target resource transfer request from the M resource transfer requests, and traversing the other resource transfer requests in the M resource transfer requests except the target resource transfer request during the current iteration process; for any other traversed resource transfer request, determining a dependent resource transfer request that satisfies the dependency condition from the other resource transfer requests according to the transfer time and transfer identifier of the traversed other resource transfer request, and the transfer time and transfer identifier corresponding to the target resource transfer request; wherein the resource transfer process of the dependent resource transfer request depends on the resource transfer process of the target resource transfer request; re-determining a new target resource transfer request from the M resource transfer requests, and returning to continue executing the other resource transfer requests in the M resource transfer requests except the target resource transfer request during the current iteration process, and stopping until a preset iteration stop condition is reached, to obtain the first connection graph.
[0008] In an exemplary embodiment, the transfer identifier includes a transferee identifier and a transferor identifier;
[0009] The determining a dependent resource transfer request that satisfies the dependency condition from the other resource transfer requests according to the transfer time and transfer identifier of the traversed other resource transfer request, and the transfer time and transfer identifier corresponding to the target resource transfer request includes:
[0010] In the case where the transfer time corresponding to the target resource transfer request is less than the transfer time of the traversed other resource transfer request, and the transferee identifier corresponding to the target resource transfer request is the same as the transferor identifier of the traversed other resource transfer request, determining the other resource transfer request as the dependent resource transfer request of the target resource transfer request.
[0011] In an exemplary embodiment, after calling N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel, the method includes: removing the N resource transfer requests in the first connection graph to obtain an updated first connection graph.
[0012] In an exemplary embodiment, the calling of N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel includes: calling N threads to determine N areas to be stored in the first storage area; the N areas to be stored are N areas where the resource data to be stored is to be stored into the first storage area; through the N threads, storing the resource data to be stored carried by the N resource transfer requests into the N areas to be stored in the first storage area in parallel.
[0013] In an exemplary embodiment, the method further includes: in the case that any one of the N areas to be stored is the same as the stored area set in the first storage area, performing an erasing process on the first storage area; the stored area is an area in the first storage area where resource data has been stored; storing the resource data to be stored carried by the N resource transfer requests into the N areas to be stored in the first storage area after the erasing process in parallel.
[0014] In an exemplary embodiment, a second storage area is further provided in the solid-state memory; the second storage area is used to store at least resource status parameters associated with the resource data stored in the first storage area; the performing of the erasing process on the first storage area includes: determining the data to be erased in the first storage area; updating the resource status parameters in the second storage area according to the data to be erased, and writing the updated resource status parameters into the second storage area; in the case of writing the updated resource status parameters into the second storage area, performing an erasing process on the first storage area based on the determined data to be erased.
[0015] In an exemplary embodiment, after the calling of N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area, it further includes: obtaining an updated first connection graph, where the first connection graph is updated after determining N resource transfer requests from the M resource transfer requests; based on the updated first connection graph, determining K resource transfer requests from the M resource transfer requests, where K is a positive integer; calling K threads to process the K resource transfer requests in parallel to store the resource data to be stored carried by the K resource transfer requests into the first storage area in parallel.
[0016] In an exemplary embodiment, the method further includes: under the request that the resource data to be stored carried in the M resource transfer requests are all stored in the first storage area, obtaining a second connection graph between the resource transfer parties of the M resource transfer requests pre-constructed; wherein, the second connection graph is obtained by connecting a plurality of key nodes, and the key nodes are used to represent the resource transfer parties corresponding to the M resource transfer requests respectively; determining O threads matching the number of the key nodes based on the number of the key nodes in the second connection graph; and updating the resource transfer record linked list associated with the M resource transfer requests in parallel based on the connection relationship between the O threads and the plurality of key nodes.
[0017] In an exemplary embodiment, a third storage area for storing a first pointer parameter and a second pointer parameter is further provided in the solid-state memory; the method further includes: obtaining a power-down instruction; wherein, the power-down instruction is an instruction generated when the solid-state memory enters a stop working state; determining the storage location of the resource data in the first storage area based on the power-down instruction, and updating the first pointer parameter and the second pointer parameter in the third storage area according to the storage location.
[0018] In an exemplary embodiment, the solid-state memory includes a first storage area, a second storage area, and a third storage area; before receiving a resource transfer request carrying resource data to be stored, the method further includes at least one of the following: clearing the first storage area, the second storage area, and the third storage area of the solid-state memory; writing an empty record in the first storage area; determining the number P of transfer parties of the resource transfer party, determining P threads, and initializing the resource state parameters in the second storage area according to the P threads, where P is a positive integer.
[0019] In an exemplary embodiment, the method further includes: responding to a resource query request; the resource query request includes at least one of a resource change query request and a resource storage query request; when the resource query request includes the resource change query request, determining a resource first change record matching the resource change query request from the resource state parameters stored in the second storage area based on the resource change query request; determining the address of the resource first change record; and determining the query result corresponding to the resource change query request according to the address; when the resource query request includes a resource storage query, directly determining the resource storage parameters from the memory corresponding to the solid-state memory.
[0020] According to another embodiment of the embodiments of the present application, there is also provided a processing device for resource data, including: a receiving module, configured to receive M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored in the first storage area; a determining module, configured to obtain a first connection graph between the M resource transfer requests, and determine N resource transfer requests from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship between the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph; a calling module, configured to call N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel.
[0021] According to still another embodiment of the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the steps in any one of the above method embodiments when running.
[0022] According to still another embodiment of the present application, there is also provided an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0023] According to still another embodiment of the present application, there is also provided a computer program product, where the computer program product includes a computer program, and the computer program implements the steps in any one of the above method embodiments when executed by a processor.
[0024] Through the present application, M resource transfer requests carrying resource data to be stored are received, where the M resource transfer requests are all used to store the resource data to be stored in the first storage area, and a first connection graph between the M resource transfer requests is obtained, and further N resource transfer requests are determined from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; thus, N threads can be called to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel. That is to say, when the present application receives M resource transfer requests, N resource transfer requests that can be processed in parallel are determined through the obtained first connection graph, and then N threads can be called to perform parallel processing on the N resource transfer requests, improving the processing efficiency of resource data. Description of the Drawings
[0025] The accompanying drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:
[0026] Figure 1 is a hardware structure block diagram of a server device for a resource data processing method according to an embodiment of the present application;
[0027] Figure 2 is a flowchart of a resource data processing method according to an embodiment of the present application;
[0028] Figure 3 is a schematic diagram of the association relationship between resource transfer parties in an embodiment of the present application;
[0029] Figure 4 is a schematic diagram of a first connection diagram according to an embodiment of the present application;
[0030] Figure 5 is a schematic diagram of a second connection diagram according to an embodiment of the present application;
[0031] Figure 6 is a structure block diagram of a resource data processing device according to an embodiment of the present application. Detailed Embodiments
[0032] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence.
[0034] The embodiment of the resource data processing method provided in the embodiment of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 is a hardware structure block diagram of a server device for a resource data processing method of an embodiment of the present application. As Figure 1 shown, the server device may include one or more ( Figure 1 only one is shown in Figure 1 processors 102 (the processors 102 may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above-mentioned server device may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that, Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned server device. For example, the server device may further include more thanFigure 1 more or fewer components as shown, or having a different configuration from that Figure 1 shown.
[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the processing method of resource data in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. Among them, only a first storage area for storing resource data is provided in the solid-state memory. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the server device through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.
[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the server device. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0037] To solve the above problems existing in the related art, in this embodiment, a method for processing resource data is provided, which is applied to a solid-state memory. Only a first storage area for storing resource data is provided in the solid-state memory, and its execution subject includes but is not limited to the SSD controller of the solid-state memory. The solid-state memory and the SSD controller can be deployed in Figure 1 the server device shown, such as Figure 2 shown, and the process includes the following steps S202 - S206:
[0038] Step S202: Receive M resource transfer requests carrying the resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored into the first storage area;
[0039] Among them, the resource data may include information of the resource transferor, information of the resource transferee, and the resource amount, etc. Among them, M may be an integer greater than 1. In an actual SSD working scenario, there may be M resource transfer requests. That is to say, there will be M pieces of resource data carried by the resource transfer requests that need to be stored in the first storage area simultaneously.
[0040] In some embodiments, in addition to storing information of the resource transferor, information of the resource transferee, and the resource amount, the first storage area also stores the next record pointer of the resource transfer linked list and the next record pointer of the resource transfer-in linked list. The resource transfer linked list is used to record the transfer situation of the user's resource data, and the resource transfer-in linked list is used to record the receiving situation of the user's resource data. When there is a resource transfer request, record the next record pointer of the resource transfer linked list and the next record pointer of the resource transfer-in linked list. Through the next record pointer of the resource transfer linked list and the next record pointer of the resource transfer-in linked list, when querying the user's resource transfer-in record and resource transfer record subsequently, the stored next record pointer can be directly read to sequentially determine the next record of the user.
[0041] In some embodiments, the area capacity of the first storage area can be preset. It can be understood that when the first storage area is full of resource data, or the resource data already stored in the first storage area has occupied a preset proportion of the area capacity, for example, has occupied 80% of the area capacity, then the first storage area can be erased. When specifically setting the area capacity of the first storage area, various influencing factors can be considered, such as the data length of the resource data to be stored each time. Since the information included in the resource data is the same, such as all including information of the resource transferor, information of the resource transferee, and the resource amount, the next record pointer of the resource transfer linked list, and the next record pointer of the resource transfer-in linked list, thus, the data length of the resource data to be stored each time can also be set to a fixed value, that is, the data length of the resource data stored each time is defaulted to be the same. In addition to the data length of the resource data to be stored, the total number of users with resource transfer service requirements and the maximum resource amount of each user also need to be considered. The embodiments of the present application do not limit this here.
[0042] In some embodiments, the first storage area is used to save all the resource data, and the resource data may specifically refer to transfer records. When the resource data is a transfer record, the resource data to be stored each time may include the following data: the transfer-out account number; the transfer-in account number; the transfer amount; the next record pointer of the resource transfer linked list; the next record pointer of the resource receiving linked list. The length of each piece of resource data is lr, then lr at least satisfies: lr ≥α + 2β +2γ.
[0043] Where α represents the number of bytes occupied by each transfer amount; β represents the number of bytes occupied by each account number (including the transferor account number and the transferee account number); γ represents the number of bytes occupied by the pointer to the next record. The capacity of the first storage area can be defined as: n2 := r * lr, where r is the number of transfer records determined according to the number of users and the account user quota, and ":=" is a symbol indicating the meaning of definition. Set lr | Cb (indicating that Cb can divide lr evenly, Cb is an erasure unit, and the size of Cb is determined by the hardware attributes of the SSD itself); , where rb represents the number of transfer records included in each erasure block. And set rb | r (indicating that r can divide rb evenly); , where rc represents the number of erasure blocks included in the entire first storage area.
[0044] In some embodiments, it can be set that: α = 8; β = 2; γ = 4; Cb = 20MB = 20 × 2 20 ; then lr = 20, m = 264, n = 216, r = 232, which exactly satisfies lr | Cb, so rb = 2 20 , which also exactly satisfies rb | r, so rc = 2 12 . The calculated capacity of the second storage area is: n1 = 80GB.
[0045] Step S204: Obtain a first connection graph among the M resource transfer requests, and determine N resource transfer requests from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship among the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph;
[0046] It should be noted that the first connection graph refers to a connection graph used to represent the dependency relationship among M resource transfer requests. There is a dependency relationship among resource transfer requests, which is specifically manifested as that if the resource transfer process included in a resource transfer request depends on the resource transfer process included in another resource transfer request, then there is a dependency edge between these two resource transfer requests.
[0047] When processing M resource transfer requests, it is first necessary to analyze the dependency relationships among these requests. For example, if request R1 needs to start execution after request R2 is completed (such as R1 needs to use the resources transferred by R2), then an edge is added from R2 to R1 in the first connection graph. In this way, by analyzing all M requests, a directed acyclic graph (DAG), that is, the first connection graph, can be constructed, which clearly represents the execution order and dependency relationships among the requests.
[0048] It can be understood that according to the first connection graph, N resource transfer requests can be determined, where N is a subset of M and M >= N. This is usually done when resources are limited and certain requests need to be processed preferentially. For example, if the current number of available parallel processing threads is N, then N requests that can be executed simultaneously without violating the dependency relationships can be selected from the M requests.
[0049] During the screening process, it can be achieved by traversing the first connection graph and looking for nodes without incoming edges (i.e., not depending on other resource transfer requests). These nodes without incoming edges can be regarded as the N resource transfer requests that can be directly executed. Specifically, breadth-first search or topological sorting algorithms can be used to extract N resource transfer requests without dependency relationships from the first connection graph for parallel processing.
[0050] Once the N resource transfer requests that can be processed in parallel are determined, threads can be allocated according to the hardware architecture (such as multi-processors or multi-processor cores). Each thread is responsible for processing one request, including but not limited to verifying the legality of the resource transfer, executing the resource transfer operation, updating the resource status and dependency relationships, etc. After processing a resource transfer request, the outgoing edges of this parallel resource transfer request (i.e., the subsequent requests depending on it) can be considered as the next possible requests for parallel processing. Such a processing strategy can not only maximize the utilization of hardware resources but also ensure the correct execution order of resource transfer requests, avoiding data inconsistency or errors caused by improper handling of dependency relationships. In the scenario of ASSD (Accounting Solid State Drive), this strategy helps to improve the write efficiency and query performance of the accounting dedicated solid-state memory, especially when facing a large number of concurrent transfer requests.
[0051] The construction of the first connection graph and the determination of N resource transfer requests are the key steps to realize the parallel processing of resource transfer requests. It ensures that requests can be effectively and correctly executed in parallel under a multi-processor or multi-processor core architecture. It can be understood that this way of parallel execution is not only limited to transfer record processing but can also be widely applied to any system that needs to process resource transfer requests and there are dependency relationships among the requests.
[0052] In an exemplary embodiment, the steps of constructing the first connection graph include: based on the M resource transfer requests, respectively extracting the corresponding transfer times and transfer identifiers from the M resource transfer requests; determining a target resource transfer request from the M resource transfer requests, and traversing the other resource transfer requests among the M resource transfer requests except the target resource transfer request in the current iteration process; for any traversed other resource transfer request, determining a dependent resource transfer request that satisfies the dependency condition from the other resource transfer requests according to the transfer time and transfer identifier of the traversed other resource transfer request, and the transfer time and transfer identifier corresponding to the target resource transfer request; wherein the resource transfer process of the dependent resource transfer request depends on the resource transfer process of the target resource transfer request; re-determining a new target resource transfer request from the M resource transfer requests, and returning to continue executing the other resource transfer requests among the M resource transfer requests except the target resource transfer request in the current iteration process until the preset iteration stop condition is reached and then stopping, to obtain the first connection graph.
[0053] Among them, the corresponding transfer times and transfer identifiers can be respectively extracted from the M resource transfer requests. In the ASSD scenario, the transfer time can specifically refer to the timestamp when the transfer occurs; the transfer identifier is used to identify the resource transfer party in the resource data, and the resource transfer party can specifically correspond to a user with a resource transfer service requirement. The resource transfer party can include a resource transfer-out party and a resource transfer-in party, and the resource transfer-in party and the resource transfer-out party are two relative concepts. In the processing scenario of resource data, for the same user, it can be both a resource transfer-out party and a resource transfer-in party. For example, for the resource transfer process between user A and user B, when user A has a resource expenditure and user B has a resource receipt, user A is the resource transfer-out party and user B can be the resource transfer-in party. Another example is that for the resource transfer process between user A and user C, when user C has a resource expenditure and user A has a resource receipt, user A is the resource transfer-in party and user B can be the resource transfer-out party.
[0054] Specifically, a target resource transfer request can be determined from the M resource transfer requests. This target resource transfer request can be any unprocessed request. In the parallel processing strategy, those requests that do not depend on other requests can be preferentially selected as the target, that is, the nodes without incoming edges in the dependency graph. In the current iteration process, traverse the other requests among the M resource transfer requests except the target resource transfer request. For each traversed request, determine whether it depends on the target resource transfer request. The determination of the dependency relationship is based on the transfer times and transfer identifiers between the requests.
[0055] For each other resource transfer request traversed, determine whether its transfer time and transfer identifier meet the conditions dependent on the target request. Specifically, if the transfer time of a request is after the request it depends on and the dependent identifier is correct, then this request is determined to be a dependent resource transfer request that meets the dependency condition. For example, in ASSD, if a transfer record R2 depends on R1 (i.e., the transfer of R1 is part of the condition for R2), then the transfer time of R2 needs to be after R1, and the receiving account of R2 is the sending account of R1.
[0056] Furthermore, determine a new target resource transfer request from the M resource transfer requests again, and return to the traversal process to continue execution until a preset iteration stop condition is reached. The iteration stop condition can be that all requests are correctly assigned to the dependency graph, or all requests without dependency relationships (i.e., the root nodes in the graph) have been processed.
[0057] The first connection graph is a directed acyclic graph (DAG), where each node represents a resource transfer request and the edges represent the dependency relationships between the requests. After constructing the first connection graph, a parallel processing strategy can be used to optimize the execution of resource transfer requests. Based on the dependency relationships of the nodes in the first connection graph, nodes without dependencies (i.e., nodes without incoming edges) can be selected for parallel processing. After processing a node, remove the node and its outgoing edges from the graph, and then check whether there are new nodes that become nodes without incoming edges and can be added to the parallel processing queue.
[0058] In the above embodiments, the aim is to optimize the parallel processing strategy by constructing a dependency graph of resource transfer requests, that is, the first connection graph, to ensure that resource transfer requests can be processed effectively and correctly under a multi-processor or multi-processor core architecture, while avoiding resource competition and dependency conflicts, and improving the overall processing efficiency and performance of the system. This method is used for parallel writing of transfer records in ASSD and is also applicable to other resource management and scheduling scenarios that need to consider the dependency relationships between requests.
[0059] In some embodiments, the first connection graph formed can specifically be a directed acyclic graph. Optionally, N can be a positive integer less than M. When determining the resource transfer requests that can be processed in parallel from the first connection graph, the determined N can be equal to M, that is, all M resource transfer requests can be processed in parallel, or it can be less than M, that is, some resource transfer requests can be processed in parallel. Generally, there is no dependency relationship between two resource transfer requests processed in parallel.
[0060] It can be understood that for different users, whether they are the resource transferor or the resource transferee, the corresponding transfer identifier is unique.
[0061] In some embodiments, transfer identifiers may be represented by numbers, letters, feature codes, etc. For example, numbers may be used to represent transfer identifiers.
[0062] In an exemplary embodiment, the transfer identifier includes a transferee identifier and a transferor identifier; determining a dependent resource transfer request that meets the dependency condition from the other resource transfer requests according to the transfer time and transfer identifier of the other resource transfer requests traversed to, and the transfer time and transfer identifier corresponding to the target resource transfer request includes: when the transfer time corresponding to the target resource transfer request is less than the transfer time of the other resource transfer requests traversed to, and the transferee identifier corresponding to the target resource transfer request is the same as the transferor identifier of the other resource transfer requests traversed to, determining the other resource transfer request as the dependent resource transfer request of the target resource transfer request.
[0063] It can be understood that the transfer identifier is used to identify the entity involved in the resource transfer request (an account in ASSD). Each resource transfer request (such as a transfer record) will include a transferee identifier and a transferor identifier, which is equivalent to in ASSD, each transfer record has a transfer-out account number and a transfer-in account number.
[0064] The process of determining the dependent resource transfer request is based on two conditions: the transfer time of the target resource transfer request should be less than the transfer time of the other resource transfer requests traversed to, and the transferee identifier of the target resource transfer request needs to be the same as the transferor identifier of the other resource transfer requests traversed to. In other words, a transfer record R2 depends on a transfer record R1 only when the transfer time of R1 is earlier than that of R2, and the transfer-in account of R1 is the transfer-out account of R2.
[0065] When traversing and processing resource transfer requests, each resource transfer request other than the target resource transfer request can be checked. For each request R2, its transfer time and transferee identifier can be compared with the corresponding information of the target resource transfer request R1. If the transfer time of R2 is later than that of R1, and the transferor identifier of R2 is the same as the transferee identifier of R1, then R2 is determined as the dependent resource transfer request of R1. This indicates that when processing the resource transfer request, the processing of R2 can be after that of R1 because the resource transfer of R2 (such as fund transfer) depends on the resource transfer result of R1.
[0066] In the above embodiments, by identifying the dependencies between all resource transfer requests and representing these relationships in the first connection graph. The first connection graph is a directed acyclic graph (DAG), where nodes represent resource transfer requests and edges represent the dependencies between requests. This enables the system to determine which requests can be processed in parallel and which requests must be processed serially based on the dependencies in the graph, thereby optimizing the execution process of resource transfer.
[0067] In an exemplary embodiment, determining N resource transfer requests from the M resource transfer requests according to the first connection graph includes: determining, based on the first connection graph, parallel resource transfer requests among the M resource transfer requests; wherein, the resource transfer process of the parallel resource transfer requests does not depend on the resource transfer processes of other resource transfer requests in the first connection graph; and determining the parallel resource transfer requests as the N resource transfer requests.
[0068] Parallel resource transfer requests are those whose resource transfer processes do not depend on the resource transfer processes of other resource transfer requests in the first connection graph. In other words, these requests do not need to wait for the completion of other requests during execution, and there is no direct dependency between them. In the ASSD scenario, this means that there is no direct association in the transfer order between transfer records, that is, the transfer-out account of one transfer record is not the transfer-in account of another transfer record, or their transfer times do not conflict.
[0069] Specifically, the process of determining parallel resource transfer requests usually includes analyzing the first connection graph (i.e., the dependency graph between resource transfer requests). In the graph, nodes without incoming edges (i.e., requests that no other requests depend on) can be identified as resource transfer requests that can be processed in parallel. This is because no incoming edges mean that the request does not need to wait for the completion of any other requests and can be started directly. For example, in the processing of transfer records in ASSD, if a dependency graph of transfer records is constructed, then nodes without incoming edges (i.e., transfer records) in the graph can be regarded as resource transfer requests that can be executed in parallel because they do not need to consider the status of other transfer records during execution.
[0070] The process of determining N resource transfer requests is actually to select the number of resource transfer requests that can be executed simultaneously according to the parallel processing ability (such as the number of processor cores). When determining N resource transfer requests, requests without dependencies can be selected from the M resource transfer requests to ensure that their resource transfer processes can be executed in parallel. Which requests are selected as resource transfer requests that can be processed in parallel is usually determined based on the number of nodes without incoming edges in the first connection graph and the number of parallel operations that can be processed.
[0071] For example, in the ASSD, if there are 4 processor cores available for parallel processing, and in the first connection graph constructed, there are also 4 transfer records found that have no incoming edges, that is, transfer records with no other transfer records depending on them. These 4 records can be selected as resource transfer requests that can be processed in parallel. If there are 5 transfer records with no other transfer records depending on them, since there are only 4 processor cores, only 4 such records can still be selected as resource transfer requests that can be processed in parallel.
[0072] Step S206: Call N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel.
[0073] It should be noted that when there are N resource transfer requests, N processes can be called to process the N resource transfer requests in parallel. When there are less than N idle threads currently, some of the resource transfer requests can be processed first, and when there are idle threads, the remaining resource transfer requests can be processed.
[0074] In an exemplary embodiment, the calling N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel, includes: calling N threads to determine N storage areas to be stored in the first storage area; the N storage areas to be stored are N areas where the resource data to be stored is to be stored into the first storage area; through the N threads, the resource data to be stored carried by the N resource transfer requests are stored into the N storage areas to be stored in the first storage area in parallel.
[0075] It can be understood that on the basis that the regional capacity of the first storage area has been determined, it is also possible to determine how much resource data can be stored in the first storage area. In the first storage area, according to the regional capacity and the data length of the resource data calculated in advance, the first storage area can be divided into multiple storage areas.
[0076] Among them, the storage area to be stored is the determined area where the resource data is to be stored into the first storage area, that is, the determined position where the data to be stored is to be written. When determining the storage area to be stored, it can be determined based on the first pointer parameter of the first storage area.
[0077] It can be understood that the first pointer parameter can be represented by numbers, letters, feature codes, etc. The first pointer parameter can correspond to the storage areas in the first storage area. For example, there are 12 storage areas in the first storage area. When representing the first pointer parameter by numbers, the pointer parameters are 0, 1, 2... 11 respectively, where 0 can correspond to the first storage area and 11 can correspond to the 12th storage area. When writing the resource data to be stored into the first storage area, it can be written in sequence. For example, first write the resource data to be stored into the first storage area, then write the resource data to be stored into the second storage area... and so on until the 12th storage area is written. At this time, it means that the first storage area has been filled up.
[0078] It should be noted that when N is equal to 3, it means that three storage areas need to be determined from the first storage area. Before storing the resource data to be stored into the first storage area, the first pointer parameter needs to be determined first, and the storage area to be stored is determined according to the first pointer parameter. If the determined first pointer parameters are 0, 1, and 2, it can indicate that the storage areas to be stored are the first storage area, the second storage area, and the third storage area.
[0079] In an exemplary embodiment, the method further includes: when any one of the N storage areas to be stored is the same as the set stored area in the first storage area, performing an erasure process on the first storage area; the stored area is the area in the first storage area where resource data has been stored; storing the resource data to be stored carried by the N resource transfer requests in parallel into the N storage areas to be stored in the first storage area after the erasure process.
[0080] Among them, the stored area is the area in the determined first storage area where resource data has been stored. When determining the stored area, it can be determined based on the second pointer parameter recorded for the first storage area. It can be understood that the second pointer parameter can be represented by numbers, letters, feature codes, etc. The second pointer parameter can correspond to the storage areas in the first storage area.
[0081] Optionally, the second pointer parameter can be corresponding to the storage area in the first storage area where resource data is first stored, that is, the second pointer parameter points to the oldest storage area. The oldest storage area can be understood as the position in the first storage area where resource data is first written.
[0082] In an exemplary embodiment, a second storage area is further provided in the solid-state memory, and the second storage area is used to store at least resource status parameters associated with the resource data stored in the first storage area; the erasing process for the first storage area includes: determining the data to be erased in the first storage area; updating the resource status parameters in the second storage area according to the data to be erased, and writing the updated resource status parameters into the second storage area; in the case of writing the updated resource status parameters into the second storage area, based on the determined data to be erased, performing an erasing process on the first storage area.
[0083] Among them, the resource status parameters may include the resource amount of the resource transferor, the first resource transfer record of the resource transferor, the transfer amount of the resource recipient, the first resource transfer record of the resource recipient, etc. Before the data to be erased is erased, the resource amounts of the resource transferor and the resource recipient related to the data to be erased, the first resource transfer record, etc. can be correspondingly updated.
[0084] In some embodiments, in addition to storing resource status parameters, the second storage area may further store the following system control parameters, and these system control parameters may be constants set based on the hardware performance of the SSD. For example, they may include the number of bytes occupied by the resource amount of each resource transferor, the number of bytes occupied by the account numbers of each resource transferor and resource recipient, the number of bytes occupied by the transfer record number, etc.
[0085] It can be understood that after determining the data to be erased, the resource status parameters in the second storage area are updated according to the data to be erased, so that even if the subsequent data to be erased is erased, its resource status parameters are updated, ensuring the accuracy of the subsequent resource query process.
[0086] In the above embodiments, after determining the data to be erased, the resource status parameters in the second storage area are further updated based on the data to be erased, avoiding errors such as resource amounts caused by data erasure processing, and effectively improving the processing accuracy of resource data. In an exemplary embodiment, after calling N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel, it further includes: obtaining an updated first connection graph, where the first connection graph is updated after determining N resource transfer requests from the M resource transfer requests; based on the updated first connection graph, determining K resource transfer requests from the M resource transfer requests, where K is a positive integer; calling K threads to process the K resource transfer requests in parallel to store the resource data to be stored carried by the K resource transfer requests into the first storage area in parallel.
[0087] It can be understood that after storing the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel, a new root node will continue to be selected from the updated first connection graph, and K resource transfer requests will be determined, where K is a positive integer less than M. Call K threads to process the K resource transfer requests in parallel and store the resource data to be stored carried by the K resource transfer requests into the first storage area in parallel.
[0088] It should be noted that after determining the K resource transfer requests, the root node will still be deleted, and then the first connection graph will be updated to obtain an updated first connection graph until all the key nodes in the first connection graph are determined as root nodes, and then the update of the first connection graph will be stopped. After each determination of the resource transfer requests to be processed in parallel, the first connection graph will be updated by deleting the root node.
[0089] In an exemplary embodiment, the resource status parameters at least include the record identifier and resource amount of the resource transfer of the resource transferor; updating the resource status parameters in the second storage area according to the data to be erased includes: when there is resource data corresponding to at least two resource transferors in the data to be erased, obtaining a second connection graph between the resource transferors; the second connection graph records the resource transfer relationship between the multiple resource transferors, and calling at least two threads; updating the record identifier and resource amount of the resource transfer of the at least two resource transferors in the second storage area in parallel according to the at least two threads and the resource transfer relationship.
[0090] That is to say, when the data to be erased includes the record identifiers of different resource transfer parties and the resource amounts, different threads and the recorded resource transfer relationships, that is, the resource is transferred from A to B, or the resource is transferred out from B to C, can be used to erase the record identifiers and resource amounts of different resource transfer parties in parallel, improving the erasing efficiency.
[0091] In an exemplary embodiment, determining the data to be erased in the first storage area includes: determining the length of the data to be erased based on the attributes of the solid-state memory; and determining the data to be erased with the length from the first storage area according to the storage time of the resource data stored in the first storage area.
[0092] Specifically, the attributes of the solid-state memory refer to the hardware performance of the solid-state memory, and the length of the resource data that can be deleted at one time is determined by the hardware performance of the solid-state memory.
[0093] It should be noted that when determining the data to be erased in the first storage area, the write time of the resource data written each time in the first storage area can be determined, and according to the write time, the resource data with the length of the data written earlier is selected and determined as the data to be erased.
[0094] In the above embodiment, the hardware performance of the solid-state memory determines the length of the data that can be erased each time the erasing process is performed, and further determines the length of the data to be erased each time, effectively improving the data erasing efficiency.
[0095] In the above steps S202 - S206, M resource transfer requests carrying the resource data to be stored are received, where the M resource transfer requests are all used to store the resource data to be stored in the first storage area, and a first connection graph between the M resource transfer requests is obtained. Further, N resource transfer requests are determined from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N. Thus, N threads can be called to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests in the first storage area in parallel. That is to say, when the present application receives M resource transfer requests, N resource transfer requests that can be processed in parallel are determined through the obtained first connection graph, and then N threads can be called to process the N resource transfer requests in parallel, improving the processing efficiency of the resource data.
[0096] In an exemplary embodiment, the method further includes: under the request that all the resource data to be stored carried in the M resource transfer requests are deposited into the first storage area, obtaining a second connection graph between the resource transfer parties of the M resource transfer requests pre-constructed; wherein, the second connection graph is obtained by connecting a plurality of key nodes, and the key nodes are used to represent the resource transfer parties corresponding to the M resource transfer requests respectively; determining O threads that match the number of key nodes based on the number of key nodes in the second connection graph; and updating the resource transfer record linked list associated with the M resource transfer requests in parallel based on the connection relationship between the O threads and the plurality of key nodes.
[0097] Among them, the second connection graph is constructed based on the key nodes of the resource transfer parties (transfer-in party and transfer-out party) in the M resource transfer requests. Here, the "key node" refers to the account in the resource transfer request, and each account is represented as a node in the second connection graph. The construction of the second connection graph is based on the connection relationship between the accounts, that is, which accounts participate in the resource transfer request as the transfer-in party and the transfer-out party. In the ASSD, this is equivalent to the relationship graph between the accounts involved in the transfer record.
[0098] It can be understood that based on the number of key nodes in the second connection graph, it can be determined how many threads (O threads) are allocated to process the resource transfer requests. The number of threads O usually matches the number of key nodes, which means that each key node (i.e., each account) will be responsible for processing all the resource transfer requests related to it by one thread. In the scenario of the ASSD, this is equivalent to allocating one thread for each account to be responsible for processing all the transfer records of this account.
[0099] Once the O threads are determined, these threads can be used to update the resource transfer record linked list associated with the M resource transfer requests in parallel. The resource transfer record linked list here refers to the transfer record linked list of each account, including the transfer-in record linked list and the transfer-out record linked list. Each thread is responsible for updating the record linked list related to its corresponding key node (account), which means that the update of the transfer record linked list of each account can be carried out in parallel in independent threads.
[0100] Specifically, each thread will traverse all the edges (i.e., resource transfer requests) connected to its corresponding key node in the second connection graph and update the resource transfer record linked list of this account. Since the second connection graph is constructed based on the connection between the resource transfer parties, each thread can independently process the resource transfer requests related to the account without conflicting with other threads.
[0101] For example, in an ASSD, if there are transfer records of multiple accounts that need to be updated, a thread can be assigned to each account. Each thread will be responsible for updating the linked lists of all incoming and outgoing records of its account, including the relevant records in the newly received M resource transfer requests. In this way, even if there are a large number of concurrent resource transfer requests, the system can efficiently update the transfer records of the accounts through parallel processing without being restricted by the bottleneck of serial processing.
[0102] In the above embodiment, by constructing the second connection graph and allocating threads based on the number of key nodes in the graph, the linked list of resource transfer records associated with the M resource transfer requests can be updated in parallel, thereby optimizing the processing efficiency of resource transfer requests in a multi-threaded environment. This method is not only applicable to the management of account transfer records in an ASSD, but also applicable to any scenario involving resource transfer parties (such as accounts, users, etc.), improving the processing speed of resource transfer requests and the system performance through parallelization technology.
[0103] In an exemplary embodiment, a third storage area for storing the first pointer parameter and the second pointer parameter is further provided in the solid-state memory; the method further includes: obtaining a power-off instruction; wherein, the power-off instruction is an instruction generated when the solid-state memory enters the stop working state; based on the power-off instruction, determining the storage location of the resource data in the first storage area, and updating the first pointer parameter and the second pointer parameter in the third storage area according to the storage location.
[0104] It can be understood that the solid-state memory entering the stop working state can be the normal sleep of the solid-state memory or the state entered when receiving a shutdown operation. Of course, the solid-state memory entering the stop working state can also be an abnormal power failure of the solid-state memory. Regardless of the reason for the solid-state memory to enter the stop working state, the storage location of the resource data in the first storage area will be determined, and the first pointer parameter and the second pointer parameter in the third storage area will be updated according to the storage location.
[0105] In the above embodiment, by updating the first pointer parameter and the second pointer parameter during power-off, when the solid-state memory works normally subsequently, the storage of resource data can be carried out based on the recorded first pointer parameter and the second pointer parameter, improving the storage efficiency.
[0106] In an exemplary embodiment, the solid-state memory includes a first storage area, a second storage area, and a third storage area; before receiving a resource transfer request carrying resource data to be stored, it further includes at least one of the following: clearing the first storage area, the second storage area, and the third storage area of the solid-state memory; determining the number of transfer parties P of the resource transfer party, determining P threads, and initializing the resource status parameters in the second storage area according to the P threads, where P is a positive integer.
[0107] In the above embodiment, before receiving a resource transfer request carrying resource data to be stored, the memory variables of the solid-state memory can be initialized. Through the initialization process, the normal operation of the subsequent solid-state memory can be ensured, and the processing accuracy of resource data can be improved.
[0108] In an exemplary embodiment, the method further includes: responding to a resource query request; the resource query request includes at least one of a resource change query request and a resource storage query request; when the resource query request includes the resource change query request, based on the resource change query request, determining a first resource change record that matches the resource change query request from the resource status parameters stored in the second storage area; determining the address of the first resource change record; according to the address, determining the query result corresponding to the resource change query request; when the resource query request includes a resource storage query, directly determining the resource storage parameters from the memory corresponding to the solid-state memory.
[0109] It should be noted that the resource change query request may include requests generated when resources are transferred out or transferred in. For the resource change query request, a first resource change record that matches the resource change query request can be determined first from the resource status parameters stored in the second storage area, that is, the first resource change record of the resource transferor in the resource change request. After determining the first resource change record, the address of the first resource change record can be determined based on the resource transfer list and resource transfer-in list of the resource transferor, and then the query result can be obtained.
[0110] In an exemplary embodiment, before determining the storage area to be stored in the first storage area in response to the resource transfer request, it further includes: storing the resource data to be stored in a cache; where the cache is a buffer set for the solid-state memory; when it is determined that the data volume of the resource data in the cache reaches a set quantity threshold, in response to the resource transfer request, determining the storage area to be stored in the first storage area.
[0111] It is understandable that there are different data granularities for flash memory (i.e., solid-state memory) write operations, and these granularities may not necessarily align with the minimum write granularity of the flash memory. To avoid the actual write granularity exceeding the requirements by a large margin, which may lead to excessive write cycles, in specific implementations, a caching mechanism is introduced to collect various write operations. When a sufficient number of write operations are accumulated in the cache, the write from the cache to the flash memory is then implemented.
[0112] In some embodiments, all data in the second storage area and the third storage area can be updated and read in the cache, and synchronized with the flash memory only when necessary. If a user needs to perform a large number of operations to query debit / credit records, some or all of the following data can be cached: some complete debit / credit records of frequently queried accounts; the latest partial debit / credit records of all accounts, to avoid multiple query operations on the flash memory.
[0113] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. To better understand the above method, the following will describe the above process in conjunction with embodiments, but it is not used to limit the technical solutions of the embodiments of the present invention. Specifically:
[0114] In the related art, there are various operations during the actual read and write processes of SSDs, such as writing data to the SSD, reading data from the SSD, formatting the SSD, erasing the SSD, etc. For each operation, there are multiple steps, and the descriptions of the implementation processes of each step are sequential (serial), which cannot be implemented on the hardware of multi-processor cores, thereby resulting in low processing efficiency of resource data.
[0115] For the above storage application, this application includes the following basic operation steps: 1. Format the ASSD, that is, initialize the accounting storage; 2. Add a transfer record to the ASSD; 3. Query all debit records of a certain account; 4. Query all credit records of a certain account; 5. Query the deposit balance of a certain account; 6. When the transfer record space is insufficient, a certain number of the oldest transfer records need to be erased, and the corresponding account information is updated at the same time; 7. Power on; 8. Power off.
[0116] First, consider the formatting (initialization) of the ASSD operation. This operation mainly includes the following steps: 1. Clear the storage spaces of the three storage areas; 2. Confirm the correctness of the system control parameters; 3. Reset the pointers; 4. Write an empty record at position 0 in the second storage area; 5. Initialize the account information in the second storage area; 6. Initialize the memory variables. Among them, step 1 is responsible for clearing the data in each data area. If the user does not care about data security, that is, performs formatting as fast as possible rather than low-level formatting, this step can be omitted. If the user enforces this step, whether this step can be executed in parallel depends on the execution method of the Nand erase command inside the SSD. If the Nand erase command can be executed in parallel, this step can also be executed in parallel. The execution method also completely depends on the call interface of the underlying Nand erase function, and the underlying command can be directly called. Step 2 is responsible for confirming and checking the value of the system configuration constant and cannot be parallelized. Step 3 resets 4 pointers, which can be executed by 4 threads respectively. However, since the computational amount is extremely small, it can also be directly executed serially because the significance of parallelization is not great. Step 4 is an independent operation and cannot be executed in parallel. Step 5 can be executed for each account separately. The pointer of the first transfer record of each account has no association with the original balance of the account, so it can be executed in parallel without communication. All the memory variables in step 6 are independent of each other for each account, so they can all be parallelized by dividing threads according to the account without communication.
[0117] Secondly, consider adding a transfer record to the ASSD. The main steps of this operation are: 1. Query the account balance of the transfer source account and determine whether the balance meets the requirements of the transfer record to be processed. If not, terminate the execution and return failure to the user. 2. Judge whether the storage space in the second storage area is full. If it is full, execute the following erasure process. 3. Write the transfer-in account number, transfer-out account number, and transfer amount to the record position pointed to by the pointer, and note that the pointer fields of the two linked list records are kept empty. 4. According to the direction of the transfer operation, write the address of this transfer record to the pointer field of the linked list record. 5. Update the address of this transfer record; 6. Update the account balance in the memory.
[0118] Among them, steps 1 and 2 are only a single judgment and do not need to be parallelized. Step 3 is used to write the transfer record information. Before writing each record, it is necessary to determine whether the record to be written is legal based on the balance of the current account. This leads to the need to limit the relationship between n records to be written when the system receives n records to be written in parallel in order to perform parallel processing. For this reason, a dependency relationship needs to be established between the records. As Figure 3 shown, where a rectangular box represents a record, and an arrow between the rectangular boxes represents a dependency relationship. Figure 3The arrows in it indicate that the record below depends on the record above, and the two cannot be written in parallel. Only after the record above is written can the record below start to be written. t1 and t2 represent the order of record submission, and the smaller the serial number, the earlier the submitted record. a1 and a2 represent the source accounts of the transfer, and b1 and b2 represent the destination accounts of the transfer. Figure 3 The dependency relationship shown exists if and only if t2 > t1 and b1 = a2. To illustrate more specifically, Figure 4 shows a specific example of a dependency relationship, that is, Figure 4 is a schematic diagram of the first connection graph. Among them, a rectangle represents a record, that is, each rectangle can correspond to a resource transfer request. For example, for rectangle A, 1:1 - 2, it can represent a resource transfer between account 1 and account 2 at time 1, where account 1 represents the source account of the transfer; account 2 represents the destination account of the transfer. The arrows between the rectangles represent the dependency relationship, that is, the resource transfer process of rectangle B depends on the resource transfer process of rectangle A. Only after the resource transfer request of rectangle A is written can the resource transfer request of rectangle B be written. The dependency graph is a directed acyclic graph, and it is possible to determine which records can be written in parallel simultaneously according to the directed acyclic graph and the number of current idle threads. Specifically, the method of breadth - first search can be used to continuously extract and delete the root nodes (nodes without incoming edges) from the graph. Step 4 needs to establish the link relationship of the linked lists of all incoming and outgoing records of each account. When the system receives multiple transfer records simultaneously, a directed graph GT can be established first according to the transfer relationship. Each node in the graph represents an account associated with any one transfer record, and each edge represents a transfer record. In other words, if account a transfers to account b, it is considered that there is an edge in GT pointing from node a to node b. For example, if the following transfer records as shown in Figure 4 are received simultaneously, it can be represented as Figure 5 , Figure 5 is a schematic diagram of the second connection graph, Figure 5 Each key node in it respectively represents the resource transfer party. Subsequently, based on the second connection graph, each account can be processed in parallel, with each thread responsible for one account (a node in the second connection graph). Each thread can independently traverse all incoming and outgoing edges of the node it is responsible for, and at the same time establish and write the link relationship. The update of the address in step 5 can also be obtained from GT and completed in parallel by independent threads responsible for each node. Step 6 needs to update and calculate the balance of each account, and the calculations between accounts are also independent of each other and can be executed by the thread responsible for that account, thus realizing parallel calculation.
[0119] Next, consider querying all incoming (outgoing) records of an account numbered k. The main execution steps are as follows: 1. Obtain from the data in the second storage area (the serial number of the first transfer record of each account); 2. Read the address of the first record of the incoming (outgoing) record linked list in this record from the first storage area; 3. Trace each incoming (outgoing) record along the incoming (outgoing) record linked list, and read the transfer-in account, transfer-out account, and transfer amount information from it and return them to the user. Since the incoming or outgoing transfer records of each account form a linked list, each record must be read one by one through the linked list to find the next record, so it cannot be executed in parallel.
[0120] The main process for querying the deposit balance of an account with a certain serial number k is as follows: 1. Directly return the account balance in the memory; since the deposit balance can be queried by directly reading the account balance from the memory, this operation does not need to be parallelized.
[0121] When the transfer record space is insufficient, an erasure process needs to be executed. Its execution steps can be described as follows: 1. Merge the transfer records in the area to be erased into the original total amount for each account, and then update the balance of each account; 2. Update the account information; 3. Update the memory variables; 4. Execute an erasure operation on the current area to be erased; 5. Set the new area to be erased; 6. Execute a system control information write to the second storage area and update the write position of the next system information in the first storage area. Among them, steps 1 - 3 can be executed separately for each account, and there is no need for communication between accounts, so each account can be handed over to an independent thread for execution, thereby achieving parallelization. Step 4 is a complete erasure command sent to Nand and cannot be split further, so it does not need to be parallelized. The workloads of steps 5 and 6 are not large and do not need to be parallelized either.
[0122] The following steps need to be executed each time the system is powered on: 1. Read the latest system control information from the first storage area, including each system constant and the account information O k , , ; 2. Read the latest record pointer information from the third storage area and assign it to variables pt, pr; 3. Update the variables in the memory according to the read O k , , Update the variables in the memory , , , , where Ok is the original amount of each account (before all transfer records occurred); k is the record serial number, is the address of the earliest existing incoming record. is the address of the latest existing incoming record, is the address of the earliest existing outgoing record, is the address of the latest billing record; 4. Scan all the incoming records of each account and calculate the total incoming amount ti; 5. Scan all the outgoing records of each account and calculate the total outgoing amount to; 6. Update the current amount of each account: dk := Ok + ti – to. Among them, in step 1, the information k of each account needs to be read. and , this part of each account can be performed independently. Therefore, step 2 only includes the reading and assignment of two pointers and does not require parallelism, or it can also be executed by two threads respectively. For and and and , the calculations of each account can be executed independently without interference. Therefore, the accounts can be simply bound to threads to achieve parallelization. In steps 4 and 5, since all incoming and outgoing records are saved as linked lists, parallelization is not possible; in step 6, the calculations of the balances d k of each account do not interfere with each other and can be bound to threads to achieve parallel execution.
[0123] Each time the system powers down, the following operations need to be performed: 1. According to the variables p t , p r, , that is, the position of the oldest transfer record saved in the second storage area and the write position of the next transfer record in the second storage area, perform a write of record pointer information to the third storage area and update Pp, that is, the write position of the next pointer information in the third storage area. Each time the system powers down, only one record needs to be written to area three, and parallelism is not required.
[0124] Through the resource data processing method of the present application, M resource transfer requests carrying the resource data to be stored are received. Among them, the M resource transfer requests are all used to store the resource data to be stored in the first storage area, and the transfer identifier in the resource data to be stored is extracted, and a first connection graph is established between the resource transfer parties of the multiple resource transfer requests according to the transfer identifier, and further N resource transfer requests are determined from the M resource transfer requests according to the first connection graph; thus, N threads can be called to process the N resource transfer requests in parallel to parallelly store the resource data to be stored carried by the N resource transfer requests into the first storage area respectively. That is to say, when the present application receives M resource transfer requests, N resource transfer requests that can be processed in parallel are determined through the determined first connection graph, and then N threads can be called to perform parallel processing on the N resource transfer requests, improving the processing efficiency of resource data.
[0125] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.
[0126] In this embodiment, a resource data processing device is further provided for implementing the above embodiments and preferred embodiments, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can implement a predetermined function. Although the modules described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.
[0127] Figure 6 is a structural block diagram of a resource data processing device according to an embodiment of the present application. The device includes:
[0128] A receiving module 602, configured to receive M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored in the first storage area;
[0129] A determining module 604, configured to obtain a first connection graph between the M resource transfer requests, and determine N resource transfer requests from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship between the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph;
[0130] A calling module 606, configured to call N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel.
[0131] The above-mentioned device receives M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored in the first storage area, and obtains a first connection graph among the M resource transfer requests. Further, N resource transfer requests are determined from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N. Thus, N threads can be called to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel. That is to say, when the present application receives M resource transfer requests, N resource transfer requests that can be processed in parallel are determined through the obtained first connection graph, and then N threads can be called to process the N resource transfer requests in parallel, improving the processing efficiency of the resource data.
[0132] In an exemplary embodiment, the determining module 604 is further configured to, based on the M resource transfer requests, respectively extract corresponding transfer times and transfer identifiers from the M resource transfer requests; determine a target resource transfer request from the M resource transfer requests, and traverse other resource transfer requests in the M resource transfer requests except the target resource transfer request in the current iteration process; for any other traversed resource transfer request, determine a dependent resource transfer request that satisfies the dependency condition from the other resource transfer requests according to the transfer time and transfer identifier of the traversed other resource transfer request, and the transfer time and transfer identifier corresponding to the target resource transfer request; where the resource transfer process of the dependent resource transfer request depends on the resource transfer process of the target resource transfer request; re-determine a new target resource transfer request from the M resource transfer requests, and return to continue executing by traversing other resource transfer requests in the M resource transfer requests except the target resource transfer request in the current iteration process until the preset iteration stop condition is reached and then stop, so as to obtain the first connection graph.
[0133] In an exemplary embodiment, the transfer identifier includes a transferee identifier and a transferor identifier; the determining module 604 is further configured to, when the transfer time corresponding to the target resource transfer request is less than the transfer time of the traversed other resource transfer request, and the transferee identifier corresponding to the target resource transfer request is the same as the transferor identifier of the traversed other resource transfer request, determine the other resource transfer request as the dependent resource transfer request of the target resource transfer request.
[0134] In an exemplary embodiment, the calling module 606 is further configured to call N threads to determine N to-be-stored areas in the first storage area; the N to-be-stored areas are N areas in the first storage area where the to-be-stored resource data is to be stored; and through the N threads, the resource data to be stored carried in the N resource transfer requests are respectively and concurrently stored into the N to-be-stored areas in the first storage area.
[0135] In an exemplary embodiment, the device further includes an erasing module, configured to, when any one of the N to-be-stored areas is the same as a set stored area in the first storage area, perform an erasing process on the first storage area; the stored area is an area in the first storage area where resource data has been stored; and through N threads, the resource data to be stored carried in the N resource transfer requests are respectively and concurrently stored into the N to-be-stored areas in the first storage area after the erasing process.
[0136] In an exemplary embodiment, the updating module is further configured to obtain an updated first connection graph, where the first connection graph is updated after determining N resource transfer requests from the M resource transfer requests; based on the updated first connection graph, determine K resource transfer requests from the M resource transfer requests, where K is a positive integer; and call K threads to concurrently process the K resource transfer requests to respectively and concurrently store the resource data to be stored carried in the K resource transfer requests into the first storage area.
[0137] In an exemplary embodiment, a second storage area is further provided in the solid-state memory, and the second storage area is used to store at least resource status parameters associated with the resource data stored in the first storage area; the erasing module is further configured to determine to-be-erased data in the first storage area; update the resource status parameters in the second storage area according to the to-be-erased data, and write the updated resource status parameters into the second storage area; and when writing the updated resource status parameters into the second storage area, perform an erasing process on the first storage area based on the determined to-be-erased data.
[0138] In an exemplary embodiment, when there is resource data corresponding to at least two resource transfer parties in the to-be-erased data, the updating module is further configured to call at least two threads; and update the record identifiers and resource amounts of the resource transfers of the at least two resource transfer parties in the second storage area in parallel according to the at least two threads.
[0139] In an exemplary embodiment, the update module is further configured to, upon a request that all the resource data to be stored carried in the M resource transfer requests are stored in the first storage area, obtain a second connection graph between the resource transfer parties of the M resource transfer requests pre-constructed; wherein, the second connection graph is obtained by connecting a plurality of key nodes, and the key nodes are used to represent the resource transfer parties corresponding to the M resource transfer requests respectively; determine O threads that match the number of key nodes based on the number of key nodes in the second connection graph; and update the resource transfer record linked list associated with the M resource transfer requests in parallel based on the connection relationship between the O threads and the plurality of key nodes.
[0140] In an exemplary embodiment, the solid-state memory includes a first storage area, a second storage area, and a third storage area; the device further includes an initialization module, and the initialization module is configured to clear the first storage area, the second storage area, and the third storage area of the solid-state memory; write an empty record in the first storage area; determine the number of transfer parties of the resource transfer parties, determine the number of transfer parties P of the resource transfer parties, determine P threads, and initialize the resource status parameters in the second storage area according to the P threads, where P is a positive integer.
[0141] In an exemplary embodiment, the device further includes a query module, and the query module is configured to respond to a resource query request; the resource query request includes at least one of a resource change query request and a resource storage query request; in a case where the resource query request includes the resource change query request, determine a first resource change record that matches the resource change query request from the resource status parameters stored in the second storage area based on the resource change query request; determine the address of the first resource change record; and determine the query result corresponding to the resource change query request according to the address; in a case where the resource query request includes a resource storage query, directly determine the resource storage parameters from the memory corresponding to the solid-state memory.
[0142] It should be noted that the above-mentioned respective modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited thereto: the above-mentioned modules are all located in the same processor; or, the above-mentioned respective modules are located in different processors in any combination form.
[0143] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, and the computer program is configured to execute the steps in any one of the above method embodiments when running.
[0144] Optionally, in this embodiment, the above computer program may be configured to perform the following steps by means of the computer program:
[0145] S1. Receive M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored into the first storage area;
[0146] S2. Obtain a first connection graph among the M resource transfer requests, and determine N resource transfer requests from the M resource transfer requests according to the first connection graph, where both M and N are positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship among the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph;
[0147] S3. Call N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel.
[0148] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as a USB flash drive, a read-only memory (ROM for short), a random access memory (RAM for short), a mobile hard disk, a magnetic disk, or an optical disc that can store a computer program.
[0149] The embodiment of the present application further provides an electronic device, which includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to execute the steps in any of the above method embodiments by means of the computer program.
[0150] Optionally, in this embodiment, the above processor may be configured to perform the following steps by means of the computer program:
[0151] S1. Receive M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored into the first storage area;
[0152] S2. Obtain a first connection graph among the M resource transfer requests, and determine N resource transfer requests from the M resource transfer requests according to the first connection graph, where both M and N are positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship among the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph;
[0153] S3. Invoke N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel.
[0154] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated here.
[0155] An embodiment of the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0156] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0157] An embodiment of the present application further provides a computer program. The computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in any one of the above method embodiments.
[0158] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the present application is not limited to any specific combination of hardware and software.
[0159] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for processing resource data, characterized in that, Applied to a solid-state memory, in which a first storage area for storing resource data is provided, the method includes: Receiving M resource transfer requests carrying the resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored into the first storage area; Obtaining a first connection graph between the M resource transfer requests, and determining N resource transfer requests from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to characterize the dependency relationship between the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph; Invoking N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel; The method further includes: Under the request that the resource data to be stored carried by the M resource transfer requests are all stored in the first storage area, obtaining a second connection graph between the resource transfer parties of the M resource transfer requests pre-constructed; where the second connection graph is obtained by connecting multiple key nodes, and the key nodes are used to characterize the resource transfer parties corresponding to the M resource transfer requests respectively; Determining O threads that match the number of key nodes based on the number of key nodes in the second connection graph; Based on the connection relationship between the O threads and the multiple key nodes, updating the resource transfer record linked list associated with the M resource transfer requests in parallel.
2. The method according to claim 1, wherein The construction step of the first connection graph includes: Based on the M resource transfer requests, respectively extracting the corresponding transfer time and transfer identifier from the M resource transfer requests; Determining a target resource transfer request from the M resource transfer requests, and traversing the other resource transfer requests in the M resource transfer requests except the target resource transfer request in the current iteration process; For any other resource transfer request traversed, according to the transfer time and transfer identifier of the traversed other resource transfer request, and the transfer time and transfer identifier corresponding to the target resource transfer request, determining a dependent resource transfer request that meets the dependency relationship condition from the other resource transfer requests; where the resource transfer process of the dependent resource transfer request depends on the resource transfer process of the target resource transfer request; Determining a new target resource transfer request from the M resource transfer requests again, and returning to traverse the other resource transfer requests in the M resource transfer requests except the target resource transfer request in the current iteration process to continue execution until reaching a preset iteration stop condition and stopping, to obtain the first connection graph.
3. The method according to claim 2, wherein The transfer identifier includes a transferee identifier and a transferor identifier; Determining a dependent resource transfer request that meets the dependency condition from the other resource transfer requests according to the transfer time and transfer identifier of the other resource transfer requests traversed to, and the transfer time and transfer identifier corresponding to the target resource transfer request, includes: When the transfer time corresponding to the target resource transfer request is less than the transfer time of the other resource transfer requests traversed to, and the transfer-in party identifier corresponding to the target resource transfer request is the same as the transfer-out party identifier of the other resource transfer requests traversed to, determining the other resource transfer request as the dependent resource transfer request of the target resource transfer request.
4. The method according to claim 1, wherein After invoking N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel, the method includes: Removing the N resource transfer requests from the first connection graph to obtain an updated first connection graph.
5. The method according to claim 1, wherein Invoking N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel, includes: Invoking N threads to determine N storage areas to be stored in the first storage area; the N storage areas to be stored are N areas where the resource data to be stored is to be stored into the first storage area; Through the N threads, storing the resource data to be stored carried by the N resource transfer requests into the N storage areas to be stored in the first storage area in parallel.
6. The method according to claim 5, wherein The method further includes: When any one of the N storage areas to be stored is the same as the stored area set in the first storage area, performing an erasure process on the first storage area; the stored area is the area in the first storage area where resource data has been stored; Through N threads, storing the resource data to be stored carried by the N resource transfer requests into the N storage areas to be stored in the first storage area after the erasure process in parallel.
7. The method according to claim 6, wherein A second storage area is further provided in the solid-state memory, and the second storage area is used to store at least resource status parameters associated with the resource data stored in the first storage area; Performing the erasure process on the first storage area, includes: Determining the data to be erased in the first storage area; Updating the resource status parameters in the second storage area according to the data to be erased, and writing the updated resource status parameters into the second storage area; When writing the updated resource status parameters into the second storage area, performing an erasure process on the first storage area based on the determined data to be erased.
8. The method according to claim 1, characterized in that, After invoking N threads to process the N resource transfer requests in parallel to store the resource data to be stored carried by the N resource transfer requests into the first storage area, it further includes: Obtaining an updated first connection graph; wherein, the first connection graph is updated after determining N resource transfer requests from the M resource transfer requests; Based on the updated first connection graph, determine K resource transfer requests from the M resource transfer requests, where K is a positive integer; Invoke K threads to process the K resource transfer requests in parallel, so as to parallelly store the resource data to be stored carried by the K resource transfer requests into the first storage area.
9. The method according to claim 1, wherein A third storage area for storing a first pointer parameter and a second pointer parameter is further provided in the solid-state memory; the method further includes: Obtain a power-down instruction; wherein, the power-down instruction is an instruction generated when the solid-state memory enters a stopped working state; Based on the power-down instruction, determine the storage location of the resource data in the first storage area, and update the first pointer parameter and the second pointer parameter in the third storage area according to the storage location.
10. The method according to claim 1, characterized in that, The solid-state memory includes a first storage area, a second storage area, and a third storage area; before receiving a resource transfer request carrying resource data to be stored, the method further includes at least one of the following: Perform a clearing process on the first storage area, the second storage area, and the third storage area of the solid-state memory; Write an empty record in the first storage area; Determine the number of transfer parties P of the resource transfer party, determine P threads, and initialize the resource status parameters in the second storage area according to the P threads, where P is a positive integer.
11. The method according to claim 10, wherein The method further includes: Respond to a resource query request; the resource query request includes at least one of a resource change query request and a resource storage query request; In the case where the resource query request includes the resource change query request, based on the resource change query request, determine a first resource change record that matches the resource change query request from the resource status parameters stored in the second storage area; Determine the address of the first resource change record; According to the address, determine the query result corresponding to the resource change query request; In the case where the resource query request includes a resource storage query, directly determine the resource storage parameters from the memory corresponding to the solid-state memory.
12. A processing device for resource data, which is applied to a solid-state memory, and only one first storage area for storing resource data is provided in the solid-state memory, and is characterized in that It includes: A receiving module, configured to receive M resource transfer requests carrying resource data to be stored, where the M resource transfer requests are all used to store the resource data to be stored into the first storage area; A determining module, configured to obtain a first connection graph between the M resource transfer requests, and determine N resource transfer requests from the M resource transfer requests according to the first connection graph, where M and N are both positive integers, and M is greater than or equal to N; the first connection graph refers to a connection graph used to represent the dependency relationship between the M resource transfer requests; the resource transfer processes of the N resource transfer requests do not depend on the resource transfer processes of other resource transfer requests in the first connection graph; A calling module, configured to call N threads to process the N resource transfer requests in parallel, so as to store the resource data to be stored carried by the N resource transfer requests into the first storage area in parallel; An updating module, configured to, upon the requests that the resource data to be stored carried by the M resource transfer requests are all stored in the first storage area, obtain a second connection graph between the resource transfer parties of the M resource transfer requests pre-constructed; wherein, the second connection graph is obtained by connecting a plurality of key nodes, and the key nodes are used to represent the resource transfer parties corresponding to the M resource transfer requests respectively; determine O threads matched with the number of the key nodes based on the number of the key nodes in the second connection graph; and update the resource transfer record linked list associated with the M resource transfer requests in parallel based on the connection relationship between the O threads and the plurality of key nodes.
13. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
14. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.
Citation Information
Patent Citations
Resource data processing method and device
CN112598510A
Resource data importing method and device
CN115794917A