Account data management method and system based on bastion host and distributed processing

Abstract

The present invention discloses an account data management method and system based on a bastion host and distributed processing, and relates to the technical field of data management. The present invention maximizes the read performance under the premise of improving the dynamic allocation write performance, adopts dynamic redirection when the load imbalance caused by the access mode of the host I\O request is large, and calculates the moving average to predict the intensity of the host I\O request in the next time window, providing a basis for assembling and releasing large pages in advance, reducing the overhead of actively performing release operations in advance, thereby triggering the release operation at an appropriate time point and an appropriate physical block, reducing the blockage of the release operation on the host I\O request, reducing the write amplification of the release operation, smoothing short-term fluctuations, better predicting the data change trend in the short term, and correcting inaccurate part of the prediction information by judging the internal load information of the chip, improving the performance and life of the flash memory module in the data end.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to an account data management method and system based on a bastion host and distributed processing. Background Art

[0002] Clients such as self-employed individuals or family offices use securities trading software to trade. There are many clients and securities institutions settled in the securities trading software. Clients can create trading accounts under the securities institutions of their choice. Due to the large number of clients, securities institutions and transaction data, the data will be classified and distributed, and in conjunction with the bastion host, in a specific network environment, in order to protect the network and data from intrusion and damage from external and internal users.

[0003] The account data of the account is stored in the storage device for centralized management, so that users can read and write data later. The data that users frequently access will be stored in the flash memory space. The storage level in the flash memory space is divided into flash memory chips, wafers, planes, physical blocks, and physical pages. The process of converting the logical address of the host I\O request into a physical address is called data allocation. The existing allocation strategy either sacrifices write parallelism to improve read parallelism, or sacrifices read parallelism to improve write parallelism. It cannot make good use of read parallelism and write parallelism at the same time, resulting in reduced response time when users access.

[0004] However, the flash memory space is composed of multiple flash memory chips, and the size of the flash memory space is limited. When there are insufficient free physical blocks in the flash memory space, the space occupied by invalid pages needs to be released. A physical block that contains both valid pages and invalid pages needs to write the valid pages to a new physical block (i.e., valid page migration), and then release the entire physical block. The migration of valid pages will increase write amplification and delay the response of the host I\O request. Moreover, when the number of free physical blocks is less than a pre-set threshold, the release operation will be triggered, and the release will be performed in advance by predicting the intervals between requests and idle cycles. Releasing too early will result in low data utilization in the flash memory space, and releasing too late will cause future requests to be blocked with transactions in the flash memory space, both of which will extend the waiting time of the host I\O request. Summary of the invention

[0005] The purpose of the present invention is to provide an account data management method and system based on a bastion host and distributed processing to solve the problems raised in the prior art.

[0006] To achieve the above object, the present invention provides the following technical solution: an account data management method based on a bastion host and distributed processing, the method comprising:

[0007] Step S100: The user registers on the device side to obtain a user account, and the user information entered by the user is stored in the data side. After logging into the user account, the user selects an organization to create a user account under the corresponding organization, and the corresponding account information entered by the user is stored in the data side;

[0008] Step S200: The user logs in to the user account on the device and sends a transaction request to the first server. After receiving the transaction request, the first server connects with the second server to perform the transaction. The first server sends the transaction result back to the user device.

[0009] Step S300: During the transaction, the host I\O request sent by the device end is written into the data end through the host interface module, and the data end includes a data classification module, a host interface module and a data flash memory module;

[0010] Step S400: During the writing process, the wafer-level load analysis module detects the occurrence of wafer load imbalance in real time, and predicts whether the flash memory transaction determined by the static address mapping allocation method will be allocated to the wafer that is being released, so as to decide whether to enable dynamic relocation;

[0011] Step S500: According to the trigger constraint information obtained by the hot and cold identification module, the time window module and the chip detection module, the trigger judgment module determines whether to start the release operation according to the trigger constraint information;

[0012] Step S600, the data classification module classifies the account information according to the multiple account identifiers corresponding to the account information, and generates an account list for the user to retrieve the corresponding account data according to the account list; the transaction security protection unit protects the transmission and storage of the account data to prevent the data from being tampered with.

[0013] The step S300 includes:

[0014] Step S301: In the data end, the host interface module receives a host I\O request sent by the device end to the data end, wherein the host I\O request is any one of a write instruction and a read instruction, and the host I\O request is divided into flash memory transactions of page size;

[0015] Step S302: The data flash memory module converts the logical address of the cached flash memory transaction into a physical address through static address mapping, determines the channel number, chip number and wafer number, and transmits the request with the physical address information to the wafer-level load analysis module.

[0016] The step S400 includes:

[0017] Step S401. The wafer-level load analysis module detects the occurrence of unbalanced wafer load in real time: calculate the wafer load value, which represents how long it takes for a flash transaction to be serviced that is to be sent to the wafer. The wafer load value represents the busy degree of a wafer within a certain period of time, and the wafer load value is T c From the sum of the number of write operations, the number of erase operations, and the write and erase flag bits, by comparing the wafer loads of different wafers, the load magnitudes on the wafers and the load differences between different wafers can be obtained, and the loads corresponding to different wafers can be obtained. The counter in the wafer is used to record the number of flash transactions of the wafer located in the cache chip, serving to record the wafer load magnitude in real time. The formula for calculating the wafer load value is:

[0018] T c =(Y r +Y e +Y re )×T b ;

[0019] Wherein, T c represents the wafer load value, Y r represents a write operation waiting to be executed in a corresponding wafer. The write operation can be any one of the following: user device write, mapped data write, and release and recycle write. Y e represents an erase operation waiting to be executed in a corresponding wafer; T b is the write delay time, which is a fixed value; Y re represents the flag bits of the erase operation and the write operation; the flag bits for the wafer to perform read, write, and erase are Br, Bw, and Be respectively, 0 means no, and 1 means yes, indicating the operation that the wafer is performing.

[0020] The step S400 further includes:

[0021] Step S402. The size of the counter can reflect the load conditions on each wafer. p is the difference between the maximum value and the minimum value of the counters set in several wafers, and q is the number of planes on a single wafer. If p≥q, it means that the wafer load distribution in the system is unbalanced, and dynamic relocation should be started to locate the request to the wafer with the minimum load; if p<q, it means that the load distribution of different wafers is balanced. By comparing the loads corresponding to different wafers, the load magnitudes on the wafers and the load differences between different wafers can be judged, so as to detect the occurrence of unbalanced wafer load in real time.

[0022] The step S400 further includes:

[0023] Step S403, predict whether the flash memory transaction determined by the allocation method of the static address mapping will be allocated to the wafer that is undergoing the release operation, so as to decide whether to enable dynamic relocation: if the count number of the erase counter on the wafer is greater than 0, it means that the release operation is being performed. If the release operation is allocated to the wafer that is undergoing the release operation due to the static address mapping, dynamic relocation is enabled, and the flash memory transaction is allocated to the wafer with the smallest load value. For wafers with the same load value, pages are sent to different wafers in a round-robin manner, so that the request is sent to the wafer with the smallest load in a round-robin manner; if the count number of the erase counter on the wafer is equal to 0, it means that the release operation is not being performed, and the write flash memory transaction does not conflict with the release operation, and step S402 is repeated to determine the load imbalance caused by the access mode of the host I\O request;

[0024] Step S404, for some flash transactions, the channel number, chip number and wafer number are re-determined, the plane number of the flash transaction is determined as the plane round on the target wafer, and the round is updated to the next bit after assignment. Finally, after the flash transaction is determined, it will enter the corresponding chip waiting queue and update the load information.

[0025] The step S500 includes:

[0026] Step S501, the hot and cold identification module migrates the page to be released to the cold block, puts the host I\O request into the hot block, realizes hot and cold separation, and determines the target erase block by updating the distance. When the data is accessed for the first time, the update distance is updated to the value of the write flash transaction counter, and the access mark position is 1, indicating the first access; when the distance is updated, the update distance is the difference between the current counter and the previous update distance, and the access mark position is 0, indicating that it is not new data, that is, it is not the first access; the initial position of the access mark bit is 0, and the physical page has no data;

[0027] Step S502, pre-set soft threshold and hard threshold, when the soft threshold of the release operation is reached, the time window module predicts the amount of data written by the host I\O request of the next time window by calculating the moving average; when the hard threshold of the release operation is reached, the release operation is executed in a traditional way; set up n time windows t1, t2, t3...tn, and count the amount of access data in each time window as S1, S2, S3...Sn, respectively, give a higher weight to the most recent time window, and increase the weight in turn according to the distance in time. The weights of each time window are G1, G2, G3...Gn from low to high, and the prediction value of the corresponding n+1 time window is calculated, and the calculation formula is:

[0028]

[0029] Step S503, set a burst access flag Yd. If a burst access is predicted to occur in the next time window, the burst access flag is set to true. Within the soft threshold range, the release operation will not be triggered. If it is predicted that no burst access will occur in the next time window, the burst access flag is set to false. The time window also obtains the estimated time cost Q1 of executing the predicted host I\O request, which provides decision information for triggering the release operation in advance. The calculation formula is:

[0030] Q1=Y J ×T r +Y K ×T b ;

[0031] Among them, T r Indicates the completion time of a read operation, Y J represents the number of write transactions generated, Y K Indicates the number of read transactions generated.

[0032] The step S500 further includes:

[0033] Step S504: According to the chip detection module, the host I\O request and the release job request can be evenly distributed to different flash memory resources. By triggering the constraint calculation formula, the host I\O request and the release job request can be effectively combined and completed within the time window, and the parallelism within the flash memory space is used for parallel processing, or the effective page is used to cache and then postpone the processing. On the contrary, in the next time window, even with the assistance of the effective page cache, the host I\O request and the release job cannot be combined and served to maximize the benefits. At this time, the release job cannot be performed in advance. The trigger constraint calculation formula is:

[0034] (Q1+Q2)÷m+Q3 <M+V×T b ;

[0035] Among them, Q2 represents the sum of the chip internal load and external load as the internal load of the flash memory space, m represents the number of chips in the flash memory space is set to m, Q3 represents the release overhead caused by erasing the target victim block, M represents the size of a time window, and V represents the remaining capacity of the effective page cache.

[0036] An account data management system based on a bastion host and distributed processing, the system comprising a connected device end, a first server, a second server and a data end, the data end comprising:

[0037] Wafer-level load analysis module, used to detect the occurrence of wafer load imbalance in real time, predict whether the flash transaction determined by the static address mapping allocation method will be allocated to the wafer in the process of releasing the operation, so as to decide whether to enable dynamic relocation;

[0038] The data flash module is used to convert the logical address of the cached flash transaction into a physical address through static address mapping, determine the channel number, chip number and wafer number, and carry the physical address information request to the wafer-level load analysis module;

[0039] The trigger judgment module is used to determine whether to start the release operation according to the trigger constraint information.

[0040] The data classification module is used to classify account information according to multiple account identifiers corresponding to the account information and generate an account list for users to retrieve corresponding account data based on the account list; the transaction security protection unit protects the transmission and storage of account data to prevent data tampering.

[0041] The data classification module comprises:

[0042] The hot and cold identification module is used to migrate the pages to be released to the cold block, put the host I\O request into the hot block, realize the hot and cold separation, and decide the target erase block by updating the distance;

[0043] The time window module is used to predict the amount of data written by the host I\O request in the next time window by calculating the moving average, and then determine whether to trigger the release job in advance;

[0044] The chip detection module is used to evenly distribute the host I\O requests and release job requests to different flash memory resources. By triggering the constraint calculation formula, the host I\O requests and release job requests can be effectively combined and completed within the time window, and then determine whether to delay the triggering of the release job.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The present invention determines the size of the load only by the number of write operations, and maximizes the read performance under the premise of improving the dynamic allocation of write performance. When the load imbalance caused by the access mode of the host I\O request is large, dynamic redirection is adopted to relocate the flash memory transaction sent to the wafer in the process of releasing operation to the wafer with a smaller load value to reduce this debt imbalance phenomenon, thereby improving the utilization and parallelism inside and outside the chip, and optimizing the user experience when making host I\O requests.

[0047] 2. The time window module in the present invention predicts the intensity of the host I\O request in the next time window by calculating the moving average, that is, the read and write access intensity in the future time window, which provides a basis for assembling and releasing large pages in advance, reduces the overhead of proactively performing release operations in advance, and triggers the release operation at the appropriate time point and the appropriate physical block, that is, appropriately advances or postpones the recovery of some pages to improve the efficiency of data management, minimizes the access conflict between the host I\O request and the release operation, reduces the blockage of the release operation on the host I\O request, reduces the write amplification of the release operation, and gives a higher weight to recent data and a lower weight to data farther away in the window. In order to capture recent changes more quickly, short-term fluctuations can be smoothed, and the trend of data changes in the short term can be better predicted. By judging the internal load information of the chip, inaccurate part of the prediction information is corrected, and the performance and life of the flash memory module in the data end are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the present invention;

[0049] Figure 2 It is a module connection diagram of the present invention;

[0050] Figure 3 It is a module composition diagram of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example: Figure 1-Figure 2 As shown, the present invention provides an account data management method based on a bastion host and distributed processing, the method comprising:

[0053] Step S100: The user registers on the device side to obtain a user account, and the user information entered by the user is stored in the data side. After logging into the user account, the user selects an organization to create a user account under the corresponding organization, and the corresponding account information entered by the user is stored in the data side;

[0054] Step S200: The user logs in to the user account on the device and sends a transaction request to the first server. After receiving the transaction request, the first server connects with the second server to perform the transaction. The first server sends the transaction result back to the user device.

[0055] Step S300: During the transaction, the host I / O request sent by the device end is written into the data end through the host interface module, and the data end includes a data classification module, a host interface module and a data flash memory module; Step S300 includes:

[0056] Step S301: In the data end, the host interface module receives a host I\O request sent by the device end to the data end, wherein the host I\O request is any one of a write instruction and a read instruction, and the host I\O request is divided into flash memory transactions of page size;

[0057] Step S302: The data flash memory module converts the logical address of the cached flash memory transaction into a physical address through static address mapping, determines the channel number, chip number and wafer number, and transmits the request with the physical address information to the wafer-level load analysis module.

[0058] Step S400: During the writing process, the wafer-level load analysis module detects the occurrence of wafer load imbalance in real time, and predicts whether the flash memory transaction determined by the allocation method of the static address mapping will be allocated to the wafer in the process of releasing the operation, so as to decide whether to enable dynamic relocation; the step S400 includes:

[0059] Step S401, the wafer-level load analysis module detects the occurrence of wafer load imbalance in real time: calculates the wafer load value, which indicates how long it takes for a flash memory transaction to be sent to the wafer to be serviced, and indicates the busyness of a wafer in the current period of time. The wafer load value T c By comparing the wafer loads of different wafers, the load size on the wafer and the load difference on different wafers can be obtained from the number of write operations and the number of erase operations as well as the sum of the write and erase flag bits, and the load corresponding to different wafers can be obtained. The counter in the wafer is used to record the number of flash memory transactions of the wafer located in the cache chip, serving to record the wafer load size in real time. The wafer load value calculation formula is:

[0060] T c =(Y r +Y e +Y re )×T b ;

[0061] Among them, T c Indicates the wafer load value, Y r Indicates the write operation waiting to be executed in the corresponding wafer. The write operation can be any of the following: user device write, mapping data write, and release recovery write. e Indicates the erase operation waiting to be executed in the corresponding wafer; T b is the write delay time, which is a fixed value; Y reFlag bits indicating erase and write operations; the flag bits for the die to perform read, write, and erase operations are Br, Bw, and Be respectively, where 0 means no and 1 means yes, indicating the operation the die is performing.

[0062] Step S402: The size of the counter can reflect the load situation on each die. p is the difference between the maximum and minimum values of the counters set for several dies, and q is the number of planes on a single die. If p ≥ q, it means the load distribution on the dies in the system is unbalanced, and dynamic relocation should be started to locate the request to the die with the minimum load. If p < q, it means the load distribution on different dies is balanced. By comparing the loads corresponding to different dies, the load magnitude on the die and the load difference between different dies can be judged, so as to detect the occurrence of unbalanced die load in real time.

[0063] Step S403: Predict whether the flash transaction determined by the allocation method of the static address mapping will be allocated to the die that is performing the release operation, so as to decide whether to start dynamic relocation: if the count of the erase counter on the die is greater than 0, it means the release operation is in progress. If the release operation is allocated to the die that is performing the release operation due to the static address mapping, dynamic relocation is started to allocate this flash transaction to the die with the minimum load value. For dies with the same load value, a polling method is selected to send pages to different dies, so that the requests are sent to the die with the minimum load in a round-robin manner. If the count of the erase counter on the die is equal to 0, it means no release operation is in progress, and the write flash transaction does not conflict with the release operation. Then repeat step S402 to judge the load imbalance situation caused by the access mode of the host I / O request.

[0064] Step S404: For some flash transactions, the channel number, chip number, and die number are re-determined. The plane number of the flash transaction is determined as the plane round on the target die. After assignment, the round is updated to the next one. Finally, after determining the flash transaction, it will enter the corresponding chip waiting queue and update the load information.

[0065] Step S500: According to the trigger constraint information obtained by the hot-cold recognition module, time window module, and chip detection module, the trigger judgment module determines whether to start the release operation according to the trigger constraint information.

[0066] Step S600: The data classification module classifies the account information according to multiple account identifiers corresponding to the account information to generate an account list for the user to retrieve the corresponding account data according to the account list. The transaction security protection unit protects the transmission and storage of the account data to prevent data from being tampered with. The said step S500 includes:

[0067] Step S501, the hot and cold identification module migrates the page to be released to the cold block, puts the host I\O request into the hot block, realizes hot and cold separation, and determines the target erase block by updating the distance. When the data is accessed for the first time, the update distance is updated to the value of the write flash transaction counter, and the access mark position is 1, indicating the first access; when the distance is updated, the update distance is the difference between the current counter and the previous update distance, and the access mark position is 0, indicating that it is not new data, that is, it is not the first access; the initial position of the access mark bit is 0, and the physical page has no data;

[0068] Step S502, pre-set soft threshold and hard threshold, when the soft threshold of the release operation is reached, the time window module predicts the amount of data written by the host I\O request of the next time window by calculating the moving average; when the hard threshold of the release operation is reached, the release operation is executed in a traditional way; set up n time windows t1, t2, t3...t6, and count the amount of access data in each time window as S1, S2, S3...S6 respectively, give a higher weight to the most recent time window, and increase the weight in turn according to the distance in time. The weights of each time window are G1, G2, G3...G6 from low to high, and the prediction value of the corresponding n+1 time window is calculated, and the calculation formula is:

[0069]

[0070] Step S503, set a burst access flag Yd. If a burst access is predicted to occur in the next time window, the burst access flag is set to true. Within the soft threshold range, the release operation will not be triggered. If it is predicted that no burst access will occur in the next time window, the burst access flag is set to false. The time window also obtains the estimated time cost Q1 of executing the predicted host I\O request, which provides decision information for triggering the release operation in advance. The calculation formula is:

[0071] Q1=Y J ×T r +Y K ×T b ;

[0072] Among them, T r Indicates the completion time of a read operation, Y J represents the number of write transactions generated, Y K Indicates the number of read transactions generated.

[0073] The step S500 further includes:

[0074] Step S504: According to the chip detection module, the host I\O request and the release job request can be evenly distributed to different flash memory resources. By triggering the constraint calculation formula, the host I\O request and the release job request can be effectively combined and completed within the time window, and the parallelism within the flash memory space is used for parallel processing, or the effective page is used to cache and then postpone the processing. On the contrary, in the next time window, even with the assistance of the effective page cache, the host I\O request and the release job cannot be combined and served to maximize the benefits. At this time, the release job cannot be performed in advance. The trigger constraint calculation formula is:

[0075] (Q1+Q2)÷m+Q3 <M+V×T b ;

[0076] Among them, Q2 represents the sum of the chip internal load and external load as the internal load of the flash memory space, m represents the number of chips in the flash memory space is set to m, Q3 represents the release overhead caused by erasing the target victim block, M represents the size of a time window, and V represents the remaining capacity of the effective page cache.

[0077] Figure 3 , an account data management system based on a bastion host and distributed processing, the system comprising a connected device end, a first server, a second server and a data end, the data end comprising:

[0078] Wafer-level load analysis module, used to detect the occurrence of wafer load imbalance in real time, predict whether the flash transaction determined by the static address mapping allocation method will be allocated to the wafer in the process of releasing the operation, so as to decide whether to enable dynamic relocation;

[0079] The data flash module is used to convert the logical address of the cached flash transaction into a physical address through static address mapping, determine the channel number, chip number and wafer number, and carry the physical address information request to the wafer-level load analysis module;

[0080] The trigger judgment module is used to determine whether to start the release operation according to the trigger constraint information.

[0081] The data classification module is used to classify account information according to multiple account identifiers corresponding to the account information and generate an account list for users to retrieve corresponding account data based on the account list; the transaction security protection unit protects the transmission and storage of account data to prevent data tampering.

[0082] The data classification module comprises:

[0083] The hot and cold identification module is used to migrate the pages to be released to the cold block, put the host I\O request into the hot block, realize the hot and cold separation, and decide the target erase block by updating the distance;

[0084] The time window module is used to predict the amount of data written by the host I\O request in the next time window by calculating the moving average, and then determine whether to trigger the release job in advance;

[0085] The chip detection module is used to evenly distribute the host I\O requests and release job requests to different flash memory resources. By triggering the constraint calculation formula, the host I\O requests and release job requests can be effectively combined and completed within the time window, and then determine whether to delay the triggering of the release job.

[0086] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. An account data management method based on a bastion host and distributed processing, characterized in that: The method includes: Step S100: The user registers on the device side to obtain a user account, and the user information entered by the user is stored in the data side. After logging into the user account, the user selects an organization to create a user account under the corresponding organization, and the corresponding account information entered by the user is stored in the data side; Step S200: The user logs in to the user account on the device and sends a transaction request to the first server. After receiving the transaction request, the first server connects with the second server to perform the transaction. The first server sends the transaction result back to the user device. Step S300: During the transaction, the host I\O request sent by the device end is written into the data end through the host interface module, and the data end includes a data classification module, a host interface module and a data flash memory module; Step S400: During the writing process, the wafer-level load analysis module detects the occurrence of wafer load imbalance in real time, and predicts whether the flash memory transaction determined by the static address mapping allocation method will be allocated to the wafer that is being released, so as to decide whether to enable dynamic relocation; Step S500: According to the trigger constraint information obtained by the hot and cold identification module, the time window module and the chip detection module, the trigger judgment module determines whether to start the release operation according to the trigger constraint information; Step S600: the data classification module classifies the account information according to the multiple account identifiers corresponding to the account information, generates an account list, and allows the user to retrieve the corresponding account data according to the account list; the transaction security protection unit protects the transmission and storage of the account data; The step S500 includes: Step S501, the hot and cold identification module migrates the page to be released to the cold block, puts the host I\O request into the hot block, realizes hot and cold separation, and determines the target erase block by updating the distance. When the data is accessed for the first time, the update distance is updated to the value of the write flash transaction counter, and the access mark position is 1, indicating the first access; when the distance is updated, the update distance is the difference between the current counter and the previous update distance, and the access mark position is 0, indicating that it is not new data, that is, it is not the first access; the initial position of the access mark bit is 0, and the physical page has no data; Step S502, pre-set soft threshold and hard threshold, when the soft threshold of the release operation is reached, the time window module predicts the amount of data written by the host I\O request of the next time window by calculating the moving average; when the hard threshold of the release operation is reached, the release operation is executed in a traditional way; set up n time windows t1, t2, t3...tn, and count the amount of access data in each time window as S1, S2, S3...Sn, respectively, give a higher weight to the most recent time window, and increase the weight in turn according to the distance in time. The weights of each time window are G1, G2, G3...Gn from low to high, and the prediction value of the corresponding n+1 time window is calculated, and the calculation formula is: ； Step S503: Set a flag bit Yd for burst access. If it is predicted that burst access will occur within the next time window, set the burst access flag bit to true. Within the soft threshold range, the release job will not be triggered. If it is predicted that burst access will not occur within the next time window, set the burst access flag bit to false. The time window also obtains the estimated time overhead Q1 for executing the predicted host I / O requests, providing decision-making information for early triggering of the release job. The calculation formula is: ； in, Indicates the completion time of a read operation. Indicates the number of write transactions generated, Indicates the number of read transactions generated; The step S500 further includes: Step S504: According to the chip detection module, the host I / O requests and release job requests can be evenly distributed to different flash resources. Through the trigger constraint calculation formula, the host I / O requests and release job requests can be effectively combined and completed within the time window, and parallel processing can be carried out using the parallelism inside the flash space, or the valid pages can be cached first and then postponed for processing. Conversely, within the next time window, even with the assistance of the valid page cache, the host I / O requests and release jobs cannot be combined and served to maximize the benefit. At this time, the release job cannot be triggered in advance. The trigger constraint calculation formula is: ； Among them, Q2 represents the sum of the chip internal load and external load as the internal load of the flash memory space, m represents the number of chips in the flash memory space is set to m, Q3 represents the release overhead caused by erasing the target victim block, Indicates the size of a time window, Indicates the remaining capacity of the effective page cache; It is the write delay time.

2. The account data management method based on bastion host and distributed processing according to claim 1 is characterized in that: The step S300 includes: Step S301: In the data end, the host interface module receives the host I / O requests sent by the device end to the data end. The host I / O requests are any one of write instructions and read instructions, and the host I / O requests are split into flash transactions of page size. Step S302: The data flash module converts the logical address of the cached flash transaction into a physical address through static address mapping, determines the channel number, chip number, and wafer number, and requests to pass in the physical address information to the wafer-level load analysis module.

3. The account data management method based on bastion host and distributed processing according to claim 1 is characterized in that: The step S400 includes: Step S401, the wafer-level load analysis module detects the occurrence of wafer load imbalance in real time: calculates the wafer load value, which indicates how long it takes for a flash memory transaction to be sent to the wafer to be serviced, and the wafer load value indicates the busyness of a wafer in the current period of time. By comparing the wafer loads of different wafers, the load size on the wafer and the load difference on different wafers can be obtained from the number of write operations and the number of erase operations as well as the sum of the write and erase flag bits, and the load corresponding to different wafers can be obtained. The counter in the wafer is used to record the number of flash memory transactions of the wafer located in the cache chip, serving to record the wafer load size in real time. The wafer load value calculation formula is: ； in, Indicates the wafer load value, Indicates the write operation waiting to be executed in the corresponding wafer. The write operation can be any of the following: user device write, mapping data write, and release recovery write. Indicates the erase operation waiting to be executed in a corresponding wafer; The flag bits indicating the erase operation and the write operation; the flag bits for the wafer to perform read, write and erase are Br, Bw and Be respectively, 0 means no, 1 means yes, indicating the operation the wafer is performing.

4. The account data management method based on bastion host and distributed processing according to claim 3 is characterized in that: The step S400 further includes: Step S402: The size of the counter can reflect the load situation on each wafer. p is the difference between the maximum and minimum values of the counters set on several wafers, and q is the number of planes on a single wafer. If p≥q, it means that the wafer load distribution in the system is unbalanced, and dynamic relocation should be started to locate the request to the wafer with the minimum load. If p<q, it means that the load distribution on different wafers is balanced. By comparing the loads corresponding to different wafers, the load size on the wafers and the load differences between different wafers are judged, so as to detect the occurrence of wafer load imbalance in real time.

5. The account data management method based on bastion host and distributed processing according to claim 4 is characterized in that: The step S400 further includes: Step S403, predict whether the flash memory transaction determined by the allocation method of the static address mapping will be allocated to the wafer that is undergoing the release operation, so as to decide whether to enable dynamic relocation: if the count number of the erase counter on the wafer is greater than 0, it means that the release operation is being performed. If the release operation is allocated to the wafer that is undergoing the release operation due to the static address mapping, dynamic relocation is enabled, and the flash memory transaction is allocated to the wafer with the smallest load value. For wafers with the same load value, pages are sent to different wafers in a round-robin manner, so that the request is sent to the wafer with the smallest load in a round-robin manner; if the count number of the erase counter on the wafer is equal to 0, it means that the release operation is not being performed, and the write flash memory transaction does not conflict with the release operation, and step S402 is repeated to determine the load imbalance caused by the access mode of the host I\O request; Step S404, for some flash transactions, the channel number, chip number and wafer number are re-determined, the plane number of the flash transaction is determined as the plane round on the target wafer, and the round is updated to the next bit after assignment. Finally, after the flash transaction is determined, it will enter the corresponding chip waiting queue and update the load information.

6. An account data management system based on a bastion host and distributed processing, applied to the account data management method based on a bastion host and distributed processing according to any one of claims 1 to 5, characterized in that: The system includes a connected device end, a first server, a second server and a data end, wherein the data end includes: Wafer-level load analysis module, used to detect the occurrence of wafer load imbalance in real time, predict whether the flash transaction determined by the static address mapping allocation method will be allocated to the wafer in the process of releasing the operation, so as to decide whether to enable dynamic relocation; The data flash module is used to convert the logical address of the cached flash transaction into a physical address through static address mapping, determine the channel number, chip number and wafer number, and carry the physical address information request to the wafer-level load analysis module; A trigger judgment module is used to determine whether to start a release operation according to the trigger constraint information; The data classification module is used to classify the account information according to the multiple account identifiers corresponding to the account information, generate an account list, and allow the user to retrieve the corresponding account data according to the account list; the transaction security protection unit protects the transmission and storage of the account data; The data classification module comprises: The hot and cold identification module is used to migrate the pages to be released to the cold block, put the host I\O request into the hot block, realize the hot and cold separation, and decide the target erase block by updating the distance; The time window module is used to predict the amount of data written by the host I\O request in the next time window by calculating the moving average, and then determine whether to trigger the release job in advance; The chip detection module is used to evenly distribute the host I\O requests and release job requests to different flash memory resources. By triggering the constraint calculation formula, the host I\O requests and release job requests can be effectively combined and completed within the time window, and then determine whether to delay the triggering of the release job.

Images