Computing method and system based on computing storage device key-value database near-storage computing

Abstract

This invention discloses a near-memory computing method and system for key-value databases based on computing storage devices, belonging to the field of computer information processing technology. The method includes the following steps: collecting access records formed by multiple storage nodes in a distributed computing storage device under a unified time scale, and labeling each access record with a corresponding time offset value to construct an access record set; performing time position callback processing based on the access record set, relocating the access records in time using the time offset value, so that the access records are merged into the same logical time period, forming a unified time period access record set. This invention achieves consistent expression of access data in logical time through unified time scale and time offset processing, improving the accuracy of hotspot identification; simultaneously, through access change splitting and progressive migration control, the migration process is advanced in stages and synchronized with access changes, reducing the concentration of resource consumption and improving system operational stability.

Description

Technical Field

[0001] This invention relates to the field of computer information processing technology, and more specifically to a near-memory computing method and system for key-value databases based on computing storage devices. Background Technology

[0002] Near-store computing for key-value databases refers to moving operations such as querying, filtering, and aggregation, which were originally performed by the host CPU, to the storage device. These operations are then performed directly by the computing-capable storage medium, allowing data to be processed at its source or storage location, thus reducing frequent data transfers between storage and the host. This approach leverages the characteristics of key-value structures—fast data location via keys and content carried by values—by performing key matching and partial value calculations or filtering on the storage side. Only necessary results are returned to the host, eliminating the need to transmit the complete data set. This reduces bus bandwidth usage and access latency, improves overall processing efficiency and system responsiveness in large-scale data scenarios, and enhances system stability and scalability in high-concurrency environments.

[0003] The existing technology has the following shortcomings: In existing distributed near-storage deployments, each storage node typically uses its local time to statistically analyze access frequency, identifying hot data and triggering data migration. However, due to slight differences in the time bases of each node, the time windows used for statistics are prone to overlap or misalignment. This can cause access traffic within the same time period to be repeatedly counted or accumulated in segments across different nodes, amplifying local access intensity and leading the system to mistakenly classify cold data that is actually in a low-access state as hot data. In this situation, the system will initiate a large-scale migration operation. The migration process itself consumes a significant amount of bandwidth and computing resources, easily triggering a chain reaction of increased load, exacerbating request queuing, and even causing a sharp drop in throughput, ultimately resulting in a performance avalanche.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a near-memory computing method and system for key-value databases based on computing storage devices, so as to solve the problems in the background art mentioned above.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a near-memory computing method for key-value databases based on computing storage devices, comprising the following steps: Access records formed by multiple storage nodes in a distributed computing storage device under a unified time scale are collected, and the corresponding time offset value is marked for each access record to construct a set of access records containing time offset information. Based on the set of access records containing time offset information, time position callback processing is performed. The access records are repositioned in time by using the time offset value, so that the access records are merged into the same logical time period and form a unified time period access record set. By continuously splitting the changes in the number of accesses in the access record set within a unified time period, a one-time surge in access volume is decomposed into multiple incremental access segments, resulting in a continuously smooth access record set. A phased cumulative judgment mechanism is constructed based on the incremental access segments in the continuous smooth access record set, and a progressive migration judgment result is generated by the cumulative results of multiple continuous logical time periods; Based on the progressive migration judgment results, the data migration process is controlled in segments according to time sequence, and the migration progress rhythm is kept synchronized with the change rate of the continuously smooth access record set, thereby suppressing the concentrated growth of migration load and reducing system throughput fluctuations.

[0007] Preferably, the steps for constructing the access record set are as follows: When collecting access behavior, key-value database access requests in multiple storage nodes are recorded, key identification information, access operation type information and access occurrence order information are extracted, and local timestamps are generated. The local timestamps are mapped to time position identifiers in a unified time scale to form access record entries. When introducing time offset information, the time offset is obtained based on the correspondence between the time position identifier in the access record entry and the internal time reference of the storage node, and the time offset is embedded in the access record entry, so that the access record entry contains access behavior information, time position identifier and time offset information. When organizing access record entries, access record entries containing time offset information are sorted according to time position identifiers and grouped according to time position intervals to form a segmented clustered access record structure. When organizing the access record set, the access record entries that have been sorted and grouped are arranged in a unified structure to maintain the correspondence between time location identifiers, time offset information and access behavior information, thus forming the access record set.

[0008] Preferably, the process for constructing a set of access records for a unified time period is as follows: When extracting access records, the time position identifier and time offset value in each access record are parsed from the set of access records containing time offset information to determine the target time position range of the access record in a unified time scale. When performing time relocation, the access record is moved based on the target time location interval, and the time location identifier of the access record is updated, while maintaining the correspondence between the time offset value and the time location identifier. When organizing access records, access records that have completed time relocation are grouped according to logical time periods and arranged according to the order in which the access occurred, forming a set of access records within the logical time period. When integrating the access record set, the access records within each logical time period are organized continuously, maintaining the correspondence between time location identifiers, time offset values ​​and access behavior information, to form a unified time period access record set.

[0009] Preferably, the steps for generating a set of continuously smooth access records are as follows: When extracting access counts, the number of access records is obtained for each logical time period in the access record set of the same time period, and a sequence of access count changes arranged by logical time period is constructed, while retaining the time position identifier and time offset information; When analyzing changes in access counts, the system identifies sudden increases in access counts based on the access count change sequence and divides the changes in access counts into multiple increasing access segments according to the logical time period order. When organizing access records, the access records in the set of access records for the same time period are reallocated according to the incremental access segments. The access records are divided into the corresponding access segments in chronological order, and the correspondence between the time position identifier, time offset information and access behavior information is maintained. When integrating access segments, multiple incremental access segments are organized sequentially according to logical time periods to form a continuous and smooth set of access records.

[0010] Preferably, when analyzing the changes in access counts, the sudden increase intervals in the access count change sequence are divided into segments according to the logical time period order, and the number of access records corresponding to each logical time period is progressively allocated, so that each increasing access segment corresponds to a continuous logical time period interval and maintains the progressively increasing relationship of access counts.

[0011] Preferably, the progressive migration judgment generation steps are as follows: When extracting access segments, the corresponding access records are obtained around the incremental access segments in the set of continuous smooth access records, and the correspondence between the incremental access segments and logical time periods is established to form a sequence of access counts; When constructing the cumulative relationship, the number of visits is stacked segment by segment according to the logical time period based on the access number sequence, and the access records within the cumulative range are uniformly marked to form a phased cumulative result sequence; When dividing the judgment levels, the cumulative stages are divided into levels according to the cumulative result sequence of each stage, and each cumulative stage is bound to the corresponding logical time period and access record set. When generating the judgment results, the migration judgment results are generated segment by segment according to the stage order based on the judgment level, and the correspondence between the migration judgment results and the time position identifier and time offset information in the access record is maintained.

[0012] Preferably, the cumulative result sequence is arranged in ascending order of access count, and the set of access records corresponding to each cumulative stage is associated with a logical time interval to form a continuously progressive migration judgment relationship.

[0013] Preferably, the data migration phased advancement control steps are as follows: When extracting the judgment stage, the range of access records corresponding to each judgment stage is obtained based on the progressive migration judgment results, and the correspondence between the judgment stage and the logical time period interval is established to form the data migration range; When dividing the migration phases, the data migration process is segmented based on the data migration scope, so that each migration phase is arranged in logical time period order, and the correspondence between the time position identifier of the access record and the time offset information is maintained. When adjusting the pace of progress, the order of each migration stage is adjusted synchronously by combining the changes in the number of accesses in the continuous and smooth access record set, so that the migration stage is consistent with the access change process within the logical time period. During the migration process, data migration is carried out step by step according to the migration stage sequence, while maintaining the correspondence between the migration stage and the scope of access records.

[0014] A near-memory computing system for key-value databases based on computing storage devices includes an access record construction module, a time base alignment module, an access smoothing module, a migration determination module, and a migration control module. The access record construction module collects access records formed by multiple storage nodes in the distributed computing storage device under a unified time scale, and marks the corresponding time offset value for each access record to construct a set of access records containing time offset information. The time base alignment module performs time position callback processing based on the set of access records containing time offset information. It relocates the access records by time offset value, so that the access records are merged into the same logical time period and form a unified time period access record set. The access smoothing module continuously splits the access frequency changes in the access record set within a unified time period, decomposing a one-time surge in access volume into multiple incremental access segments to obtain a continuously smoothed access record set. The migration determination module constructs a phased cumulative determination mechanism based on the incremental access segments in the continuous smooth access record set, and generates progressive migration determination results through the cumulative results of multiple continuous logical time periods. The migration control module performs segmented control of the data migration process in chronological order based on the progressive migration judgment results, and keeps the migration progress rhythm synchronized with the rate of change of the continuously smooth access record set, thereby suppressing the concentrated growth of migration load and reducing system throughput fluctuations.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention introduces a unified time scale and time offset annotation to access records, and combines time position callback processing to enable access data that was originally scattered across different storage nodes to be consistently expressed within a unified logical time range. This avoids statistical deviations caused by misaligned time windows, makes the calculation results of access frequency more stable, improves the accuracy of hotspot identification, reduces misjudgments caused by statistical errors, and enables data access status to more realistically reflect actual load changes.

[0016] This invention breaks down access changes into continuous segments and constructs a progressive migration determination mechanism, transforming data migration from a one-time trigger to a phased process. Simultaneously, it synchronizes the migration process with access changes, allowing the migration behavior to unfold gradually along the time dimension. This reduces the concentrated occupation of bandwidth and computing resources during the migration process, alleviates fluctuations in request processing pressure, makes the overall processing process more stable, and improves the system's operational stability in high-concurrency access scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of the key-value database near-memory computing method based on computing storage devices according to the present invention.

[0019] Figure 2 This is a schematic diagram of the key-value database near-memory computing system based on computing storage devices according to the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0021] This invention provides, for example Figure 1 The key-value database near-memory computing method shown includes the following steps: Access records formed by multiple storage nodes in a distributed computing storage device under a unified time scale are collected, and the corresponding time offset value is marked for each access record to construct a set of access records containing time offset information. In the operating environment of distributed computing storage devices, in order to ensure that the access behaviors generated by different storage nodes have a unified reference in the time dimension, it is necessary to collect access records in detail and combine them with time offset information to complete the structured organization, thereby forming a data foundation that can support subsequent time position callback processing. The specific implementation steps are as follows: As multiple storage nodes respond to key-value database access requests, each access action is recorded one by one. During the recording process, the key identifier information, access operation type information, and access sequence information corresponding to the access request are extracted synchronously. A corresponding local timestamp is generated at the instant the access action occurs. At the same time, a unified time scale mapping relationship is pre-established within each storage node, so that the local timestamp can be converted into a time position identifier in the unified time scale. During the conversion process, according to the division granularity of the unified time scale, the local timestamp is mapped to the corresponding time scale interval position, and the time position identifier is bound to the access action information, thereby forming an access record entry containing the unified time scale position.

[0022] For each existing access record entry, time offset information is introduced. By parsing the correspondence between the time position identifier in the unified time scale and the internal time reference of the current storage node, the actual offset of the access record in the unified time scale is obtained. This offset is embedded into the access record entry according to fixed data fields, and the association between the time offset information and the time position identifier is maintained during the embedding process. This ensures that each access record has three types of content: access behavior information, time position identifier, and time offset information. At the same time, the time offset information is written one by one according to the access order during processing, so that the access record has the ability to express continuously in the time dimension.

[0023] Access records containing time offset information are centrally processed. By sorting access records from different storage nodes according to time position identifiers in a unified time scale, access records with adjacent time positions are arranged continuously. At the same time, the integrity of time offset information and access behavior information is maintained during the arrangement process, so that access records generated by different storage nodes can form a continuous distribution within the same time scale framework. In the processing, access records are grouped according to time position intervals, and access records within the same time scale interval are grouped into the same group, so that access records present a segmented clustered state in the time dimension, providing a continuous data foundation for subsequent unified time period processing.

[0024] Based on the completed sorting and grouping of access records, the access records are organized into a unified structure. Each access record is arranged and combined according to a preset data organization format, so that the access record set as a whole forms a data sequence with a unified structure. In the process of organization, the correspondence between time position identifier, time offset information and access behavior information is preserved. This allows the access record set to reflect the distribution of access behavior under a unified time scale and to support subsequent time position adjustment processing through time offset information, thus forming an access record set containing time offset information.

[0025] Based on the set of access records containing time offset information, time position callback processing is performed. The access records are repositioned in time by using the time offset value, so that the access records are merged into the same logical time period and form a unified time period access record set. To ensure consistent representation of access records generated by different storage nodes on a unified time scale, a time position callback process is performed on the access records. This allows each access record to be relocated based on its time offset information and then merged and organized within a unified logical time period, thus forming a unified time period access record set. The specific implementation process is as follows: Access records are extracted one by one from the set of access records containing time offset information. For each access record, the time position identifier and time offset value contained therein are parsed. During the parsing process, a unified time scale is used as a reference to locate the time interval corresponding to the time position identifier. At the same time, the time offset direction and offset magnitude represented by the time offset value are read. The time position identifier and time offset value are combined to determine the target time position interval of the access record in the unified time scale. During the processing, the key identifier, access operation type and access sequence of the access record are completely preserved, so that the access record can still maintain the original sequential relationship between access behaviors during the time position adjustment process, thereby providing a clear positioning basis for subsequent time position adjustment. For each access record within a defined target time interval, a time relocation operation is performed. During this process, the access record's current time position is moved based on the time offset value, transferring it from its original time position to the target time interval. Following a unified time scale, the access record is then assigned to its corresponding logical time period. During this assignment, the access record's time position identifier is updated synchronously to ensure consistency with the target time interval. Simultaneously, the correlation between the time offset value and the updated time position identifier is preserved, allowing the access record to still reflect its original time offset characteristics after time relocation. By processing each access record sequentially according to the access order, time relocation from different storage nodes gradually converges within a unified time scale. For access records after time relocation processing, access records within the same logical time period are centrally organized. During the organization process, access records with the same time interval identifier are grouped into the same set based on the logical time period. The access records grouped into the same logical time period are then arranged according to the order of access occurrence, so that the access records form a continuous distribution structure within the logical time period. At the same time, the time offset value and access behavior information of each access record are retained during the organization process. This allows the access records to reflect the distribution density of access behavior within the logical time period, and the time offset value can reflect the positional differences of access behavior in the original time axis. The access record sets within each logical time period are arranged according to the chronological order of the logical time periods, so that multiple logical time periods form a continuous connection relationship. Based on the completion of the access records within the logical time period, the access records in each logical time period are uniformly integrated and processed. The access records in each logical time period are linked together according to a unified data organization method, so that the access record set as a whole forms a continuous data sequence with the logical time period as the main line. During the integration process, the correspondence between the time position marker, time offset value and access behavior information in the access records is maintained. This ensures that the access record set for the unified time period can not only reflect the concentrated distribution of access behavior under a unified time scale, but also support subsequent access change analysis and processing through time offset information, thereby completing the construction of the access record set for the unified time period.

[0026] By continuously splitting the changes in the number of accesses in the access record set within a unified time period, a one-time surge in access volume is decomposed into multiple incremental access segments, resulting in a continuously smooth access record set. Based on the existing set of access records for a unified time period, to mitigate the judgment bias caused by a concentrated surge in access counts within a single time period, the process of access count changes is continuously split. This breaks down the originally concentrated changes in access volume into multiple access segments with a progressive relationship, thereby constructing a continuous and smooth set of access records. The specific implementation steps are as follows: For each logical time period in the unified time period access record set, the number of access records in each logical time period is extracted segment by segment, and the access count change sequence is constructed according to the arrangement order of the logical time periods. In the construction process, the total number of access records in each logical time period is used as the access count of the corresponding time period, and the access counts between adjacent logical time periods are arranged continuously so that the access counts form an ordered change relationship in the time dimension. At the same time, the time position identifier and time offset information of the access records in each logical time period are retained in the processing, so that the access count change sequence can reflect the true distribution of access behavior under a unified time scale, thus providing a complete data foundation for subsequent access count change decomposition. Based on the sequence of changes in access counts, the sudden increase intervals in access counts are analyzed segment by segment. By comparing the changes in access counts between adjacent logical time periods, intervals in which access counts increase intensively within a short period of time are identified as sudden increase intervals. In the identification process, the process of changing access counts within the sudden increase intervals is segmented using logical time periods as the unit of division. The originally concentrated increase in access counts is broken down into multiple continuously increasing access segments. In the process of segmentation, the access counts are progressively allocated according to the logical time period sequence, so that each access segment corresponds to an independent logical time period interval, and the access counts form a gradual increasing trend among multiple logical time periods, thereby transforming a one-time sudden increase in access counts into multiple increasing access segments. Based on the multiple incremental access segments after the splitting process, the access records are reorganized. The access records in the original unified time period access record set are redistributed according to the split access segments, so that each access segment corresponds to a set of access records. During the allocation process, according to the access frequency range corresponding to the access segment, the access records are divided into the corresponding access segments one by one in chronological order, while maintaining the correlation between the time position identifier, time offset information and access behavior information of the access records. This allows the access records to reflect the progressive change process of the access frequency after the reorganization. At the same time, the number of access records between each access segment is kept consistent with the corresponding access frequency during the processing, so that the access record set presents a continuous progressive distribution pattern in the time dimension. Based on the reorganization of access records, the access records in all access segments are integrated as a whole. Multiple increasing access segments are connected in logical time sequence to form a continuous connection between the access segments, thereby constructing a continuous and smooth access record set. During the integration process, the correspondence between the time position markers, time offset information and access behavior information in the access records is maintained. This ensures that the continuous and smooth access record set can reflect the change in the number of accesses from a concentrated surge to a gradual progression, and also maintain the continuous distribution of access records under a unified time scale. This completes the continuous decomposition processing of the change in the number of accesses and obtains the continuous and smooth access record set.

[0027] A phased cumulative judgment mechanism is constructed based on the incremental access segments in the continuous smooth access record set, and a progressive migration judgment result is generated by the cumulative results of multiple continuous logical time periods; Based on the continuous and smooth set of access records and the temporal continuity between incremental access segments, in order to transform the migration judgment process from a single trigger to a progressive one, a phased cumulative judgment mechanism is constructed. This mechanism continuously overlays incremental access segments along the time dimension and generates progressive migration judgment results accordingly. The specific implementation process is as follows: Based on the multiple incremental access segments already formed in the continuous smooth access record set, the corresponding access records for each incremental access segment are extracted one by one according to the logical time period arrangement. The correspondence between access segments and time intervals is established in units of logical time periods. During the extraction process, the access records in each incremental access segment are arranged according to the time position identifier, while retaining the time offset information and access behavior information in the access records, so that each incremental access segment corresponds to a certain logical time period range. On this basis, the number of access records in each logical time period is counted one by one, so that the incremental access segments form a continuous sequence of access counts in the time dimension, thereby providing a continuous and orderly data representation for subsequent phased cumulative processing. For the established access count sequence, a segmented cumulative relationship is constructed according to the chronological order of logical time periods. The access counts within adjacent logical time periods are sequentially superimposed. During the superposition process, the access count of the first logical time period is used as the starting value, and the access counts of subsequent logical time periods are added to the current cumulative result segment by segment. This allows the cumulative result to gradually expand its coverage in the time dimension. At the same time, after each superposition is completed, all access records involved in the current cumulative range are uniformly marked, so that the cumulative result not only reflects the changes in access counts but also corresponds to a specific set of access records. During the processing, the continuity of time position identifiers and time offset information is maintained in the accumulation process, so that the cumulative result presents a continuous expansion state in the time dimension, thereby forming a phased cumulative result sequence in the order of logical time periods. After completing the segmented accumulation process, the resulting phased accumulation result sequence is hierarchically divided. The access count range corresponding to each accumulation phase is bound to the logical time period it covers, and an independent judgment level is established for each accumulation phase. During the division process, each phase is arranged in ascending order of the accumulated results, so that a clear progressive relationship is formed between different phases. At the same time, the corresponding set of access records, time position identifiers and time offset information are retained in each phase, so that each judgment level can fully reflect the distribution of access behavior within its covered time range. Through phase division, the originally continuous access change process is transformed into multiple judgment intervals with hierarchical relationships, thereby constructing a judgment expression structure with phased characteristics. Based on the established judgment hierarchy, migration judgment results are generated segment by segment according to the access frequency range corresponding to each stage. During the generation process, the process is advanced step by step according to the stage sequence. The cumulative results of the earlier stages are used as the initial judgment basis, and the cumulative results of the later stages are used as the judgment enhancement basis. This allows the migration judgment results to gradually evolve as the cumulative access frequency increases. In each stage, the migration judgment results are associated with the corresponding set of access records, so that the migration judgment results can clearly correspond to the specific access behavior range. At the same time, the mapping relationship between the migration judgment results and the time location identifier and time offset information is maintained. This allows the entire migration judgment process to unfold step by step along the logical time period sequence, thereby forming a progressive migration judgment result based on the cumulative results of multiple consecutive logical time periods.

[0028] Based on the progressive migration judgment results, the data migration process is controlled in segments according to time sequence, and the migration progress rhythm is kept synchronized with the change rate of the continuously smooth access record set, thereby suppressing the concentrated growth of migration load and reducing system throughput fluctuations. Based on the progressive migration judgment results and the access changes in the continuous and smooth access record set, in order to ensure that the data migration process unfolds gradually in chronological order, segmented advancement control is implemented on the migration process to synchronize the migration behavior with the access change process in the time dimension. The specific implementation steps are as follows: Based on the progressive migration judgment results, the access record range corresponding to each judgment stage is extracted according to the logical time period arrangement. The correspondence between each judgment stage and the logical time period it covers is established. During the extraction process, the access records involved in each judgment stage are arranged according to the time position identifier, while retaining the time offset information and access behavior information in the access records. This ensures that each judgment stage corresponds to the access changes within a continuous time range. On this basis, a corresponding data migration range expression is generated for each judgment stage, so that the migration range and access records are consistent in the time dimension. This provides a clear segmentation basis for the subsequent migration in chronological order. Based on the established correspondence between the judgment stage and the migration scope, the data migration process is divided into segments, breaking down the overall migration process into multiple continuous stages. Each migration stage is arranged sequentially according to the logical time period. During the division process, each migration stage corresponds to the access record range covered by only one judgment stage. At the same time, the time position identifier and time offset information of the access records remain unchanged in each migration stage, so that the migration stage can directly reflect the access changes within the corresponding time range. Through this segmentation method, the originally centralized migration process is transformed into a continuous migration process that unfolds step by step along the time axis, thus forming a migration structure that progresses in chronological order. By combining the changes in the incremental access segments in the continuous smooth access record set, the pace of each migration stage is adjusted synchronously. During the adjustment process, the changes in the number of accesses in each logical time period in the continuous smooth access record set are used as a reference to ensure that the progress of each migration stage is consistent with the growth process of access changes within the corresponding time period. When the number of accesses shows an incremental change within the logical time period, the corresponding migration stage is advanced step by step in the same order, so that the migration process is consistent with the access changes in the time dimension. At the same time, the order between migration stages is kept from overlapping during the processing, so that the migration behavior always unfolds in the logical time period order, thereby keeping the pace of migration synchronized with the speed of access changes. Based on the completion of migration phase division and rhythm adjustment, the overall data migration process is executed continuously, so that each migration phase unfolds in a predetermined order. During the execution, each migration phase only processes the access records within its corresponding time range, so that the migration behavior presents a gradual progress in the time dimension. By synchronizing the migration progress rhythm with the continuous smooth change rate of the access record set, the migration load is distributed in a distributed manner on the time axis, thereby avoiding the concentrated execution of multiple migration phases within the same time range, thus suppressing the concentrated growth of migration load in a short period of time, and keeping the data access processing process in a stable state during the migration process, reducing the occurrence of throughput fluctuations.

[0029] This invention introduces a unified time scale and time offset annotation to access records, and combines time position callback processing to enable access data that was originally scattered across different storage nodes to be consistently expressed within a unified logical time range. This avoids statistical deviations caused by misaligned time windows, makes the calculation results of access frequency more stable, improves the accuracy of hotspot identification, reduces misjudgments caused by statistical errors, and enables data access status to more realistically reflect actual load changes.

[0030] This invention breaks down access changes into continuous segments and constructs a progressive migration determination mechanism, transforming data migration from a one-time trigger to a phased process. Simultaneously, it synchronizes the migration process with access changes, allowing the migration behavior to unfold gradually along the time dimension. This reduces the concentrated occupation of bandwidth and computing resources during the migration process, alleviates fluctuations in request processing pressure, makes the overall processing process more stable, and improves the system's operational stability in high-concurrency access scenarios.

[0031] This invention provides, for example Figure 2 The key-value database near-memory computing system shown includes an access record construction module, a time base alignment module, an access smoothing module, a migration determination module, and a migration control module. The access record construction module collects access records formed by multiple storage nodes in the distributed computing storage device under a unified time scale, and marks the corresponding time offset value for each access record to construct a set of access records containing time offset information. The time base alignment module performs time position callback processing based on the set of access records containing time offset information. It relocates the access records by time offset value, so that the access records are merged into the same logical time period and form a unified time period access record set. The access smoothing module continuously splits the access frequency changes in the access record set within a unified time period, decomposing a one-time surge in access volume into multiple incremental access segments to obtain a continuously smoothed access record set. The migration determination module constructs a phased cumulative determination mechanism based on the incremental access segments in the continuous smooth access record set, and generates progressive migration determination results through the cumulative results of multiple continuous logical time periods. The migration control module performs segmented control of the data migration process in chronological order based on the progressive migration judgment results, and keeps the migration progress rhythm synchronized with the rate of change of the continuously smooth access record set, thereby suppressing the concentrated growth of migration load and reducing system throughput fluctuations.

[0032] The key-value database near-storage calculation method based on computing storage device provided in this embodiment of the invention is implemented by the aforementioned key-value database near-storage calculation system based on computing storage device. For details of the specific methods and processes of the key-value database near-storage calculation system based on computing storage device, please refer to the embodiments of the key-value database near-storage calculation method based on computing storage device, which will not be repeated here.

[0033] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A near-memory computation method for key-value databases based on computing storage devices, characterized in that, Includes the following steps: Access records generated by multiple storage nodes in a distributed computing storage device under a unified time scale are collected, and a corresponding time offset value is marked for each access record to construct an access record set. Based on the set of access records, the time position callback process is executed. The access records are repositioned by time offset value, so that the access records are merged into the same logical time period and form a unified time period access record set. By continuously splitting the changes in the number of accesses in the access record set within a unified time period, a one-time surge in access volume is decomposed into multiple incremental access segments, resulting in a continuously smooth access record set. A phased cumulative judgment mechanism is constructed based on the incremental access segments in the continuous smooth access record set, and a progressive migration judgment result is generated by the cumulative results of multiple continuous logical time periods; Based on the progressive migration judgment results, the data migration process is controlled in segments according to time sequence, and the migration progress rhythm is kept synchronized with the rate of change of the continuously smooth access record set.

2. The key-value database near-memory computation method based on computing storage devices according to claim 1, characterized in that, The steps for constructing the access record set are as follows: When collecting access behavior, key-value database access requests in multiple storage nodes are recorded, key identification information, access operation type information and access occurrence order information are extracted, and local timestamps are generated. The local timestamps are mapped to time position identifiers in a unified time scale to form access record entries. When introducing time offset information, the time offset is obtained based on the correspondence between the time position identifier in the access record entry and the internal time reference of the storage node, and the time offset is embedded in the access record entry. When organizing access record entries, access record entries containing time offset information are sorted according to time position identifiers and grouped according to time position intervals to form a segmented clustered access record structure. When organizing the access record set, the access record entries that have been sorted and grouped are arranged in a unified structure to maintain the correspondence between time location identifiers, time offset information and access behavior information, thus forming the access record set.

3. The key-value database near-memory computation method based on computing storage devices according to claim 2, characterized in that, The process of constructing a unified time period access record set is as follows: When extracting access records, the time position identifier and time offset value in each access record are parsed from the access record set to determine the target time position range of the access record in a unified time scale; When performing time relocation, the access record is moved based on the target time location interval, and the time location identifier of the access record is updated, while maintaining the correspondence between the time offset value and the time location identifier. When organizing access records, access records that have completed time relocation are grouped according to logical time periods and arranged according to the order in which the access occurred, forming a set of access records within the logical time period. When integrating the access record set, the access records within each logical time period are organized continuously, maintaining the correspondence between time location identifiers, time offset values ​​and access behavior information, to form a unified time period access record set.

4. The key-value database near-memory computation method based on computing storage devices according to claim 3, characterized in that, The steps for generating a continuous smooth access record set are as follows: When extracting access counts, the number of access records is obtained for each logical time period in the access record set of the same time period, and a sequence of access count changes arranged by logical time period is constructed, while retaining the time position identifier and time offset information; When analyzing changes in access counts, the system identifies sudden increases in access counts based on the access count change sequence and divides the changes in access counts into multiple increasing access segments according to the logical time period order. When organizing access records, the access records in the set of access records for the same time period are reallocated according to the incremental access segments. The access records are divided into the corresponding access segments in chronological order, and the correspondence between the time position identifier, time offset information and access behavior information is maintained. When integrating access segments, multiple incremental access segments are organized sequentially according to logical time periods to form a continuous and smooth set of access records.

5. The key-value database near-memory computation method based on computing storage devices according to claim 4, characterized in that, When analyzing changes in access counts, the sudden increase intervals in the access count change sequence are divided into segments according to logical time periods, and the number of access records corresponding to each logical time period is progressively allocated.

6. The key-value database near-memory computation method based on computing storage devices according to claim 5, characterized in that, The progressive transition judgment generation steps are as follows: When extracting access segments, the corresponding access records are obtained around the incremental access segments in the set of continuous smooth access records, and the correspondence between the incremental access segments and logical time periods is established to form a sequence of access counts; When constructing the cumulative relationship, the number of visits is stacked segment by segment according to the logical time period based on the access number sequence, and the access records within the cumulative range are uniformly marked to form a phased cumulative result sequence; When dividing the judgment levels, the cumulative stages are divided into levels according to the cumulative result sequence of each stage, and each cumulative stage is bound to the corresponding logical time period and access record set. When generating the judgment results, the migration judgment results are generated segment by segment according to the stage order based on the judgment level, and the correspondence between the migration judgment results and the time position identifier and time offset information in the access record is maintained.

7. The key-value database near-memory computation method based on computing storage devices according to claim 6, characterized in that, In the phased cumulative result sequence, each cumulative stage is arranged in ascending order of access count, and the set of access records corresponding to each cumulative stage is associated with a logical time interval to form a continuously progressive migration judgment relationship.

8. The key-value database near-memory computation method based on computing storage devices according to claim 6, characterized in that, The data migration phased implementation control steps are as follows: When extracting the judgment stage, the range of access records corresponding to each judgment stage is obtained based on the progressive migration judgment results, and the correspondence between the judgment stage and the logical time period interval is established to form the data migration range; When dividing the migration phases, the data migration process is segmented based on the data migration scope, so that each migration phase is arranged in logical time period order, and the correspondence between the time position identifier of the access record and the time offset information is maintained. When adjusting the pace of progress, the order of each migration stage is adjusted synchronously by taking into account the changes in the number of accesses in the continuous smooth access record set. During the migration process, data migration is carried out step by step according to the migration stage sequence, while maintaining the correspondence between the migration stage and the scope of access records.

9. A key-value database near-memory computing system based on computing storage devices, used to implement the key-value database near-memory computing method based on computing storage devices according to any one of claims 1-8, characterized in that, This includes an access record construction module, a time base alignment module, an access smoothing module, a migration determination module, and a migration control module. The access record construction module collects access records formed by multiple storage nodes in the distributed computing storage device under a unified time scale, and marks the corresponding time offset value for each access record to construct an access record set. The time base alignment module performs time position callback processing based on the access record set. It relocates the access records by time offset value, so that the access records are merged into the same logical time period and form a unified time period access record set. The access smoothing module continuously splits the access frequency changes in the access record set within a unified time period, decomposing a one-time surge in access volume into multiple incremental access segments to obtain a continuously smoothed access record set. The migration determination module constructs a phased cumulative determination mechanism based on the incremental access segments in the continuous smooth access record set, and generates progressive migration determination results through the cumulative results of multiple continuous logical time periods. The migration control module performs segmented control of the data migration process in chronological order based on the progressive migration judgment results, and keeps the migration progress pace synchronized with the rate of change of the continuously smooth access record set.

Images