Edge computing low-power-consumption Hash acceleration and compression storage system

Abstract

The invention relates to the technical field of electric digital data processing system architecture, and discloses an edge computing low-power-consumption Hash acceleration and compression storage system, which comprises a data collector based on hardware logic, and the data collector is used for receiving data fragments and adding the data fragments to a current page in a temporary storage pool of a nonvolatile memory during the dormancy period of a main processor and a main memory; the independent batch processing accelerator is awakened to execute hash and compression when the current page data accumulation meets a preset threshold value; the data collector is further configured to maintain volatile page pointers and counters, alternately use temporary pages through hardware ring buffer logic, and write data and metadata characterizing the states of the pointers and counters in an atomized binding manner when the data is appended, and perform a scan to reconstruct the states of the pointers and counters when power-off recovery is performed.

Description

Technical Field

[0001] This invention relates to a low-power hash acceleration and compression storage system for edge computing, belonging to the technical field of electronic digital data processing system architecture. Background Technology

[0002] Currently, for data processing tasks in edge computing, algorithms such as hashing and compression are typically designed in block processing mode to ensure their operational efficiency. This means they require a relatively complete block of data to be input at once for effective operation. However, in massive edge computing applications, such as environmental monitoring or asset tracking scenarios, data streams are not continuous high-throughput streams but rather sparse and fragmented. Sensors may generate only a few bytes of data packets over long periods. To adapt to the requirements of these block processing algorithms, the electronic digital data processing system must set up a temporary buffer to continuously accumulate these arriving data fragments until a complete block is formed. In existing electronic digital data processing system architectures, volatile memory (RAM) is widely used as a temporary buffer due to its high-speed read / write and byte-addressing characteristics. However, volatile memory requires continuous power to maintain data, resulting in static refresh power consumption or leakage power consumption. For low-power edge devices where the main processor is in deep sleep most of the time, the system is forced to maintain the entire volatile memory or its partitions in a continuously powered self-refreshing state in order to protect the data that has not yet accumulated in the temporary buffer from being lost. This architecture leads to the phenomenon that static maintenance power consumption far exceeds dynamic processing power consumption.

[0003] This static power consumption constraint caused by the need for temporary data storage constitutes a fundamental bottleneck for current electronic digital data processing architectures to achieve low power consumption in sparse data applications. Currently, those skilled in the art often focus their research on optimizing dynamic power consumption and computational efficiency during the data processing stage to reduce power consumption, rather than addressing the static power consumption bottleneck during the data accumulation stage. For example, Chinese invention patent CN116957022B discloses a sparse binary neural network hardware accelerator for gesture recognition. This scheme aims to bypass the binary neural network by introducing sparsity and designing a dedicated value prediction unit and compression / decompression module. The redundant computation present in the computation reduces dynamic power consumption and improves energy efficiency during the computation cycle. However, such solutions are essentially optimizations for specific algorithms (such as neural networks) during the execution phase, and their premise is that the data has been collected and is ready. They do not solve the core problem that this invention focuses on: the high static power consumption that the system is forced to generate to maintain the volatile memory (RAM) temporary storage area during the long waiting and accumulation period of massive sparse data (such as sensor fragments). In other words, no matter how efficient the subsequent computing accelerator is, if the data temporary storage front-end itself causes the main energy leakage, the overall low power consumption goal of the system cannot be achieved.

[0004] Therefore, the technical problem to be solved by this invention is how to design an electronic digital data processing system that can autonomously accumulate sparse data fragments with low static power consumption when the main processor and main volatile memory are in deep sleep or power-off state, and efficiently wake up the dedicated processing unit to execute batch processing tasks after accumulation, while solving the durability and reliability problems caused by the introduction of non-volatile storage media in this process. Summary of the Invention

[0005] This invention provides a low-power hash acceleration and compressed storage system for edge computing. Its main purpose is to solve the problem that existing electronic digital data processing architectures rely on volatile memory for temporary data storage, resulting in high static power consumption during the main processor's sleep period, and lack of autonomous accumulation and batch processing mechanisms.

[0006] To achieve the above objectives, the present invention provides a low-power hash acceleration and compressed storage system for edge computing, comprising: Data collector, batch processing accelerator and non-volatile memory, the non-volatile memory contains a pre-defined temporary pool with multiple temporary pages; The data collector is configured to receive data fragments and append them to a current temporary page in the temporary pool when the main processor and main volatile memory are in a low-power state; the data collector is also configured to maintain a volatile page pointer and a volatile hardware accumulator counter; the batch processing accelerator is independent of the main processor and is configured to be woken up when the volatile hardware accumulator counter tracks that the amount of accumulated data in the current temporary page reaches a preset batch processing threshold, and to perform hash calculations and compression processing on the accumulated data; The data collector is also configured to: increment the volatile page pointer according to the logical rules of the circular buffer after the current temporary page has been processed, so as to use different temporary pages in the temporary pool in rotation; and the data collector is also configured to: when appending data fragments, bind the data fragments with an embedded metadata characterizing the current state of the volatile page pointer and the volatile hardware accumulator counter as an atomic write unit for writing; and when the system recovers from a power outage, the data collector is configured to execute a recovery rule: scan the temporary pool to identify the embedded metadata last successfully written, and reconstruct the state of the volatile page pointer and the volatile hardware accumulator counter based on the identified embedded metadata.

[0007] Preferably, the data collector further includes: a history state latch for latching the content of previously successfully appended data fragments, and a content comparator; the data collector is further configured to: upon receiving a data fragment, first instruct the content comparator to perform a hardware comparison of the content of the data fragment with the content of the previously successfully appended data fragment latched in the history state latch; and the data collector is configured to execute a filtering rule: only when the content comparator determines that the two contents are inconsistent, will the operation of appending the data fragment to the current temporary page be performed, and the content of the data fragment be updated to the history state latch; when the content comparator determines that the two contents are consistent, the append operation to the non-volatile memory is skipped.

[0008] Preferably, the data collector also includes a redundancy counter; the data collector is further configured to: increment the redundancy counter when an append operation is skipped; and when an append operation is performed due to inconsistency, write the count value of the redundancy counter in association with the data fragment to the current temporary page and reset the redundancy counter; the batch processing accelerator is configured to read the count value to reconstruct the original data stream when performing hash calculations and compression processing.

[0009] Preferably, the batch processing accelerator is further configured to: detect the existence of a processing failure state during hash calculation and compression processing; and the batch processing accelerator is configured to: when a processing failure state is detected, perform a bypass archiving operation, which includes: writing the unprocessed accumulated data that caused the processing failure in the current temporary page to a dedicated error log page in non-volatile memory; the batch processing accelerator is further configured to still send a signal to the data collector to trigger the incrementing of the volatile page pointer after the bypass archiving operation is completed.

[0010] Preferably, the batch processing accelerator includes its private volatile memory; the batch processing accelerator is configured to read accumulated data from the current staging page into its private volatile memory, and to perform hash calculations and compression processing in its private volatile memory.

[0011] Preferably, the batch processing accelerator is further configured to: start its internal cycle counter before performing hash calculations and compression processing; stop the cycle counter after processing is complete to obtain the execution time; the batch processing accelerator is further configured to: compare the execution time with a preset processing time threshold; and, when the execution time exceeds the processing time threshold, set a status flag to indicate data anomalies, which can be queried by the main processor.

[0012] Preferably, the data collector is a finite state machine implemented based on hardware logic; the data collector also includes an SRAMFIFO; the data collector is configured to: first store the received data fragments into the SRAMFIFO, and then read them from the SRAMFIFO and perform an append operation.

[0013] Preferably, the data collector also includes a batch timeout timer; the data collector is further configured to start the batch timeout timer when appending data fragments to a currently unwritten temporary page begins; the decision rule for the data collector to generate a trigger signal is limited to: when the count value of the volatile hardware cumulative counter... The count value of the batch processing timeout timer The following conditions must be met or At that time, a trigger signal is generated; where, This is the current count value of the volatile hardware cumulative counter. This is the current count value of the batch processing timeout timer. The preset batch processing threshold, The data collector is also configured to reset the volatile hardware cumulative counter and batch timeout timer after the current temporary page has been processed.

[0014] Preferably, the batch processing accelerator is further configured to write the processed data resulting from the hash calculation and compression to an archive page of non-volatile memory after performing hash calculation and compression on the accumulated data.

[0015] Preferably, the batch processing accelerator is also configured to send an instruction to erase the currently processed temporary page after the processed data has been written to the archive page.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. This solution provides an electrical digital data processing system architecture that uses a data collector and a batch processing accelerator independent of the main processor to work together. The data collector is responsible for directly appending arriving data fragments to non-volatile memory, while the batch processing accelerator is woken up after the data accumulates to a preset threshold and reads data from the non-volatile memory to perform processing. This structure at the electrical digital data processing system level separates the data fragment accumulation process from the operating state of the main processor and the main volatile memory. The main processor and the main volatile memory can remain in a deep sleep or power-off state during data accumulation, thereby avoiding the static power consumption generated in existing electrical digital data processing architectures to maintain data in the volatile memory temporary storage area.

[0017] 2. This solution uses non-volatile memory as a data temporary storage medium. At the same time, a temporary page circular buffer management mechanism is integrated into the control logic of the data collector. The data collector does not write to a single temporary page in a fixed manner, but according to its internal control logic state, it adds continuous data fragments to multiple different temporary pages in the temporary pool in turn. This combination of control logic and storage management distributes the write and erase operations of the temporary pages evenly to multiple physical storage blocks at the control logic level, avoiding the problem of local excessive wear that may be caused by using non-volatile memory as a temporary storage area, and improving the long-term operational reliability of the electronic digital data processing system. In the control logic of the data collector in this solution, in addition to integrating a hardware cumulative counter, a batch processing timeout timer is also integrated. The control logic is configured to start this timer simultaneously when receiving data fragments. The decision of the control logic to generate the batch processing trigger signal no longer depends on a single capacity threshold, but on the first arrival of either the capacity threshold or the time threshold. This capacity-time dual triggering mechanism at the control logic level enables the system's batch processing timing to autonomously adapt to the fluctuations in data flow rate in edge scenarios, ensuring processing timeliness when data is depleted, and achieving a system-level balance between power efficiency and processing timeliness.

[0018] 3. When performing data append operations, the data collector in this solution binds data fragments with metadata representing the current system's accumulated state and writes them to non-volatile memory as atomic write units. When the system recovers from a power outage, the data collector's control logic is configured to first scan the metadata in the non-volatile memory. By examining the continuity and integrity of this metadata through recovery logic, the control logic can automatically determine the location of the last successful write and reconstruct its internal accumulated state based on this. This achieves state persistence and fault-tolerant breakpoint resumption at the system level, avoiding data overwriting caused by the loss of volatile state in the control logic. In the control logic of the data collector, a content comparator and a historical state latch are further integrated. When a new data fragment is received, the control logic first performs a comparison operation, comparing the current data with the latched historical data. Only when the comparator determines that the data content has changed will the control logic trigger an append write operation to the non-volatile memory; otherwise, the write operation is skipped. This fragment-level content awareness and filtering mechanism implemented at the front end eliminates redundant data write operations before the data stream enters the storage medium. In edge application scenarios that process low-entropy data streams, the write and erase losses of the storage medium are reduced. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the asynchronous data stream and low-power batch processing triggering mechanism of the system of the present invention; Figure 2 This is a comparison chart showing the optimization effect of the filtering mechanism of this invention on the number of NVM writes; Figure 3 This is a timing diagram of power outage recovery based on metadata scanning according to the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are only some embodiments of this invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] The edge computing low-power hash acceleration and compressed storage system disclosed in this technical solution is an electro-digital data processing system architecture. Its core system architecture consists of a data collector, an independent batch processing accelerator, and a non-volatile memory, such as flash memory, which is pre-configured with a temporary pool containing multiple temporary pages. The system's data processing flow is designed as an asynchronous batch processing mode. The data collector, as a low-power front-end implemented based on hardware logic, autonomously receives data fragments from external sensors when the system's main processor and main RAM are in a low-power or power-off state, and appends these fragments to a current temporary page in the temporary pool. The independent batch processing accelerator, as a coprocessor independent of the main processor, remains dormant until the current temporary page accumulates. The data collector is only awakened when the amount of data meets the triggering conditions set by the control logic, performing a one-time hash calculation and compression process. This architectural design eliminates the system's static power consumption during the long data accumulation cycle. In a specific system implementation, the data collector is configured as a hardware-based finite state machine (FSM) that integrates a small-capacity SRAM FIFO buffer, such as 128 bytes, to temporarily accommodate arriving data fragments. The data collector is also responsible for maintaining the state required for system memory management, specifically maintaining a volatile page pointer and a volatile hardware accumulation counter. The volatile page pointer indicates which temporary page in the temporary pool should be written to. The control logic of the data collector is configured to follow the logic rules of a ring buffer, i.e., when the batch accelerator completes writing to a temporary page (e.g., Page...). After processing, the pointer automatically increments, allowing subsequent data fragments to be appended to the next temporary page (e.g., Page). By using all the temporary pages in the temporary pool in rotation, this storage management mechanism distributes the write and erase operations of NVM evenly across multiple physical storage blocks in the temporary pool at the logical level, thereby avoiding the problem of localized excessive wear that may occur due to the use of non-volatile memory as the temporary storage medium.

[0022] Given that the storage management state page pointer and the accumulator counter are maintained in the volatile register of the data collector, this state information is at risk of being lost during power outages, which must be considered in the fault-tolerant design of edge device systems. This could lead to data overwriting or corruption after system recovery. To address this recovery issue, the control logic of this solution is designed to execute an atomic state persistence procedure. When the data collector performs an append operation, it does not simply write data fragments, but binds the data fragments to an embedded metadata representing the current state of the volatile page pointer and the volatile hardware accumulator counter, forming an indivisible atomic state. The write unit (AWU) performs the write operation; for example, a 64-byte data fragment is combined with a 4-byte status signature encoding the current page number and the current page offset into a 68-byte AWU for a single NVM write operation. Correspondingly, when the system recovers from a power outage, the data collector's control logic is configured to first execute a recovery rule, i.e., enter scan mode, starting from the beginning of the temporary storage pool, and sequentially read and verify the validity and continuity of the embedded metadata (status signature) of each AWU. The recovery logic continues scanning until it identifies the last successfully written embedded metadata. Based on the state represented by this metadata, its internal volatile page pointers and cumulative counters are reconstructed, thereby achieving state persistence and breakpoint resumption; the batch processing accelerator, as an independent processing unit, such as a low-power Cortex-M0 microprocessor, is configured to perform core processing in its private SRAM; the capacity of this private SRAM, for example 4KB, is determined to match a temporary page size of the NVM and the optimal data block size required to process by the hash or compression hardware IP core; when the accelerator is woken up, its data flow path is determined. The process is as follows: First, the accumulated raw data (e.g., 4KB) in the current temporary page is read into its private volatile memory in one go. Then, in its private volatile memory, its internal hardware IP core is invoked to perform hash calculation and compression processing on the data block. After processing, the accelerator writes the resulting processed data (e.g., 1KB of compressed data and 32 bytes of hash value) into a dedicated archive page in non-volatile memory. After the archive write operation is confirmed, the batch processing accelerator can also be configured to send a command to erase the processed current temporary page, preparing for the next use of the circular buffer.

[0023] To adapt the batch processing triggering timing to fluctuations in data flow rates in edge applications, the data collector's decision logic is configured with a capacity-time dual triggering mechanism. To this end, in addition to integrating a volatile hardware accumulator counter, the data collector also integrates a batch processing timeout timer, a standard component. The data collector's control logic is configured to simultaneously start the batch processing timeout timer and reset the hardware accumulator counter when the first data fragment is appended to an unwritten (i.e., empty) current temporary page. The decision rule used by the data collector to generate the trigger signal is limited to an OR gate logic, i.e., when the current count value of the volatile hardware accumulator counter... Compared to the current count value of the batch processing timeout timer The following conditions must be met ( )or( When ), a trigger signal is generated; where, This is a preset batch processing threshold (e.g., 4096 bytes, corresponding to the NVM page size), and A preset time threshold (e.g., 300 seconds, corresponding to the application's data timeliness requirements); this control logic enables the system to be triggered by capacity when there is sufficient data flow. First to arrive), to achieve high power efficiency; while when the data stream is sparse, it is time-triggered ( First-come, first-served (FAR) to ensure data timeliness; in a robust implementation, the batch accelerator's control logic is further configured to monitor the execution status of its processing cores; the challenge of system error detection is that accumulated data blocks in the temporary pages may be corrupted due to sensor failure or transmission errors, causing hashing or compression processing to fail; to address this, the batch accelerator is configured to perform a bypass archiving operation when a processing failure is detected, which includes writing the accumulated unprocessed data (i.e., the original faulty data block) that caused the processing failure in the current temporary page (i.e., its private SRAM) to a dedicated error log page in non-volatile memory; importantly, after the bypass archiving operation is completed, the batch accelerator still sends a signal to the data collector to trigger the incrementing of the volatile page pointer. This mechanism ensures that the logic flow of the system's circular buffer does not get deadlocked due to processing failure, while the original data evidence for post-event diagnosis is also preserved.

[0024] In one implementation that reuses functionality, the batch processing accelerator is configured to utilize information accompanying its processing to perform data anomaly detection; the execution time of the compression algorithm... The accelerator is highly correlated with the information entropy (compressibility) of the input data; therefore, it is configured to: start its internal cycle counter (an internal component) before performing hash calculations and compression; and stop the cycle counter after processing is complete to obtain execution time. Its control logic then executes the time. With a preset processing time threshold Compare; the It can be calibrated offline to be set to the 99th percentile time required to process normal entropy data blocks, for example; and, when execution time... Exceeding the processing time threshold The logic for setting a status flag indicating data anomalies, which can be queried by the subsequently woken main processor, provides bypass awareness for fault detection with near-zero additional power consumption. In another implementation designed to optimize NVM durability, the data collector's front-end logic is further extended to handle low-entropy data streams common in edge applications, where data content remains unchanged for extended periods. To mitigate storage media wear caused by blindly appending duplicate data, the data collector also includes a historical state latch for storing previously successfully appended data fragments or their hash values, and a content comparator (a comparison logic). The data collector's control logic is configured to execute a filtering rule: upon receiving a data fragment, it first commands the content comparator to perform a hardware comparison of the data fragment's content with the content latched in the historical state latch. Its decision logic is limited to: only when the content comparator determines that the two contents are inconsistent (i.e., the data has changed) will the data fragment be processed. The data collector performs a storage operation that appends data fragments to the current temporary page and updates the historical state latch with the new content of the data fragments. When the content comparator determines that the two contents are consistent (i.e., data redundancy), the append operation to the non-volatile memory is skipped. This front-end filtering mechanism eliminates the write operation of redundant data before the data enters the storage medium, thereby further reducing write-erase losses at the storage level while maintaining low power consumption. To ensure that the above filtering mechanism does not lose information of the original data stream, the data collector may also include a redundancy counter. Its logic is configured to increment the redundancy counter when the append operation is skipped due to content consistency. Furthermore, when a subsequent append operation is performed due to inconsistency, this logic writes the current count value of the redundancy counter to the current temporary page in association with the new data fragment, for example, as its prefix, and resets the redundancy counter after the write. Correspondingly, when the batch processing accelerator performs hash calculation and compression processing subsequently, its processing logic is configured to read and parse the count value in order to reconstruct the original data stream containing all redundant copies in its private SRAM.

[0025] Example 1: In an environmental monitoring terminal deployed in a remote, high-altitude region, powered solely by small photovoltaic panels and batteries, its main processor and main volatile memory are configured to remain in deep sleep mode for the vast majority of the time. The terminal's sensors generate data in a sparse and fragmented manner, for example, generating a 64-byte data fragment every 10 minutes. Subsequent data processing tasks, such as hashing and compression, require the data to accumulate to a batch processing threshold of 4096 bytes. This allows for efficient execution. Under these conditions, if a traditional system architecture is used, maintaining the main volatile memory (RAM) powered on to temporarily store the data that has not yet accumulated will result in static power consumption becoming the main energy overhead of the system, causing the terminal to be unable to maintain operation during consecutive rainy days. In this invention, the data collector maintains low power consumption during this period, while the main processor and main volatile memory remain dormant. When a 64-byte data fragment arrives, only the data collector is awakened. The data collector performs an append operation, binding the data fragment with embedded metadata representing the current state of the volatile page pointer and the volatile hardware accumulation counter into an atomic write unit, and writing it to a currently temporary page in the non-volatile memory. This temporary page is selected according to the logical rules of the circular buffer. Subsequently, the data collector immediately goes to sleep. The system power consumption of this data accumulation process is limited to the instantaneous dynamic write power consumption of the data collector and the NVM interface, and its static maintenance power consumption is close to zero.

[0026] After running for several hours, the terminal enters a period of data depletion, with the sensor reporting rate dropping to one data fragment per hour. At this point, the capacity-time dual triggering mechanism in the control logic begins to function; although the count value of the volatile hardware cumulative counter... Far from reaching (4096 bytes), but the count value of the batch timeout timer inside the data collector. For example, at the 300th second, it was the first to reach its preset time threshold. ;this The event, through OR logic, also generates a trigger signal, waking up the dormant batch processing accelerator. The batch processing accelerator then reads the accumulated data in the current temporary page, for example, only 2KB, into its private volatile memory, performs hash calculation and compression processing, writes the processed data to an archive page, and instructs to erase the processed current temporary page before returning to sleep. Through the coordination of this power management and control logic, this system architecture achieves zero static power accumulation of data fragments while the main processor and main volatile memory are in sleep mode. Furthermore, its capacity-time dual trigger mechanism resolves the technical contradiction between system power consumption and data processing timeliness. When data is sparse, the system can prioritize data timeliness at the expense of some batch processing efficiency, while automatically switching back to high-efficiency capacity batch processing when the data flow is normal. This achieves an adaptive balance between power consumption, efficiency, and timeliness, improving the long-term operational reliability of the monitoring terminal data processing system.

[0027] Example 2: To objectively verify the low-power buffering advantage claimed by this technical solution for sparse fragmented data streams in electronic digital data processing, the following comparative experiment was conducted. The purpose of the experiment was to quantify the static power saving effect of the system architecture of this solution compared to the classic architecture during data accumulation. The test platform consisted of an experimental group and a control group. Both groups used the same main processor core and main volatile memory type, and used the same high-precision power analyzer to monitor the total system power consumption. This analyzer has the capability to measure milliwatt-level static power consumption. The control group adopted the classic architecture, in which data fragments were received by interrupts of the main processor and accumulated in the buffer of the main volatile memory, i.e., RAM. The experimental group of this invention was constructed according to the aforementioned specific implementation method, integrating a data collector based on a finite state machine and an independent batch processing accelerator. The main processor and main volatile memory were configured to a deep sleep state during accumulation, and the accumulated data was autonomously written to the NVM buffer pool by the data collector. The test environment was set to simulate two typical working conditions, with the data source simulating a sensor and sending data in 64-byte fragment units. The test parameters were set as follows: batch processing threshold... Set to 4096 bytes, this value is intended to match the NVM page size with the optimal block size for compression algorithms; batch timeout threshold. Set to 300 seconds, this value aims to balance data timeliness with batch processing power efficiency. The setting is based on ensuring processing can still be triggered even when the data rate is less than one fragment every 5 seconds. The experiment consists of two phases: Phase 1 (normal cumulative operating conditions) simulates data arriving stably at a frequency of once every 60 seconds, expected to reach the capacity threshold. Trigger batch processing; Phase 2 (data depletion condition): Simulated data arrives sparsely every 600 seconds, or 10 minutes, with an expected arrival time threshold. Batch processing was triggered; the core indicator of the test measurement was the average static power consumption of the system during the cumulative waiting period between the two phases of batch processing operations; the test process and data recording are as follows: after the two groups of devices were powered on, they began to receive data streams; in phase one (normal cumulative condition), in order to maintain the data in the temporary buffer that had not yet been filled in the main volatile memory from being lost, the control group was set to the main CPU sleep mode and the main RAM to keep in a self-refresh state; the sample group of this invention was set to the main CPU and main RAM being powered off, and only the data collector was in a low-power standby mode; in phase two (data depletion condition), the state settings of the two groups were consistent with phase one; the power analyzer measured and recorded the average static power consumption of the two groups of systems during the cumulative waiting period of the two phases, and the data summary is shown in Table 1.

[0028] Table 1: Comparison of system power consumption test data.

[0029]

[0030] Referring to the experimental data in Table 1, in Phase 1 (normal accumulation condition), the control group, in order to maintain data in RAM, had an average static power consumption of 30.5mW. However, the prototype system of this invention, due to its system architecture allowing power outages for the main CPU and main RAM, had its data accumulation process autonomously completed by the data collector, resulting in an average static power consumption of only 0.1mW, a reduction of more than two orders of magnitude. In Phase 2 (data depletion condition), the control group system, in order to wait for the arrival of sparse data, still had to continuously consume 30.5mW of RAM static power consumption, while the prototype system of this invention maintained a static power consumption of only 0.1mW in this phase. Furthermore, in Phase 2, it was observed that the batch processing accelerator of the prototype system of this invention... When the timer logic of the data collector is reached 300 seconds, it is awakened and the current (not full) data is processed, which verifies the timeliness guarantee function of the capacity-time dual triggering mechanism under the condition of data depletion. The test data objectively confirms the system architecture of this technical solution. By changing the data accumulation process from the main processor that depends on volatile memory to the way that the data collector autonomously adds data to non-volatile memory, the static power consumption of the electronic digital data processing system during the long accumulation waiting period of processing sparse and fragmented data is reduced.

[0031] Example 3: This example combines Figures 1 to 3 This section describes a low-power hash acceleration and compressed storage system for edge computing, such as... Figure 1As shown, the data first passes through a content comparator to filter redundant data. When the content is determined to be inconsistent, the data fragment and embedded metadata are atomically bound and written to the staging pool. At the same time, the system relies on a dual-trigger decision mechanism, namely, when the data volume reaches a threshold or the cumulative timeout occurs, to wake up the dormant independent batch processing accelerator. The accelerator performs hashing and compression on the cumulative data read from the staging pool in its private SRAM. The processed data is written to the archive page, while the original fault data that failed to be processed is bypassed and archived to a dedicated error log page. The process also shows that the main processor and main volatile memory remain in a low-power dormant state throughout the process, and the system state is restored by scanning the metadata in the staging pool during power failure recovery.

[0032] like Figure 2 As shown in the graph, the horizontal axis represents runtime in hours, the left vertical axis represents the number of writes, and the right vertical axis represents the percentage of redundant data. The curves show that when the percentage of redundant data remains high at 95%, the number of writes without a filtering mechanism (long dashed line) linearly increases to over 1400 times within 24 hours, while the number of writes with a filtering mechanism (solid line) remains at a very low level. This clearly demonstrates the effectiveness of the filtering mechanism in eliminating redundant writes and reducing write / erase losses. Figure 3 As shown, when a sudden power outage causes the loss of the volatile states of the page pointer and the accumulator counter, the data collector enters recovery mode upon power-up. This data collector actively scans the temporary pool in the non-volatile memory, checks the validity of each atomic write unit (AWU), and verifies the continuity of metadata until it identifies the metadata of the last successfully written data. Based on this, the data collector reconstructs the states of the page pointer and the accumulator counter, finally exits recovery mode, enters normal operating mode, and continues to receive data from the breakpoint.

[0033] Example 4: To further illustrate the advantages of this technical solution compared to the prior art, the following comparative example is set up; this comparative example aims to simulate a scenario where data is temporarily stored using a classic architecture in the prior art. It is consistent with the sample of the present invention in Example 2 in terms of hardware configuration, including the main processor core type, main volatile memory type, data source (simulating 64-byte data fragments generated by a sensor), and power consumption measurement device, i.e., a high-precision power analyzer, as well as batch processing threshold. It is also set to 4096 bytes; the core difference lies in the data accumulation method. This comparative example adopts the classic architecture, that is, using main volatile memory, i.e., RAM, as a temporary buffer. The main processor is woken up after receiving data fragments, writes the data into the RAM temporary storage area, and then the main processor enters sleep mode. However, in order to keep the data in RAM from being lost, the main volatile memory maintains a self-refresh state during this period. The test process reproduces the two stages in Example 2. Stage 1 (normal accumulation condition) data arrives at a frequency of once every 60 seconds; Stage 2 (data depletion condition) data arrives at a frequency of once every 600 seconds. The same power analyzer as in Example 2 is used to measure the average static power consumption of the system during the cumulative waiting period between batch processing operations in the two stages. It should be noted that this scenario simulates the technical reality that is common in the field of electronic digital data processing, where a temporary buffer must be introduced due to the mismatch between the algorithm processing granularity, i.e., block processing and the data source granularity, i.e., fragmentation.

[0034] The test results are recorded as follows: During the accumulation waiting period in Phase 1 (normal accumulation condition), the average static power consumption of this comparative system was measured to be 30.5mW; during the accumulation waiting period in Phase 2 (data depletion condition), due to the longer data arrival interval, the RAM needs to maintain a self-refresh state for a longer time to wait for the data to accumulate to 4096 bytes or trigger a timeout. If configured, its average static power consumption was also measured to be 30.5mW. Compared with the 0.1mW static power consumption measured in the sample group of this invention under the same conditions in Example 2, the static power consumption of this comparative system is two orders of magnitude higher. The difference in power consumption is directly at the order of magnitude above the standard. This difference stems from the fact that the classic architecture used in this comparative example must maintain RAM power supply to perform data storage functions. In edge computing scenarios dominated by low-power design, this static power consumption constitutes the main energy consumption. The experimental results show that in application scenarios that process sparse and fragmented data streams, the classic architecture that relies on main volatile memory for data storage has a power consumption disadvantage compared to the system architecture of this technical solution that uses non-volatile memory for asynchronous accumulation during the data accumulation waiting period.

[0035] Example 5: In an electrical digital data processing application scenario, this technical solution is deployed at a long-term, unattended bridge structure stress monitoring node. This node collects data via strain sensors, with data fragment sizes of 128 bytes and an average generation rate of approximately once every 5 minutes. However, short-term bursts of data flow occur due to vehicle traffic. The system requires the maximum data latency to not exceed 1 hour. The node's expected lifespan is 10 years. The non-volatile memory used, NVM, is a NAND Flash memory device with a guaranteed write / erase cycle count (P / ECycles) of 10,000 times per physical block. The temporary page size and physical block size are both 4096 bytes. Set to 4096 bytes; batch processing timeout threshold for this application requirement. The determination procedure is as follows: First, based on the application layer's requirement for maximum data latency, which is 1 hour (3600 seconds), this value constitutes... First, consider the upper limit of the system power consumption optimization and the amortization of NVM write overhead, and set a lower limit for actual operation. This can be set to a value no less than a multiple of the average data generation interval, such as 10 minutes (600 seconds). Within this upper and lower limit range, different performance levels can be evaluated through offline simulation or small-batch testing. The value was chosen by considering the combined impact of increasing the timeliness of data upload on the system's average power consumption and data upload performance, for example, by increasing the timeliness of data upload in 100-second increments from 600 seconds to 3600 seconds. In this scenario, considering the existence of occasional bursts of data flow, a value was ultimately selected to balance power consumption and low latency. The timeout is 900 seconds, which ensures that even at extremely low data rates, accumulated data will be processed within 15 minutes.

[0036] The size of the circular buffer is the number of temporary pages contained in the temporary pool. The procedure for determining this is as follows: First, calculate the total data write volume of the system over its expected 10-year lifespan. Assuming an average of 128 bytes of fragmentation is generated every 5 minutes, the average daily write volume is approximately... KB; Total write volume over 10 years is approximately MB; secondly, calculate the total amount of writing a single physical block can withstand over its lifetime, i.e. MB; therefore, to meet the 10-year lifespan requirement, the minimum number of physical blocks required, corresponding to the number of temporary pages, is the total write volume divided by the writable capacity of a single block, i.e. Considering a certain engineering margin, such as taking a 2x margin, and the fact that circular buffer designs typically use powers of 2 to facilitate control logic implementation, therefore, [the following option is chosen]. Therefore, under this storage management configuration, the system reserves eight 4096-byte temporary pages in Flash as a temporary storage pool, and the data collector's control logic will use a 3-bit volatile page pointer to cyclically write between these eight pages.

[0037] The processing time threshold used for anomaly detection is referred to here as... The calibration procedure is as follows: First, in a laboratory environment, using the target hardware platform (i.e., a system containing a batch processing accelerator), a series of representative normal data blocks are input. These data blocks should cover various expected data contents and entropy ranges, including constant value data blocks, slowly changing data blocks, and noisy data blocks within the normal operating range of the sensor. For each input data block, the execution time required for hash calculation and compression processing is precisely measured using the cycle counter inside the batch processing accelerator. Record at least 1000 valid measurement results; subsequently, analyze the collected data... The dataset is statistically analyzed to calculate its probability distribution and determine, for example, the 99th percentile, or P99; this P99 value, for example, is calculated as... ms, which is the threshold time for abnormal events. During system operation, if a certain batch processing If the time exceeds 15.2ms, the control logic sets the abnormal status flag to indicate that the data block may be abnormal, such as sensor data becoming pure random noise, resulting in low compression efficiency and extended processing time. Through the calibration and setting procedures of the above system parameters, the batch processing timeout threshold, the size of the circular buffer, and the abnormal time threshold all have clear engineering basis and reproducible determination methods.

[0038] Example 6: In an electrical digital data processing application scenario, this technical solution is deployed at a long-term, unattended bridge structure stress monitoring node. This node collects data via strain sensors, with data fragments of 128 bytes each, generated at an average rate of approximately once every 5 minutes. There are also short-term bursts of data caused by vehicle traffic. The system requires a maximum data latency of no more than 1 hour. The node's expected lifespan is 10 years. The non-volatile memory used, NVM, is a NAND Flash memory device with a guaranteed write / erase cycle count (P / ECycles) of 10,000 per physical block. The temporary page size and physical block size are both 4096 bytes. Based on this, the batch processing threshold is determined. Set to 4096 bytes; batch processing timeout threshold for this application requirement. The determination procedure is as follows: the application layer's maximum data latency requirement is 1 hour, or 3600 seconds, constituting... The upper limit is set; considering the balance between system power consumption optimization and NVM write overhead amortization, an operating lower limit is set, which can be selected as a multiple of the average data generation interval, here set to 600 seconds; within this range of 600 seconds to 3600 seconds, different... The values ​​are selected, for example, in 100-second increments, to quantify the average power consumption and data latency distribution of the system under simulated data streams, such as the 95th percentile latency, and to choose a value that can simultaneously meet the requirements of power consumption and latency. In this scenario, to balance power consumption and low latency when processing bursty data streams, a value is selected. The timeout is 900 seconds, which ensures that even at low data rates, accumulated data will be processed within 15 minutes.

[0039] The size of the circular buffer is the number of temporary pages contained in the temporary pool. The procedure is as follows: calculate the total data write volume of the system over its expected 10-year lifespan, assuming an average of 128 bytes of fragmentation every 5 minutes, resulting in a daily write volume of approximately... KB, total write volume over 10 years is approximately MB; calculates the total amount of writing a single physical block can withstand over its lifetime, i.e. MB; the minimum number of physical blocks required to meet a 10-year lifespan requirement, corresponding to the number of scratch pages, is the total write volume divided by the writable capacity per block, i.e. Block; consider setting engineering margins, for example, taking a margin of 2, and setting the buffer size to a power of 2 to facilitate control logic implementation, select The system reserves eight 4096-byte temporary pages in Flash as a temporary storage pool. The control logic of the data collector uses a 3-bit volatile page pointer to write to these eight pages in a loop.

[0040] The processing time threshold used for anomaly detection is referred to in this embodiment as... The calibration procedure is as follows: In a laboratory environment, using the target hardware platform, a series of representative normal data blocks are input. These data blocks cover a variety of expected data contents and entropy ranges, including constant values, slowly changing data, and noisy data within the normal operating range of the sensor. For each input data block, the execution time required for hash calculation and compression processing is precisely measured using the cycle counter inside the batch processing accelerator. Record at least 1000 valid measurement results; for the collected data... The dataset is subjected to statistical analysis to calculate its probability distribution and determine, for example, the value of the 99th percentile (P99); this P99 value, for example, is calculated as follows: ms, which is the threshold time for abnormal events. During system operation, if a certain batch processing If the time exceeds 15.2ms, the control logic sets the abnormal status flag to indicate that the data block may be abnormal, such as sensor data becoming pure random noise, which leads to an extension of compression processing time. Through the application of the above system parameter calibration and setting procedures, the batch processing timeout threshold, the size of the circular buffer, and the abnormal time threshold can be determined based on specific engineering requirements and device characteristics.

[0041] Example 7: In an electro-digital data processing node deployed for long-term warehouse monitoring, the monitored temperature and humidity sensor data streams exhibit low entropy characteristics; for example, temperature readings may remain at 22.5°C for several hours. The data collector will report once per minute, unchanged, based on the fundamental scheme of this invention. Each report will be 22.5... Data fragments are appended to temporary pages in non-volatile memory, which leads to unnecessary write-erase overhead under low-entropy conditions. To address this issue, the data collector in this embodiment additionally integrates a historical state latch for latching the content of previously successfully appended data fragments, along with a content comparator. When the data collector receives a new data fragment, it executes a filtering rule. First, the content comparator compares the new data fragment (e.g., 22.5) with a filtering rule. ) and the previous content latched in the historical state latch (assuming it is also 22.5). Hardware comparison is performed.

[0042] When the content comparator determines that the two contents match, the data collector skips the append operation to the non-volatile memory. The data collector also includes a redundancy counter, which is incremented when the append operation is skipped. After several hours of operation, when the sensor reports a changed data point such as 22.6... When the content comparator determines that the two contents are inconsistent, the data collector then performs an append operation, setting the current count value of the redundancy counter to 180, indicating that the previous data was repeated 180 times, and appending this new data fragment 22.6. The current temporary page is written in association, and the redundancy counter is reset after the write, while setting 22.6. The historical state latch is updated; subsequently, when the batch accelerator is awakened, its processing logic is configured to read and parse these count values, thereby reconstructing the original data stream containing all redundant copies in its private volatile memory, i.e., 180 times 22.5. And 1 time 22.6 To ensure the integrity of hash calculation and compression processing; this embodiment eliminates redundant data writing operations before the data stream enters the storage medium by implementing a fragment-level content awareness and filtering mechanism at the front end of the data collector, thereby reducing the write and erase losses of non-volatile storage media in low-entropy data stream scenarios.

[0043] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An edge computing low-power hash acceleration and compressed storage system, characterized in that, Comprising: a data collector, a batch accelerator and a non-volatile memory, the non-volatile memory having a pre-set staging pool containing a plurality of staging pages; the data collector is configured to receive data fragments and append them to a current staging page in the staging pool when the host processor and the host volatile memory are in a low power state; the data collector is further configured to maintain a volatile page pointer and a volatile hardware accumulation counter; the batch accelerator is independent of the host processor and is configured to be woken up when the volatile hardware accumulation counter tracks that the accumulated data volume of the current staging page reaches a pre-set batch threshold, to perform hash computation and compression processing on the accumulated data; the data collector is further configured to increment the volatile page pointer following the logical rule of a circular buffer to rotate different staging pages in the staging pool after the current staging page is processed; and the data collector is further configured to bind the data fragments with an embedded metadata representing the current state of the volatile page pointer and the volatile hardware accumulation counter as an atomic write unit for writing; and the data collector is configured to perform a recovery rule when the system recovers from power outage: to scan the staging pool to identify the embedded metadata of the last successful write, and to reconstruct the state of the volatile page pointer and the volatile hardware accumulation counter based on the identified embedded metadata.

2. The edge computing low-power hash acceleration and compressed storage system of claim 1, wherein, the data collector further comprises a history state latch for latching the content of previously successfully appended data fragments, and a content comparator; the data collector is further configured to first instruct the content comparator to perform hardware comparison between the content of the data fragments and the content of the previously successfully appended data fragments latched in the history state latch when receiving the data fragments; and the data collector is configured to perform a filtering rule: only when the content comparator judges that the contents are inconsistent, the operation of appending the data fragments to the current staging page is performed, and the content of the data fragments is updated to the history state latch; when the content comparator judges that the contents are consistent, the appending operation to the non-volatile memory is skipped.

3. The edge computing low power hash acceleration and compressed storage system of claim 2, wherein, the data collector further comprises a redundancy counter; the data collector is further configured to increment the redundancy counter when the appending operation is skipped; and write the count value of the redundancy counter to the current staging page in association with the data fragments and reset the redundancy counter when the appending operation is performed due to inconsistency; the batch accelerator is configured to read the count value to reconstruct the original data stream when performing the hash computation and compression processing.

4. The edge computing low power hash acceleration and compressed storage system of claim 1, wherein, the batch accelerator is further configured to detect whether there is a processing failure state during the execution of the hash computation and compression processing; and the batch accelerator is configured to perform a bypass archiving operation when the processing failure state is detected, the bypass archiving operation including: writing the unprocessed accumulated data in the current staging page that causes the processing failure to a dedicated error log page in the non-volatile memory; the batch accelerator is further configured to still send a signal to the data collector to trigger the increment of the volatile page pointer after the completion of the bypass archiving operation.

5. The edge computing low power hash acceleration and compressed storage system of claim 1, wherein, The batch accelerator comprises its private volatile memory; the batch accelerator is configured to read the accumulated data from the current staging page into its private volatile memory, and perform the hash calculation and compression processing in its private volatile memory.

6. The edge computing low power hash acceleration and compressed storage system of claim 1, wherein, The batch accelerator is further configured to start a cycle counter inside it before it performs the hash calculation and compression processing; stop the cycle counter to obtain the execution time after the processing is completed; the batch accelerator is further configured to compare the execution time with a preset processing time threshold value; And when the execution time exceeds the processing time threshold value, set a status flag indicating data anomaly, which can be queried by the host processor.

7. The edge computing low power hash acceleration and compressed storage system of claim 1, wherein, The data collector is a finite state machine implemented based on hardware logic; the data collector further comprises an SRAM FIFO; the data collector is configured to first store the received data fragments into the SRAM FIFO, and then read out from the SRAM FIFO and perform the appending operation.

8. The edge computing low power hash acceleration and compressed storage system of claim 1, wherein, The data collector also includes a batch timeout timer; the data collector is further configured to start the batch timeout timer when it begins to append data fragments to a currently unwritten temporary page; the decision rule for the data collector to generate a trigger signal is limited to: when the count value of the volatile hardware cumulative counter... The count value of the batch processing timeout timer The following conditions must be met or At that time, a trigger signal is generated; where, This is the current count value of the volatile hardware cumulative counter. This is the current count value of the batch processing timeout timer. The preset batch processing threshold, The data collector is also configured to reset the volatile hardware cumulative counter and batch timeout timer after the current temporary page has been processed.

9. The edge computing low power hash acceleration and compressed storage system of claim 1, wherein, The batch accelerator is further configured to write the post-processing data generated by the hash calculation and compression processing to an archive page of the non-volatile memory after performing the hash calculation and compression processing on the accumulated data.

10. The edge computing low power hash acceleration and compressed storage system of claim 9, wherein, The batch accelerator is further configured to send an instruction to erase the current staging page that has been processed after writing the post-processing data to the archive page.

Images