A hierarchical memory storage and adaptive cyclic cleanup method and apparatus

Abstract

This invention discloses a hierarchical memory storage and adaptive cyclic cleanup method and apparatus, relating to the field of data storage management technology. The method can be implemented through pure software, pure hardware, firmware, or a combination of both. It includes: atomically writing interactive sessions into a storage domain in a chained structure, with each session record containing a unique identifier and timing information; recording session metadata in a non-volatile log area before writing; updating progress data during the writing process; marking invalid data according to the log when a writing error occurs; and triggering an adaptive cyclic cleanup process based on session priority when the remaining space in the storage domain falls below a warning threshold. The apparatus includes an atomic writing module, a power-loss protection module, and an adaptive cleanup module, which are sequentially connected in communication. This invention achieves highly reliable storage and efficient adaptive cleanup of interactive session data, reducing the risk of data corruption and accidental deletion of core data. It has wide adaptability and can cover various implementation scenarios.

Description

Technical Field

[0001] This invention relates to the field of data storage management technology, and in particular to a hierarchical memory storage and adaptive cyclic cleanup method and apparatus. Background Technology

[0002] With the rapid development of intelligent chatbots and intelligent interactive terminals, devices need to store large amounts of interactive session data to achieve functions such as context memory, historical session retrieval, and user habit learning. Existing technologies for storing interactive sessions mostly adopt a general file system storage model, which has several shortcomings: if a power outage occurs during session writing, data corruption and file system anomalies are likely to occur; when storage space is insufficient, fixed time-order cleanup rules are often used, which can easily lead to the accidental deletion of core sessions and the occupation of large amounts of storage space by non-core sessions; at the same time, existing solutions are mostly limited to a single hardware or software implementation method, making it difficult to adapt to intelligent interactive devices with different storage capacities and computing power, resulting in poor adaptability.

[0003] Meanwhile, the storage structure and cleanup rules of the existing solutions are not specifically optimized for interactive session scenarios, resulting in low efficiency in historical session retrieval, which makes it difficult to meet the real-time requirements of intelligent interactive devices. Furthermore, the scope of protection is limited and cannot fully cover different implementation scenarios. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art by proposing a hierarchical memory storage and adaptive cyclic cleaning method and apparatus, which can achieve highly reliable storage and efficient adaptive cleaning of interactive session data, reduce the risk of data corruption and accidental deletion of core data, and can be adapted to various implementation methods to cover different application scenarios of intelligent interactive devices.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A hierarchical memory storage and adaptive cyclic cleanup method can be implemented through pure software, pure hardware, firmware, or a combination of software and hardware, and includes the following steps: S1. Write the interactive session atomically to the storage field in a chain structure. Each session record contains a unique identifier and timing information. S2. Before writing to the session, record the session metadata to be written in the non-volatile log area. During the writing process, update the writing progress data synchronously. If a writing error occurs or power failure occurs, mark the written error data as invalid according to the log area record. S3. When the remaining space in the storage domain is lower than the preset or dynamically configured warning threshold, an adaptive cyclic cleanup process based on session priority is triggered.

[0006] Further, in step S1, the unique identifier of the session record includes the speaker identifier and the corresponding device hardware identifier, and each session record is strongly bound to a unique speaker and device hardware; the device hardware identifier includes one or more of the following: terminal chip ID, Bluetooth MAC address, and device unique serial number.

[0007] Further, in step S3, the adaptive cyclic cleanup process is executed sequentially from high to low priority: the first priority is to clean up sessions marked as priority cleanup due to deduplication and merging; the second priority is to clean up non-core sessions with the lowest popularity value, wherein the non-core sessions are ordinary sessions that are not marked as favorites by the user and are not system configuration sessions; the third priority is to clean up the earliest non-core sessions according to a preset or dynamically configured ratio, or to generate a prompt to notify the user to clean them up manually.

[0008] Furthermore, in the first priority cleanup, the sessions that can be cleaned up first are sorted from oldest to newest according to the last access timestamp, and the aggregated index record referencing the session is updated synchronously during cleanup.

[0009] Furthermore, in the second priority cleanup, the heat value is calculated based on the time decay factor: each time a session is accessed, the increased heat value is equal to the base value multiplied by the decay coefficient, and the decay coefficient is inversely proportional to the time elapsed since the session was created; the heat value is managed using a double-buffered heap structure, the retrieval process reads the currently active heap, the background reconstruction process builds a new heap in the spare heap, and after the construction is completed, the heap pointer is switched atomically.

[0010] Furthermore, in the second priority cleanup, the read reference count of the session is checked before the session is cleaned up. If the reference count is greater than zero, the session is skipped and the session with the second lowest popularity value is selected for cleanup.

[0011] Furthermore, in step S1, each session record also includes content semantic features, which are stored in variable length and the storage granularity is adaptively selected according to the length of the speech content; during retrieval, features of different granularities are used with different weights to participate in similarity calculation.

[0012] Furthermore, in step S2, when the device performs a self-test upon power-on next time, it scans the non-volatile log area. If an abnormal session is found where the write progress data does not match the session metadata, the abnormal data of that session is forcibly marked as invalid.

[0013] Furthermore, it also includes traversal protection steps: when traversing the session chain structure, a maximum traversal depth counter is set. If the traversal depth exceeds a preset or dynamically configured threshold, the traversal is forcibly terminated and the valid data that has been read is returned.

[0014] The present invention also provides a hierarchical memory storage and adaptive cyclic cleanup device, comprising an atomic writing module, a power-down protection module, and an adaptive cleanup module connected in sequence. The atomic writing module is used to perform the above step S1, the power-down protection module is used to perform the above step S2, and the adaptive cleanup module is used to perform the above step S3. Each module can be implemented by pure hardware logic circuits, software programs, firmware, or a combination of hardware and software.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves highly reliable storage and efficient cleanup of interactive session data through a fully closed-loop chain of chain-structured atomic writing, write-ahead log power-loss protection, and priority adaptive cyclic cleanup. It effectively reduces the risk of data corruption and power failure during the writing process, while avoiding the problem of core sessions being accidentally deleted, thus improving the reliability and rationality of storage management.

[0016] 2. The solution of this invention can be adapted to various implementation methods such as pure software, pure hardware, firmware, and a combination of software and hardware. It can cover intelligent interactive devices with different storage capacities and computing power, thus having wider adaptability and comprehensively covering different implementation scenarios, thereby expanding the application scope of the solution.

[0017] 3. This invention improves the retrieval efficiency and the rationality of cleanup rules of historical sessions through auxiliary mechanisms such as time decay-based heat value calculation, dual-buffered heap structure management, and variable-length semantic feature storage; at the same time, through a configurable parameter mechanism, it can adapt to application scenarios with different storage specifications and different performance requirements, making it more flexible. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to specific embodiments.

[0019] This embodiment provides a hierarchical memory storage and adaptive cyclic cleanup method, which can be implemented through pure software, pure hardware, firmware, or a combination of software and hardware, and is applied to conversational intelligent interactive robots. Specifically, it includes the following steps: S1. The robot atomically writes each round of human-computer interaction conversation into the local non-volatile storage field using a doubly linked list structure. Each conversation record contains the speaker's unique identifier, device hardware identifier, timestamp, conversation content data, and pointers to the adjacent conversation records before and after it.

[0020] Each conversation record also includes semantic features, which are stored in a variable-length manner: short text content is stored using sentence-level features, and long text content is stored using paragraph-level features. The storage granularity is adaptively selected based on the length of the speech content. When retrieving historical conversations, features of different granularities are used with different weights in similarity calculation.

[0021] S2. Before writing session data, record session metadata such as the session ID and the expected number of records to be written in the non-volatile log area. During the writing process, update the number of written records in the log area synchronously after each record is written. If an abnormal situation such as writing failure or device power failure occurs during the writing process, mark the incomplete abnormal data that has been written as invalid according to the records in the log area after the device recovers.

[0022] During the next power-on self-test, the device automatically scans the non-volatile log area. If it finds an abnormal session where the expected number of records to be written does not match the number of records already written, it will forcibly mark the abnormal data of that session as invalid and complete the abnormal data repair.

[0023] S3. When the remaining storage space of the storage domain is lower than the preset 20% warning threshold, an adaptive cyclic cleanup process based on session priority is triggered.

[0024] The adaptive cyclic cleanup process is executed sequentially from highest to lowest priority: First priority: Clean up sessions marked as priority cleanup due to session deduplication and merging. Priority cleanup sessions are sorted from oldest to newest by last access timestamp. During cleanup, the aggregated index record referencing the session is updated synchronously. The second priority is to clean up non-core conversations with the lowest popularity. Non-core conversations are ordinary conversations that users have not marked as favorites and are not related to system configuration. Core conversations are not included in the cleanup scope. The third priority is to clean up the earliest non-core sessions at a preset rate of 10%, or generate a prompt to notify the user to manually trigger the cleanup.

[0025] Furthermore, in the second priority cleanup, the session popularity value is calculated based on a time decay factor: each time a session is accessed, the increased popularity value equals the base value of 100 multiplied by the decay factor, which is inversely proportional to the number of months since the session was created. The popularity value is managed using a double-buffered heap structure: the session retrieval process reads the currently active main heap, while the background popularity value reconstruction process builds a new popularity value sorting heap in the backup secondary heap, and then atomically switches the heap pointer after construction. Before cleaning up a session, the session's read reference count is checked. If the reference count is greater than zero, it means the session is currently in use, so the session is skipped, and the session with the second lowest popularity value is selected for cleanup.

[0026] Optionally, this embodiment also includes a traversal protection step: when traversing the doubly linked list of the session for retrieval, a maximum traversal depth counter is set, and the maximum traversal depth is set to 1000 records; if the traversal depth exceeds the preset threshold, the traversal is forcibly terminated, and the valid data that has been read is returned to avoid system freezing caused by abnormal linked list structure.

[0027] This embodiment also provides a hierarchical memory storage and adaptive cyclic cleanup device, including an atomic writing module, a power-down protection module, and an adaptive cleanup module that are sequentially communicated with each other, wherein: The atomic write module is used to perform the session chain structure atomic write operation in step S1 above; The power-down protection module is connected in communication with the atomic writing module and is used to perform the log recording, abnormal data marking and power-on repair operations in step S2 above. The adaptive cleanup module is connected in communication with the power failure protection module and is used to perform the priority adaptive cyclic cleanup operation in step S3 above. The above modules can be implemented through storage control hardware circuits, embedded software programs, chip firmware, or a combination of hardware and software.

[0028] Those skilled in the art will understand that the above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hierarchical memory storage and adaptive cyclic cleanup method, which can be implemented through pure software, pure hardware, firmware, or a combination of software and hardware, characterized in that, Includes the following steps: S1. Write the interactive session atomically to the storage field in a chain structure. Each session record contains a unique identifier and timing information. S2. Before writing to the session, record the session metadata to be written in the non-volatile log area. During the writing process, update the writing progress data synchronously. If a writing error occurs or power failure occurs, mark the written error data as invalid according to the log area record. S3. When the remaining space in the storage domain is lower than the preset or dynamically configured warning threshold, an adaptive cyclic cleanup process based on session priority is triggered.

2. The method according to claim 1, characterized in that, In step S1, the unique identifier of the session record includes the speaker identifier and the corresponding device hardware identifier. Each session record is strongly bound to a unique speaker and device hardware. The device hardware identifier includes one or more of the following: terminal chip ID, Bluetooth MAC address, and device unique serial number.

3. The method according to claim 1, characterized in that, In step S3, the adaptive cyclic cleanup process is executed sequentially from high to low priority: first priority, clean up sessions marked as priority cleanup due to deduplication and merging; The second priority is to clean up the non-core sessions with the lowest popularity value. The non-core sessions are ordinary sessions that are not marked as favorites by the user and are not related to system configuration. The third priority is to clean up the earliest non-core sessions according to a preset or dynamically configured ratio, or to generate a prompt to notify the user to clean them up manually.

4. The method according to claim 3, characterized in that, In the first priority cleanup, sessions that can be cleaned up first are sorted from oldest to newest according to their last access timestamp, and the aggregated index records referencing the sessions are updated synchronously during cleanup.

5. The method according to claim 3, characterized in that, In the second priority cleanup, the heat value is calculated based on the time decay factor: each time a session is accessed, the increased heat value is equal to the base value multiplied by the decay coefficient, and the decay coefficient is inversely proportional to the time elapsed since the session was created; the heat value is managed using a double buffer heap structure, the retrieval process reads the currently active heap, the background reconstruction process builds a new heap in the spare heap, and after the construction is completed, the heap pointer is switched atomically.

6. The method according to claim 3, characterized in that, In the second priority cleanup, the read reference count of the session is checked before the session is cleaned up. If the reference count is greater than zero, the session is skipped and the session with the second lowest popularity value is selected for cleanup.

7. The method according to claim 1, characterized in that, In step S1, each session record also includes content semantic features, which are stored in variable length and the storage granularity is adaptively selected according to the length of the speech content; during retrieval, features of different granularities are used with different weights to participate in similarity calculation.

8. The method according to claim 1, characterized in that, In step S2, when the device performs a self-test upon power-on next time, it scans the non-volatile log area. If an abnormal session is found where the write progress data does not match the session metadata, the abnormal data of that session is forcibly marked as invalid.

9. The method according to claim 1, characterized in that, It also includes traversal protection steps: when traversing the session chain structure, a maximum traversal depth counter is set. If the traversal depth exceeds the preset or dynamically configured threshold, the traversal is forcibly terminated and the valid data that has been read is returned.

10. A hierarchical memory storage and adaptive cyclic cleaning device, characterized in that, The system includes an atomic writing module, a power-down protection module, and an adaptive cleanup module that are connected in sequence. The atomic writing module is used to execute step S1 as described in claim 1, the power-down protection module is used to execute step S2 as described in claim 1, and the adaptive cleanup module is used to execute step S3 as described in claim 1. Each module can be implemented through pure hardware logic circuits, software programs, firmware, or a combination of hardware and software.