Method and system for optimizing dynamic memory management of long text intelligent processing

Abstract

The application provides a long text intelligent processing dynamic memory management optimization method and system, and relates to the field of artificial intelligence, which comprises the following steps: dividing semantic units and extracting feature vectors, constructing a dependency propagation graph to quantify the correlation between units. When the memory occupation is too high, the target unit is released according to the dependency depth and storage occupation, and the semantic offset of the target unit and the core associated unit is stored. When accessed again, the target unit is reconstructed by superimposing the offset and the anchor unit vector. The method efficiently manages long text data under limited memory, balances the processing performance and resource consumption.

Description

Technical Field

[0001] This invention relates to artificial intelligence technology, and more particularly to a dynamic memory management optimization method and system for intelligent processing of long text. Background Technology

[0002] In the field of long-text intelligent processing, such as large-scale language model inference, document summarization, or question-answering systems, efficient memory resource management is one of the core technical challenges. Current conventional approaches typically rely on static or simple dynamic memory management strategies to handle text data exceeding the length of a single computation context. One common method is to use a fixed-size sliding window mechanism, retaining only a limited length of text fragments near the current processing position in high-speed memory, while discarding historical information directly or writing it to low-speed external storage. Another common approach is cache replacement based on the least recently used principle, determining the priority of retention or eviction based on the access timestamp of data units, aiming to remove the least recently accessed units from memory.

[0003] These conventional approaches reveal significant shortcomings when processing texts with complex, long-range semantic dependencies. The sliding window mechanism, due to its fixed window size, inevitably severs semantic connections across window boundaries, causing gaps in the model's understanding of the overall text coherence and severely impacting the accuracy of tasks requiring global context. Furthermore, caching strategies based on simple timestamps consider only the single dimension of access time, completely ignoring the inherent, structured dependencies between different semantic units. This can lead to the premature and unreasonable replacement of background information that, while not frequently accessed, is crucial for subsequent key semantic reasoning. When this information is needed for later processing, the system must reload it from slow storage or generate degraded or even erroneous results due to missing information, making it difficult to strike a balance between system response latency and processing accuracy. Summary of the Invention

[0004] This invention provides a dynamic memory management optimization method and system for intelligent processing of long texts, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a dynamic memory management optimization method for intelligent processing of long texts, comprising:

[0006] The long text data to be processed is acquired and divided into semantic units to obtain multiple semantic units. Features are extracted from each semantic unit to generate a semantic representation vector.

[0007] The spatiotemporal coupling strength is calculated based on the semantic representation vector and the access time of each semantic unit, and a dependency propagation graph is constructed based on the spatiotemporal coupling strength.

[0008] When the memory space occupancy rate reaches the preset space threshold, the dependency propagation depth of each semantic unit is calculated through the dependency propagation graph, and the target release unit is determined based on the dependency propagation depth and storage occupancy.

[0009] Extract the core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units;

[0010] Release the storage area occupied by the target release unit. When the target release unit is accessed again, select the anchor unit residing in memory from the core associated units, and superimpose the semantic representation vector of the anchor unit with the semantic offset to generate the reconstructed semantic representation vector.

[0011] Based on the reconstruction of semantic representation vectors, the target release unit is recovered and long text intelligent processing tasks are performed.

[0012] The process involves acquiring long text data to be processed and dividing it into semantic units to obtain multiple semantic units. For each semantic unit, features are extracted to generate a semantic representation vector, including:

[0013] Semantic coherence analysis is performed between adjacent text paragraphs in the long text data to be processed, and the semantic coherence between each adjacent text paragraph is calculated.

[0014] Identify adjacent text paragraph pairs whose semantic coherence is below a preset coherence threshold, and mark the positions between adjacent text paragraph pairs as break points;

[0015] Based on the break points, the long text data to be processed is divided into multiple text segments, and each text segment is treated as a semantic unit.

[0016] The text content in each semantic unit is segmented to obtain a word sequence, and each word in the word sequence is embedded and encoded to generate a word vector sequence.

[0017] Calculate the alignment degree between each word vector in the word vector sequence and the overall semantic direction of the semantic unit, use the alignment degree as the contribution weight of each word vector, and perform a weighted summation of each word vector in the word vector sequence with its corresponding contribution weight to generate an initial semantic representation vector.

[0018] The cosine similarity between the initial semantic representation vector of a semantic unit and the initial semantic representation vector of adjacent semantic units is calculated and used as the semantic connection strength of the semantic unit. The initial semantic representation vector and the semantic connection strength are concatenated to generate the semantic representation vector of the semantic unit.

[0019] The spatiotemporal coupling strength is calculated based on the semantic representation vector and the access time of each semantic unit, and a dependency propagation graph is constructed based on the spatiotemporal coupling strength, including:

[0020] Calculate the cosine similarity between the semantic representation vectors of any two semantic units, and use it as the semantic association strength between the two semantic units;

[0021] Obtain the access time sequence of each semantic unit, extract the difference between the corresponding access times in the access time sequences of any two semantic units, and construct a time difference set;

[0022] Frequency statistics are performed on the set of time differences, and the time difference with the highest frequency is identified as the temporal association feature value between two semantic units. The temporal co-occurrence degree between the two semantic units is generated based on the temporal association feature value.

[0023] Semantic association strength and temporal synergy are concatenated to generate the spatiotemporal coupling strength between two semantic units.

[0024] Using each semantic unit as a node, select semantic unit pairs whose spatiotemporal coupling strength meets the preset coupling conditions, establish a directed edge between the corresponding two nodes, and assign the spatiotemporal coupling strength as the edge weight value;

[0025] Combine all nodes with directed edges to construct a dependency propagation graph.

[0026] When the memory space occupancy rate reaches a preset threshold, the dependency propagation depth of each semantic unit is calculated using the dependency propagation graph. Based on the dependency propagation depth and storage occupancy, the target release unit is determined, including:

[0027] Monitor memory space usage and trigger a memory release process when the usage reaches a preset threshold.

[0028] In the dependency propagation graph, a depth-first traversal is performed along the directed edges, starting from the node corresponding to each semantic unit, to construct a set of propagation paths originating from the starting node.

[0029] Extract different termination nodes from the propagation path set, count the number of different termination nodes, and use the number as the dependency propagation depth of the semantic unit;

[0030] In the dependency propagation graph, trace the direct predecessor node of the starting node and count the number of the direct predecessor nodes as the dependency strength of the semantic unit.

[0031] The dependency propagation depth and the strength of the dependency are fused by vector encoding to generate a dependency feature representation of the semantic unit;

[0032] Obtain the storage usage of the semantic unit, fuse the dependency feature representation with the storage usage, and generate a release priority score for the semantic unit.

[0033] A candidate release unit set is constructed by selecting semantic units whose release priority scores meet the preset release conditions. The outgoing edge connection status of the nodes corresponding to each semantic unit in the candidate release unit set is detected in the dependency propagation graph. Semantic units whose corresponding nodes have outgoing edges are removed from the candidate release unit set, and the remaining semantic units are determined as target release units.

[0034] Extract the core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units, including:

[0035] In the dependency propagation graph, locate the node corresponding to the target release unit, calculate the coupling strength from each adjacent associated node to the target release unit node, obtain the coupling strength transmission sequence, filter the adjacent associated node paths based on the transmission attenuation value in the coupling strength transmission sequence, and determine the semantic unit corresponding to the adjacent associated node path as the core associated unit.

[0036] Extract the semantic representation vector of the core associated unit, perform orthogonalization processing on the semantic representation vector to generate an orthogonal semantic basis vector set, and project the semantic representation vector of the target release unit onto the orthogonal semantic basis vector set to obtain the projection decomposition coefficients;

[0037] A mapping relationship is established between the vector index of the orthogonal semantic basis vector group and the projection decomposition coefficients to generate a semantic offset feature mapping table. The projection decomposition coefficients are sorted based on their magnitude and filtered according to a preset importance threshold.

[0038] A sparse representation vector is constructed using the filtered projection decomposition coefficients, and the sparse representation vector is concatenated with the core association unit identifier to generate a semantic offset mapping code.

[0039] The semantic offset mapping code is divided into blocks of fixed length to generate semantic offset storage blocks. The correspondence between the semantic offset storage blocks and the storage addresses is established and stored in the persistent storage area.

[0040] Release the storage area occupied by the target release unit. When the target release unit is accessed again, select the anchor unit residing in memory from the core associated units, and superimpose the semantic representation vector of the anchor unit with the semantic offset to generate the reconstructed semantic representation vector, including:

[0041] Write the memory data and memory address information of the target release unit into the persistent storage area to generate a persistent storage record. Add the persistent storage record to the memory space mapping table and release the memory space of the target release unit.

[0042] Receive the access request for the target release unit, read the persistent storage record from the memory space mapping table, and obtain the storage location of the target release unit in the persistent storage area;

[0043] Read the semantic offset mapping code of the target release unit from the storage location in the persistent storage area, and parse the semantic offset mapping code to obtain the feature vector of the target release unit and the core associated unit identifier;

[0044] Based on the node position of the target release unit in the dependency propagation graph, the adjacent nodes of the target release unit are identified, the set of semantic units residing in memory is selected from the adjacent nodes, the association strength value between each unit in the semantic unit set and the target release unit is calculated, and the semantic unit with the highest association strength value and matching the core association unit identifier is selected as the anchor unit.

[0045] Extract the semantic representation vector of the anchoring unit, and perform spatial projection operation on the semantic representation vector and the feature vector of the target release unit to generate the reconstructed semantic representation vector.

[0046] The task of recovering the target release unit based on reconstructed semantic representation vectors and performing intelligent long text processing includes:

[0047] Semantic feature sequences are extracted from the reconstructed semantic representation vectors. The cosine similarity between adjacent semantic features in the semantic feature sequences is calculated to obtain the inter-unit association strength. Based on the inter-unit association strength, the semantic feature sequences are reorganized to obtain the text semantic unit sequence.

[0048] Calculate the vector distance between each pair of semantic units in the text semantic unit sequence, cluster semantic units with vector distance less than a preset distance threshold to form sub-topic nodes, calculate the semantic similarity between sub-topic nodes, perform hierarchical clustering of sub-topic nodes based on semantic similarity to obtain parent topic nodes, construct a hierarchical semantic topic tree containing sub-topic nodes and parent topic nodes, extract the hierarchical relationship and vector distance between nodes in the semantic topic tree, and generate topic boundary vectors.

[0049] The semantic processing window is divided based on the topic boundary vector, the attention distribution between semantic units within the semantic processing window is calculated, a multi-level attention weight matrix is ​​generated, and the long text processing boundary is determined based on the multi-level attention weight matrix.

[0050] Within the boundaries of long text processing, semantic parsing is performed on the target release unit to generate text processing results. The text processing results are then written to the original memory address of the target release unit, thus completing the recovery of the target release unit and the intelligent processing of long text.

[0051] A second aspect of the present invention provides a dynamic memory management optimization system for intelligent processing of long texts, comprising:

[0052] The segmentation unit is used to acquire long text data to be processed and divide it into semantic units to obtain multiple semantic units. Features are extracted from each semantic unit to generate a semantic representation vector.

[0053] The coupling unit is used to calculate the spatiotemporal coupling strength based on the semantic representation vector and the access time of each semantic unit, and to construct the dependency propagation graph based on the spatiotemporal coupling strength.

[0054] The release unit is used to calculate the dependency propagation depth of each semantic unit through the dependency propagation graph when the memory space occupancy rate reaches a preset space threshold, and determine the target release unit based on the dependency propagation depth and storage occupancy.

[0055] Offset unit, used to extract core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units;

[0056] The reconstruction unit is used to release the storage area occupied by the target release unit. When the target release unit is accessed again, the anchor unit residing in memory is selected from the core associated units, and the semantic representation vector of the anchor unit is superimposed with the semantic offset to generate the reconstructed semantic representation vector.

[0057] The recovery unit is used to recover the target release unit based on the reconstructed semantic representation vector and perform long text intelligent processing tasks.

[0058] A third aspect of the present invention provides an electronic device, comprising:

[0059] processor;

[0060] Memory used to store processor-executable instructions;

[0061] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0062] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0063] In this embodiment, by constructing a dependency propagation graph and calculating the dependency propagation depth, the semantic units with the least impact on subsequent processing can be accurately identified and prioritized for release, thereby achieving intelligent dynamic memory reclamation when memory space is limited. This method retains the key semantic information of the released units by storing semantic offsets while releasing memory. When data is accessed again, the semantic representation vector can be quickly and accurately reconstructed using the associated units residing in memory and the stored offsets. This method enhances the adaptability of long text processing systems to complex and continuous tasks. The dynamic memory management strategy can automatically adjust the data residency state according to real-time processing progress and memory pressure, enabling the system to handle ultra-long text sequences far exceeding the physical memory capacity. By maintaining the dependency propagation graph and core associated units, the system can understand and maintain the semantic coherence between data blocks; even if some intermediate data is temporarily removed, the overall processing logic and contextual consistency will not be disrupted. This provides an efficient and reliable memory management solution for application scenarios requiring long-term memory and contextual association, such as streaming long text analysis and continuous dialogue modeling. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the dynamic memory management optimization method for intelligent long text processing according to an embodiment of the present invention.

[0065] Figure 2 This is a flowchart illustrating the semantic offset mapping encoding and storage logic of an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0068] Figure 1 This is a flowchart illustrating the dynamic memory management optimization method for intelligent long text processing according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0069] The long text data to be processed is acquired and divided into semantic units to obtain multiple semantic units. Features are extracted from each semantic unit to generate a semantic representation vector.

[0070] The spatiotemporal coupling strength is calculated based on the semantic representation vector and the access time of each semantic unit, and a dependency propagation graph is constructed based on the spatiotemporal coupling strength.

[0071] When the memory space occupancy rate reaches the preset space threshold, the dependency propagation depth of each semantic unit is calculated through the dependency propagation graph, and the target release unit is determined based on the dependency propagation depth and storage occupancy.

[0072] Extract the core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units;

[0073] Release the storage area occupied by the target release unit. When the target release unit is accessed again, select the anchor unit residing in memory from the core associated units, and superimpose the semantic representation vector of the anchor unit with the semantic offset to generate the reconstructed semantic representation vector.

[0074] Based on the reconstruction of semantic representation vectors, the target release unit is recovered and long text intelligent processing tasks are performed.

[0075] In this embodiment, long text data is first acquired and divided into multiple semantic units, generating corresponding semantic representation vectors. The spatiotemporal coupling strength is then calculated based on the access time to construct a dependency propagation graph. When memory usage reaches a threshold, a target release unit is determined based on the dependency propagation depth and storage usage. Its semantic offset relative to the core associated unit is calculated and stored, and then its memory space is released. When the target unit is accessed again, the anchor unit residing in memory is selected, and the anchor vector is superimposed with the semantic offset to generate a reconstructed semantic representation vector. This restores the target unit and allows the intelligent processing task to continue, achieving efficient long text processing with limited memory resources.

[0076] The process involves acquiring long text data to be processed and dividing it into semantic units to obtain multiple semantic units. For each semantic unit, features are extracted to generate a semantic representation vector, including:

[0077] Semantic coherence analysis is performed between adjacent text paragraphs in the long text data to be processed, and the semantic coherence between each adjacent text paragraph is calculated.

[0078] Identify adjacent text paragraph pairs whose semantic coherence is below a preset coherence threshold, and mark the positions between adjacent text paragraph pairs as break points;

[0079] Based on the break points, the long text data to be processed is divided into multiple text segments, and each text segment is treated as a semantic unit.

[0080] The text content in each semantic unit is segmented to obtain a word sequence, and each word in the word sequence is embedded and encoded to generate a word vector sequence.

[0081] Calculate the alignment degree between each word vector in the word vector sequence and the overall semantic direction of the semantic unit, use the alignment degree as the contribution weight of each word vector, and perform a weighted summation of each word vector in the word vector sequence with its corresponding contribution weight to generate an initial semantic representation vector.

[0082] The cosine similarity between the initial semantic representation vector of a semantic unit and the initial semantic representation vector of adjacent semantic units is calculated and used as the semantic connection strength of the semantic unit. The initial semantic representation vector and the semantic connection strength are concatenated to generate the semantic representation vector of the semantic unit.

[0083] When segmenting long text data into semantic units, semantic coherence is quantified by calculating the lexical overlap rate, topic consistency, and referential relevance of adjacent text paragraphs. Specifically, entity words and topic words are extracted from adjacent paragraphs, the proportion of co-occurring words between the two paragraphs is calculated, and the relationship between pronouns and their preceding referents is identified. These indicators are weighted and fused to obtain a numerical semantic coherence score, ranging from 0 to 1. When the score is below a preset coherence threshold of 0.4, it indicates a significant shift in topic or a lack of logical connection between adjacent paragraphs, and this is marked as a break point. The long text is then segmented based on all identified break points, with each segmented text fragment maintaining its internal semantic integrity as an independent semantic unit.

[0084] When segmenting semantic units, a segmentation algorithm combining dictionary and statistical methods is used to decompose the text content into a sequence of words. Each word in the sequence is embedded and encoded, mapping it to a 300-dimensional dense vector space to obtain a word vector sequence. To determine the contribution of each word to the overall semantics of the semantic unit, the overall semantic direction vector of the semantic unit is calculated, which is obtained by simply averaging the word vector sequence. The cosine value of each word vector relative to the overall semantic direction vector is calculated; the closer the cosine value is to 1, the more aligned the word is with the overall semantic direction, and this cosine value is used as the alignment degree. The alignment degree is normalized to obtain contribution weights, ensuring that the sum of the contribution weights of all words is 1. Each word vector in the word vector sequence is multiplied by its corresponding contribution weight and then summed to obtain a weighted semantic vector, which is the initial semantic representation vector of the semantic unit.

[0085] To capture the contextual relationships between semantic units, the preceding and following semantic units of the current semantic unit are extracted, and the cosine similarity between the initial semantic representation vector of the current semantic unit and the initial semantic representation vectors of the preceding and following semantic units is calculated. The forward and backward similarities are averaged to obtain the semantic cohesion strength of the current semantic unit. When the semantic unit is at the beginning of the text, only the similarity with subsequent units is calculated; when it is at the end of the text, only the similarity with the preceding unit is calculated. The initial semantic representation vector is used as the main feature, and the semantic cohesion strength is used as a scalar additional feature. After the scalar is expanded into a one-dimensional vector through vector concatenation, it is connected to the initial semantic representation vector to form an enhanced semantic representation vector with a dimension of 301. This vector contains both the internal semantic information of the semantic unit and retains the association strength features with adjacent units, serving as the basic representation for subsequent spatiotemporal coupling analysis and dependency propagation modeling.

[0086] The spatiotemporal coupling strength is calculated based on the semantic representation vector and the access time of each semantic unit, and a dependency propagation graph is constructed based on the spatiotemporal coupling strength, including:

[0087] Calculate the cosine similarity between the semantic representation vectors of any two semantic units, and use it as the semantic association strength between the two semantic units;

[0088] Obtain the access time sequence of each semantic unit, extract the difference between the corresponding access times in the access time sequences of any two semantic units, and construct a time difference set;

[0089] Frequency statistics are performed on the set of time differences, and the time difference with the highest frequency is identified as the temporal association feature value between two semantic units. The temporal co-occurrence degree between the two semantic units is generated based on the temporal association feature value.

[0090] Semantic association strength and temporal synergy are concatenated to generate the spatiotemporal coupling strength between two semantic units.

[0091] Using each semantic unit as a node, select semantic unit pairs whose spatiotemporal coupling strength meets the preset coupling conditions, establish a directed edge between the corresponding two nodes, and assign the spatiotemporal coupling strength as the edge weight value;

[0092] Combine all nodes with directed edges to construct a dependency propagation graph.

[0093] In actual implementation, for multiple generated semantic units and their semantic representation vectors, the semantic association strength is quantified by calculating the cosine similarity between any two semantic units. Specifically, for semantic unit i and semantic unit j, their semantic representation vectors v are extracted respectively. i and v jThe cosine similarity is calculated by the ratio of the vector inner product to the magnitude of each vector. This value reflects the degree of similarity between two semantic units in the semantic space. The closer the value is to 1, the stronger the semantic association.

[0094] The system synchronously acquires the access time sequence of each semantic unit during processing. This sequence records the specific timestamp of each semantic unit being called or read by the system. For any two semantic units, their access time sequences are aligned and matched, and the difference between the access times at corresponding positions is extracted one by one. For example, the time when semantic unit i is accessed for the kth time is t. {i，k} The semantic unit j is accessed for the kth time at time t. {j，k} Then calculate the difference Δt k =|t {i，k} -t {j，k} |, sum up all the differences to form a time difference set.

[0095] Frequency analysis was performed on the set of time differences, counting the number of times each time difference occurred in the set. The time difference with the highest frequency was identified as the temporal association feature value between the two semantic units. This feature value characterizes the temporal coordination pattern of the two semantic units in terms of access patterns. Based on this temporal association feature value, a negative exponential function was used to generate the temporal coordination degree. The smaller the time difference and the higher its frequency, the larger the corresponding temporal coordination degree value, reflecting the intensity of collaborative access between the two semantic units in the time dimension.

[0096] The semantic association strength and temporal synergy are concatenated to form a coupled feature vector that integrates both spatiotemporal dimensions. This feature vector is then weighted and summed, or a comprehensive score is calculated using a gating mechanism, to generate a scalar value for the spatiotemporal coupling strength between the two semantic units. This value considers both semantic similarity and temporal synergy, enabling a more accurate characterization of the dependencies between semantic units.

[0097] Using each semantic unit as a node in the graph structure, all semantic unit pairs are traversed, and pairs with spatiotemporal coupling strength exceeding a preset coupling condition are selected. The preset coupling condition can be set to a fixed threshold or employ an adaptive threshold strategy. For semantic unit pairs that meet the condition, a directed edge is established between the corresponding two nodes. The direction of the directed edge is determined based on the access sequence or dependency direction, and the calculated spatiotemporal coupling strength is assigned as the weight value of the directed edge. All nodes and the established directed edges are combined to construct a dependency propagation graph. This graph structure fully depicts the dependency propagation relationships between semantic units, providing a basis for subsequent memory management decisions.

[0098] When the memory space occupancy rate reaches a preset threshold, the dependency propagation depth of each semantic unit is calculated using the dependency propagation graph. Based on the dependency propagation depth and storage occupancy, the target release unit is determined, including:

[0099] Monitor memory space usage and trigger a memory release process when the usage reaches a preset threshold.

[0100] In the dependency propagation graph, a depth-first traversal is performed along the directed edges, starting from the node corresponding to each semantic unit, to construct a set of propagation paths originating from the starting node.

[0101] Extract different termination nodes from the propagation path set, count the number of different termination nodes, and use the number as the dependency propagation depth of the semantic unit;

[0102] In the dependency propagation graph, trace the direct predecessor node of the starting node and count the number of the direct predecessor nodes as the dependency strength of the semantic unit.

[0103] The dependency propagation depth and the strength of the dependency are fused by vector encoding to generate a dependency feature representation of the semantic unit;

[0104] Obtain the storage usage of the semantic unit, fuse the dependency feature representation with the storage usage, and generate a release priority score for the semantic unit.

[0105] A candidate release unit set is constructed by selecting semantic units whose release priority scores meet the preset release conditions. The outgoing edge connection status of the nodes corresponding to each semantic unit in the candidate release unit set is detected in the dependency propagation graph. Semantic units whose corresponding nodes have outgoing edges are removed from the candidate release unit set, and the remaining semantic units are determined as target release units.

[0106] In the process of intelligent processing of long texts, real-time monitoring of memory space occupancy is a prerequisite for dynamic memory management. By setting up a memory monitoring module, the ratio of currently allocated memory to total available memory is collected at fixed time intervals. When this ratio exceeds a preset space threshold (e.g., 85%), a memory release process is immediately initiated. The setting of the preset space threshold needs to comprehensively consider the system's response speed and memory utilization efficiency. A threshold that is too low will lead to frequent triggering of release operations, while a threshold that is too high may cause the risk of memory overflow.

[0107] In the dependency propagation graph, each semantic unit corresponds to a node, and directed edges between nodes represent semantic dependencies. When performing a depth-first traversal starting from a node and following directed edges, all reachable paths are recorded. Specifically, for a starting node A, if it points to nodes B and C, and B points to node D, then paths A→B→D and A→C can be formed from A. During the traversal, visited nodes need to be marked to avoid infinite traversal caused by circular dependencies. The number of unique terminal nodes (those with no outgoing edges) of all traversed paths is the dependency propagation depth of the semantic unit. This metric reflects the scope of influence of the current unit on subsequent processing chains; a higher dependency propagation depth indicates that the unit influences more downstream tasks.

[0108] Simultaneously, the dependency propagation graph is traced back to all direct predecessor nodes pointing to the current node. For example, if the outgoing edges of nodes E and F both point to node G, then the set of direct predecessor nodes of G is {E, F}, and the dependency strength is 2. The dependency strength reflects how frequently the current unit is referenced by upstream tasks; a higher value indicates that releasing the unit may affect the recovery of more preceding processing results.

[0109] After normalizing the dependency propagation depth and the strength of the dependent element, they are concatenated into a two-dimensional vector as the feature representation of the dependency relationship. The normalization operation uses a min-max scaling method to map the values ​​to the range of 0 to 1, avoiding the bias of indicators with different dimensions in subsequent calculations. This feature vector comprehensively characterizes the forward influence and backward constraint of semantic units in the dependency network.

[0110] The number of bytes occupied by each semantic unit is taken as the storage usage. The two-dimensional vector representing the dependency features is weighted and fused with the storage usage, and a release priority score is calculated using a linear combination method. The specific formula is: Score = Normalized value of dependency propagation depth × 0.3 + Normalized value of dependent strength × 0.4 - Normalized value of storage usage × 0.3. This design allows semantic units with low dependency propagation depth, low dependent strength, and high storage usage to receive higher release priority scores.

[0111] The release condition is set as follows: semantic units with a score greater than 0.6 are included in the candidate release unit set. Further, the outgoing edges of each node in the candidate set are examined in the dependency propagation graph. If a node has directed edges pointing to other nodes, it indicates that the semantic unit still directly supports downstream tasks and needs to be removed from the candidate set. After screening, the remaining semantic units are the final target release units. These units satisfy the release priority condition and will not directly interrupt the dependency propagation chain.

[0112] like Figure 2The diagram illustrates the semantic offset mapping encoding and storage logic flow of this embodiment, which aims to achieve efficient semantic encoding and storage. First, the target release unit is located in the dependency propagation graph, its coupling strength with adjacent nodes is calculated, and strongly correlated paths are selected based on propagation attenuation values, discarding invalid paths. The semantic vectors of core correlated units are extracted and orthogonalized to generate an orthogonal semantic basis vector set. The target unit vector is projected onto the basis vector set to obtain decomposition coefficients, which are then sorted by magnitude. A secondary selection of coefficients is performed using an importance threshold, eliminating small coefficients to retain key features and constructing a sparse representation vector. The sparse vector is concatenated with the unit identifier to generate the semantic offset mapping code. After fixed-length block segmentation and address mapping, the code is written to the persistent storage area, completing data compression and persistence.

[0113] In one optional embodiment, core associated units that have a dependency relationship with the target release unit are extracted, and the semantic offset of the target release unit relative to the core associated units is calculated and stored, including:

[0114] In the dependency propagation graph, locate the node corresponding to the target release unit, calculate the coupling strength from each adjacent associated node to the target release unit node, obtain the coupling strength transmission sequence, filter the adjacent associated node paths based on the transmission attenuation value in the coupling strength transmission sequence, and determine the semantic unit corresponding to the adjacent associated node path as the core associated unit.

[0115] Extract the semantic representation vector of the core associated unit, perform orthogonalization processing on the semantic representation vector to generate an orthogonal semantic basis vector set, and project the semantic representation vector of the target release unit onto the orthogonal semantic basis vector set to obtain the projection decomposition coefficients;

[0116] A mapping relationship is established between the vector index of the orthogonal semantic basis vector group and the projection decomposition coefficients to generate a semantic offset feature mapping table. The projection decomposition coefficients are sorted based on their magnitude and filtered according to a preset importance threshold.

[0117] A sparse representation vector is constructed using the filtered projection decomposition coefficients, and the sparse representation vector is concatenated with the core association unit identifier to generate a semantic offset mapping code.

[0118] The semantic offset mapping code is divided into blocks of fixed length to generate semantic offset storage blocks. The correspondence between the semantic offset storage blocks and the storage addresses is established and stored in the persistent storage area.

[0119] In one optional embodiment, after locating the node corresponding to the target release unit in the dependency propagation graph, all adjacent associated nodes of that node are traversed, and the coupling strength from the adjacent associated nodes to the target release unit node is calculated. This process is achieved by extracting the edge weights between the two nodes, which are determined by the aforementioned spatiotemporal coupling strength. The coupling strengths of each adjacent associated node are arranged according to the graph topology to form a coupling strength transmission sequence. This sequence reflects the strength of semantic information transmission from the associated nodes to the target node.

[0120] For the coupling strength transmission sequence, the transmission attenuation value between adjacent nodes is calculated. The transmission attenuation value is defined as the ratio of the coupling strength of the current node to the coupling strength of the previous node. The closer the attenuation value is to 1, the smaller the information loss on the transmission path. A transmission attenuation threshold of 0.75 is set. When the transmission attenuation value is lower than this threshold, the information transmission efficiency of the path is considered insufficient, and the adjacent associated nodes corresponding to the path are removed. Nodes with transmission attenuation values ​​higher than the threshold are retained, and the semantic units corresponding to these nodes are identified as core associated units.

[0121] Semantic representation vectors of the core associative units are extracted, assuming a total of K vectors are obtained, each with a dimension of D. Schmitt orthogonalization is performed on these K vectors, subtracting their projection components onto the orthogonalized vectors for each vector, followed by normalization. This results in K mutually orthogonal unit vectors, forming an orthogonal semantic basis vector set. This vector set spans a K-dimensional semantic subspace, capable of representing the semantic feature range of the core associative units.

[0122] The semantic representation vector of the target release unit is projected onto a set of orthogonal semantic basis vectors. By calculating the inner product of this vector with each orthogonal basis vector, K projection decomposition coefficients are obtained. These coefficients reflect the magnitude of the components of the target release unit in each semantic basis direction, and essentially describe the semantic offset relationship of the target release unit relative to the core association unit.

[0123] Assign a unique vector index to each vector in the orthogonal semantic basis vector set, with index numbers ranging from 1 to K. Establish a mapping relationship between the vector indices and their corresponding projection decomposition coefficients to generate a semantic offset feature mapping table. This mapping table is stored in key-value pair format, where the key is the vector index and the value is the projection decomposition coefficient. Sort the projection decomposition coefficients in descending order of their absolute values, set an importance threshold of 0.15, and filter out projection decomposition coefficients with absolute values ​​greater than this threshold, discarding coefficients with smaller values ​​to achieve sparse representation.

[0124] A sparse representation vector is constructed using the filtered projection decomposition coefficients. This vector remains K-dimensional, but only the positions corresponding to the retained coefficients are filled with non-zero values, while the remaining positions are set to zero. This sparsification process significantly reduces storage overhead while preserving the main semantic offset information. The sparse representation vector is then concatenated sequentially with the core association unit identifier, which uses a fixed-length numerical code. The concatenation results in a semantic offset mapping code.

[0125] The semantic offset mapping encoding is divided into blocks of fixed length, with each block being 64 bytes long. If the total encoding length is not a multiple of 64, zeros are padded to the end to make it a multiple of 64. Each data block after segmentation is called a semantic offset storage block, and a unique storage address is assigned to each block, which is recorded in an address index table. A mapping relationship between the target release unit identifier and the storage address is established to ensure that the storage block can be quickly located through the unit identifier later. All semantic offset storage blocks are written to a persistent storage area implemented using solid-state drives to ensure that data is not lost when power is off and to provide fast read and write speeds.

[0126] Release the storage area occupied by the target release unit. When the target release unit is accessed again, select the anchor unit residing in memory from the core associated units, and superimpose the semantic representation vector of the anchor unit with the semantic offset to generate the reconstructed semantic representation vector, including:

[0127] Write the memory data and memory address information of the target release unit into the persistent storage area to generate a persistent storage record. Add the persistent storage record to the memory space mapping table and release the memory space of the target release unit.

[0128] Receive the access request for the target release unit, read the persistent storage record from the memory space mapping table, and obtain the storage location of the target release unit in the persistent storage area;

[0129] Read the semantic offset mapping code of the target release unit from the storage location in the persistent storage area, and parse the semantic offset mapping code to obtain the feature vector of the target release unit and the core associated unit identifier;

[0130] Based on the node position of the target release unit in the dependency propagation graph, the adjacent nodes of the target release unit are identified, the set of semantic units residing in memory is selected from the adjacent nodes, the association strength value between each unit in the semantic unit set and the target release unit is calculated, and the semantic unit with the highest association strength value and matching the core association unit identifier is selected as the anchor unit.

[0131] Extract the semantic representation vector of the anchoring unit, and perform spatial projection operation on the semantic representation vector and the feature vector of the target release unit to generate the reconstructed semantic representation vector.

[0132] Once memory resources reach a space threshold, a persistence operation is performed on the target release unit. The memory data block corresponding to the target release unit is serialized and encoded, and its starting address, length, and access permission identifier in physical memory are recorded. This information constitutes a persistent storage record, which is written to a designated area of ​​the non-volatile storage medium. The persistent storage record contains a unique identifier, timestamp, and data checksum for the unit, ensuring data integrity can be verified during subsequent reads. The newly generated persistent storage record is inserted as a mapping entry into the memory space mapping table, which uses a hash index structure to maintain the correspondence between memory addresses and persistent storage locations. After updating the mapping table, the memory release interface is called to reclaim the memory space occupied by the target release unit, and this space is marked as allocatable.

[0133] When an access request for a target release unit is detected, the unit identifier in the request is parsed, and a lookup operation is performed in the memory space mapping table using this identifier as the lookup key. After obtaining the matched mapping entry, the storage location parameters of the persistent storage area, including the storage block index and offset, are extracted. The persistent storage area is accessed according to the storage location parameters, and the pre-encoded semantic offset mapping data is read. This encoding uses a compact binary format. The front end stores the feature vector of the target release unit, with the feature vector dimension consistent with the initial semantic representation vector, and each dimension component is stored in floating-point array form. The back end of the encoding stores an array of identifiers of the core associated units, with each identifier occupying a fixed byte length. A parsing operation is performed on the semantic offset mapping encoding, separating the feature vector and the core associated unit identifier list according to the predefined data format specification.

[0134] In the dependency propagation graph, locate the node corresponding to the target release unit, traverse the set of adjacent edges of that node, and obtain the unit identifiers of all adjacent nodes. For each adjacent node, query its memory residency status and determine whether the unit is still retained in physical memory by checking the memory management table. Form a candidate semantic unit set by traversing all memory-residing adjacent nodes. For each unit in the candidate set, extract its semantic representation vector, and calculate the cosine similarity between this vector and the feature vector of the target release unit as the association strength value. The association strength value is calculated by dividing the inner product of the two vectors by the product of their magnitudes. After traversing the candidate set to obtain the association strength value of each unit, sort these values ​​in descending order. From the sorted results, check whether the unit identifier appears in the core associated unit identifier list, and select the first semantic unit that simultaneously satisfies the highest association strength and identifier matching condition, and determine it as the anchor unit.

[0135] The complete semantic representation vector of the anchoring unit is read from memory, which stores the semantic feature distribution of the anchoring unit. This vector and the feature vector of the target release unit are input into the spatial projection operation module. The spatial projection operation first aligns the feature vectors in dimensions, mapping them to the same representation space as the semantic representation vector through a linear transformation matrix. Vector addition is then performed, adding each dimension component of the anchoring unit's semantic representation vector to the corresponding dimension component of the aligned feature vector one by one. The addition result is normalized to ensure that the magnitude of the generated vector conforms to a preset range. The normalized vector is the reconstructed semantic representation vector, which inherits the basic semantic information of the anchoring unit and incorporates the differentiated features of the target release unit, enabling it to replace the original semantic representation vector in subsequent processing to complete relevant computational tasks.

[0136] The task of recovering the target release unit based on reconstructed semantic representation vectors and performing intelligent long text processing includes:

[0137] Semantic feature sequences are extracted from the reconstructed semantic representation vectors. The cosine similarity between adjacent semantic features in the semantic feature sequences is calculated to obtain the inter-unit association strength. Based on the inter-unit association strength, the semantic feature sequences are reorganized to obtain the text semantic unit sequence.

[0138] Calculate the vector distance between each pair of semantic units in the text semantic unit sequence, cluster semantic units with vector distance less than a preset distance threshold to form sub-topic nodes, calculate the semantic similarity between sub-topic nodes, perform hierarchical clustering of sub-topic nodes based on semantic similarity to obtain parent topic nodes, construct a hierarchical semantic topic tree containing sub-topic nodes and parent topic nodes, extract the hierarchical relationship and vector distance between nodes in the semantic topic tree, and generate topic boundary vectors.

[0139] The semantic processing window is divided based on the topic boundary vector, the attention distribution between semantic units within the semantic processing window is calculated, a multi-level attention weight matrix is ​​generated, and the long text processing boundary is determined based on the multi-level attention weight matrix.

[0140] Within the boundaries of long text processing, semantic parsing is performed on the target release unit to generate text processing results. The text processing results are then written to the original memory address of the target release unit, thus completing the recovery of the target release unit and the intelligent processing of long text.

[0141] When the target release unit needs to re-enter the processing flow, a semantic feature sequence is extracted from the reconstructed semantic representation vector in dimensional order. This sequence reflects the semantic contribution of the target release unit in the original text. The extraction process is implemented through vector slicing, dividing the reconstructed semantic representation vector into several feature segments according to a fixed-dimensional window. For two adjacent semantic features in the sequence, the cosine similarity is calculated as a measure of the strength of the association between units. The cosine similarity is obtained by dividing the dot product of the two feature vectors by the product of their respective magnitudes, and the value ranges from negative one to positive one. When the cosine similarity of adjacent features is greater than a preset association threshold, the two features are considered to belong to the same semantically coherent region and are classified into the same text semantic unit. By traversing the entire semantic feature sequence and applying this rule, the semantic feature sequence is reorganized to generate a text semantic unit sequence.

[0142] For any two semantic units in the text semantic unit sequence, the Euclidean distance between their corresponding semantic representation vectors is calculated as the vector distance. All semantic unit pairs are traversed, and when the vector distance is less than a preset distance threshold, these semantic units are grouped into the same cluster, forming sub-topic nodes. Each sub-topic node uses the mean of the representation vectors of all semantic units within its cluster as its node representation. The cosine similarity between the representation vectors of any two sub-topic nodes is calculated as the semantic similarity between the nodes. A hierarchical clustering algorithm is used, initially treating each sub-topic node as an independent cluster, iteratively merging the two clusters with the highest semantic similarity until the inter-cluster similarity is lower than the hierarchical threshold. The merged cluster serves as the parent topic node, and its representation vector is generated by a weighted average of the representation vectors of its child nodes. By recording the parent-child relationships during the merging process, a hierarchical semantic topic tree is constructed. The difference in hierarchical depth between each pair of parent and child nodes in the tree and the Euclidean distance between their node representation vectors are extracted, and the difference in hierarchical depth and the vector distance are concatenated to form the topic boundary vector.

[0143] The topic boundary vector indicates the switching position and intensity of semantic topics. The topic boundary vector is scanned along the text sequence direction. When a local peak appears in the distance component of the vector, a boundary point of the semantic processing window is set at the corresponding text position. The text region between two adjacent boundary points constitutes a semantic processing window. Within each semantic processing window, attention scores are calculated for all semantic units pairwise. The attention score is obtained by normalizing the dot product of the query vector and the key vector using a scaling factor, and then applying the softmax function. Multiple sets of query key parameter matrices are set for different semantic levels to generate a multi-level attention weight matrix. The rows and columns of this matrix correspond to the semantic units within the window, and the matrix elements represent the attention allocation ratio between units. Analyzing the numerical distribution of the multi-level attention weight matrix, when the sum of the attention weights in a row or column is significantly lower than the mean within the window, the corresponding position is marked as a long text processing boundary, indicating that the content after that position contributes less to the current processing task.

[0144] Within the defined long text processing boundaries, semantic parsing is performed on the target release unit. Semantic parsing includes lexical analysis, syntactic analysis, and semantic role labeling, outputting structured text processing results. The processing results are serialized into a byte stream and written to the original memory address recorded in the memory management table of the target release unit. After the write operation is completed, the memory occupancy bitmap is updated to mark the storage area as occupied, and the node state of the target release unit in the dependency propagation graph is restored. At this point, the target release unit has completed the entire process from release to reconstruction and can continue to participate in subsequent long text intelligent processing tasks.

[0145] A second aspect of this invention provides a dynamic memory management optimization system for intelligent processing of long texts, the system comprising:

[0146] The segmentation unit is used to acquire long text data to be processed and divide it into semantic units to obtain multiple semantic units. Features are extracted from each semantic unit to generate a semantic representation vector.

[0147] The coupling unit is used to calculate the spatiotemporal coupling strength based on the semantic representation vector and the access time of each semantic unit, and to construct the dependency propagation graph based on the spatiotemporal coupling strength.

[0148] The release unit is used to calculate the dependency propagation depth of each semantic unit through the dependency propagation graph when the memory space occupancy rate reaches a preset space threshold, and determine the target release unit based on the dependency propagation depth and storage occupancy.

[0149] Offset unit, used to extract core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units;

[0150] The reconstruction unit is used to release the storage area occupied by the target release unit. When the target release unit is accessed again, the anchor unit residing in memory is selected from the core associated units, and the semantic representation vector of the anchor unit is superimposed with the semantic offset to generate the reconstructed semantic representation vector.

[0151] The recovery unit is used to recover the target release unit based on the reconstructed semantic representation vector and perform long text intelligent processing tasks.

[0152] A third aspect of the present invention provides an electronic device, comprising:

[0153] processor;

[0154] Memory used to store processor-executable instructions;

[0155] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0156] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0157] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic memory management optimization method for intelligent processing of long texts, characterized in that, include: The long text data to be processed is acquired and divided into semantic units to obtain multiple semantic units. Features are extracted from each semantic unit to generate a semantic representation vector. The spatiotemporal coupling strength is calculated based on the semantic representation vector and the access time of each semantic unit, and a dependency propagation graph is constructed based on the spatiotemporal coupling strength. When the memory space occupancy rate reaches the preset space threshold, the dependency propagation depth of each semantic unit is calculated through the dependency propagation graph, and the target release unit is determined based on the dependency propagation depth and storage occupancy. Extract the core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units; Release the storage area occupied by the target release unit. When the target release unit is accessed again, select the anchor unit residing in memory from the core associated units, and superimpose the semantic representation vector of the anchor unit with the semantic offset to generate the reconstructed semantic representation vector. Based on the reconstruction of semantic representation vectors, the target release unit is recovered and long text intelligent processing tasks are performed.

2. The method according to claim 1, characterized in that, The process involves acquiring long text data to be processed and dividing it into semantic units to obtain multiple semantic units. For each semantic unit, features are extracted to generate a semantic representation vector, including: Semantic coherence analysis is performed between adjacent text paragraphs in the long text data to be processed, and the semantic coherence between each adjacent text paragraph is calculated. Identify adjacent text paragraph pairs whose semantic coherence is below a preset coherence threshold, and mark the positions between adjacent text paragraph pairs as break points; Based on the break points, the long text data to be processed is divided into multiple text segments, and each text segment is treated as a semantic unit. The text content in each semantic unit is segmented to obtain a word sequence, and each word in the word sequence is embedded and encoded to generate a word vector sequence. Calculate the alignment degree between each word vector in the word vector sequence and the overall semantic direction of the semantic unit, use the alignment degree as the contribution weight of each word vector, and perform a weighted summation of each word vector in the word vector sequence with its corresponding contribution weight to generate an initial semantic representation vector. The cosine similarity between the initial semantic representation vector of a semantic unit and the initial semantic representation vector of adjacent semantic units is calculated and used as the semantic connection strength of the semantic unit. The initial semantic representation vector and the semantic connection strength are concatenated to generate the semantic representation vector of the semantic unit.

3. The method according to claim 1, characterized in that, The spatiotemporal coupling strength is calculated based on the semantic representation vector and the access time of each semantic unit, and a dependency propagation graph is constructed based on the spatiotemporal coupling strength, including: Calculate the cosine similarity between the semantic representation vectors of any two semantic units, and use it as the semantic association strength between the two semantic units; Obtain the access time sequence of each semantic unit, extract the difference between the corresponding access times in the access time sequences of any two semantic units, and construct a time difference set; Frequency statistics are performed on the set of time differences, and the time difference with the highest frequency is identified as the temporal association feature value between two semantic units. The temporal co-occurrence degree between the two semantic units is generated based on the temporal association feature value. Semantic association strength and temporal synergy are concatenated to generate the spatiotemporal coupling strength between two semantic units. Using each semantic unit as a node, select semantic unit pairs whose spatiotemporal coupling strength meets the preset coupling conditions, establish a directed edge between the corresponding two nodes, and assign the spatiotemporal coupling strength as the edge weight value; Combine all nodes with directed edges to construct a dependency propagation graph.

4. The method according to claim 1, characterized in that, When the memory space occupancy rate reaches a preset threshold, the dependency propagation depth of each semantic unit is calculated using the dependency propagation graph. Based on the dependency propagation depth and storage occupancy, the target release unit is determined, including: Monitor memory space usage and trigger a memory release process when the usage reaches a preset threshold. In the dependency propagation graph, a depth-first traversal is performed along the directed edges, starting from the node corresponding to each semantic unit, to construct a set of propagation paths originating from the starting node. Extract different termination nodes from the propagation path set, count the number of different termination nodes, and use the number as the dependency propagation depth of the semantic unit; In the dependency propagation graph, trace the direct predecessor node of the starting node and count the number of the direct predecessor nodes as the dependency strength of the semantic unit. The dependency propagation depth and the strength of the dependency are fused by vector encoding to generate a dependency feature representation of the semantic unit; Obtain the storage usage of the semantic unit, fuse the dependency feature representation with the storage usage, and generate a release priority score for the semantic unit. A candidate release unit set is constructed by selecting semantic units whose release priority scores meet the preset release conditions. The outgoing edge connection status of the nodes corresponding to each semantic unit in the candidate release unit set is detected in the dependency propagation graph. Semantic units whose corresponding nodes have outgoing edges are removed from the candidate release unit set, and the remaining semantic units are determined as target release units.

5. The method according to claim 1, characterized in that, Extract the core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units, including: In the dependency propagation graph, locate the node corresponding to the target release unit, calculate the coupling strength from each adjacent associated node to the target release unit node, obtain the coupling strength transmission sequence, filter the adjacent associated node paths based on the transmission attenuation value in the coupling strength transmission sequence, and determine the semantic unit corresponding to the adjacent associated node path as the core associated unit. Extract the semantic representation vector of the core associated unit, perform orthogonalization processing on the semantic representation vector to generate an orthogonal semantic basis vector set, and project the semantic representation vector of the target release unit onto the orthogonal semantic basis vector set to obtain the projection decomposition coefficients; A mapping relationship is established between the vector index of the orthogonal semantic basis vector group and the projection decomposition coefficients to generate a semantic offset feature mapping table. The projection decomposition coefficients are sorted based on their magnitude and filtered according to a preset importance threshold. A sparse representation vector is constructed using the filtered projection decomposition coefficients, and the sparse representation vector is concatenated with the core association unit identifier to generate a semantic offset mapping code. The semantic offset mapping code is divided into blocks of fixed length to generate semantic offset storage blocks. The correspondence between the semantic offset storage blocks and the storage addresses is established and stored in the persistent storage area.

6. The method according to claim 1, characterized in that, Release the storage area occupied by the target release unit. When the target release unit is accessed again, select the anchor unit residing in memory from the core associated units, and superimpose the semantic representation vector of the anchor unit with the semantic offset to generate the reconstructed semantic representation vector, including: Write the memory data and memory address information of the target release unit into the persistent storage area to generate a persistent storage record. Add the persistent storage record to the memory space mapping table and release the memory space of the target release unit. Receive the access request for the target release unit, read the persistent storage record from the memory space mapping table, and obtain the storage location of the target release unit in the persistent storage area; Read the semantic offset mapping code of the target release unit from the storage location in the persistent storage area, and parse the semantic offset mapping code to obtain the feature vector of the target release unit and the core associated unit identifier; Based on the node position of the target release unit in the dependency propagation graph, the adjacent nodes of the target release unit are identified, the set of semantic units residing in memory is selected from the adjacent nodes, the association strength value between each unit in the semantic unit set and the target release unit is calculated, and the semantic unit with the highest association strength value and matching the core association unit identifier is selected as the anchor unit. Extract the semantic representation vector of the anchoring unit, and perform spatial projection operation on the semantic representation vector and the feature vector of the target release unit to generate the reconstructed semantic representation vector.

7. The method according to claim 1, characterized in that, The task of recovering the target release unit based on reconstructed semantic representation vectors and performing intelligent long text processing includes: Semantic feature sequences are extracted from the reconstructed semantic representation vectors. The cosine similarity between adjacent semantic features in the semantic feature sequences is calculated to obtain the inter-unit association strength. Based on the inter-unit association strength, the semantic feature sequences are reorganized to obtain the text semantic unit sequence. Calculate the vector distance between each pair of semantic units in the text semantic unit sequence, cluster semantic units with vector distance less than a preset distance threshold to form sub-topic nodes, calculate the semantic similarity between sub-topic nodes, perform hierarchical clustering of sub-topic nodes based on semantic similarity to obtain parent topic nodes, construct a hierarchical semantic topic tree containing sub-topic nodes and parent topic nodes, extract the hierarchical relationship and vector distance between nodes in the semantic topic tree, and generate topic boundary vectors. The semantic processing window is divided based on the topic boundary vector, the attention distribution between semantic units within the semantic processing window is calculated, a multi-level attention weight matrix is ​​generated, and the long text processing boundary is determined based on the multi-level attention weight matrix. Within the boundaries of long text processing, semantic parsing is performed on the target release unit to generate text processing results. The text processing results are then written to the original memory address of the target release unit, thus completing the recovery of the target release unit and the intelligent processing of long text.

8. A dynamic memory management optimization system for intelligent long text processing, used to implement the method of any one of claims 1-7, characterized in that, include: The segmentation unit is used to acquire long text data to be processed and divide it into semantic units to obtain multiple semantic units. Features are extracted from each semantic unit to generate a semantic representation vector. The coupling unit is used to calculate the spatiotemporal coupling strength based on the semantic representation vector and the access time of each semantic unit, and to construct the dependency propagation graph based on the spatiotemporal coupling strength. The release unit is used to calculate the dependency propagation depth of each semantic unit through the dependency propagation graph when the memory space occupancy rate reaches a preset space threshold, and determine the target release unit based on the dependency propagation depth and storage occupancy. Offset unit, used to extract core related units that have a dependency relationship with the target release unit, calculate and store the semantic offset of the target release unit relative to the core related units; The reconstruction unit is used to release the storage area occupied by the target release unit. When the target release unit is accessed again, the anchor unit residing in memory is selected from the core associated units, and the semantic representation vector of the anchor unit is superimposed with the semantic offset to generate the reconstructed semantic representation vector. The recovery unit is used to recover the target release unit based on the reconstructed semantic representation vector and perform long text intelligent processing tasks.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images