A Semantic-Aware Fine-Grained Data Vector Generation Method and System

Abstract

This invention discloses a method and system for generating fine-grained storage data vectors based on semantic awareness. The method includes: first, acquiring the original storage data stream carrying storage time tags, storage location identifiers, and data block boundary markers; performing semantic boundary identification on continuously recorded storage data segment units to form a set of semantic boundaries within each segment; and then segmenting the data into multiple semantic atom units. Subsequently, a semantic topology graph is constructed based on the offset position markers and length attributes of the semantic atom units. Each semantic atom unit is used as a topology node, and a fine-grained semantic context vector representing both local semantic environment features and global topological position features is obtained through semantic context aggregation. Further, a granular alignment transformation is performed using the length attribute to generate a fine-grained storage data vector with a unified dimension, significantly improving the accuracy and semantic integrity of the storage data vector representation.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and more specifically, to a method and system for generating fine-grained storage data vectors based on semantic awareness. Background Technology

[0002] With the widespread application of distributed storage, object storage, log storage, and backup archiving systems, the scale of data streams generated in storage systems is constantly expanding, and the types of data content and storage structures are becoming increasingly complex. To improve data retrieval, classification, deduplication, recovery, and intelligent management capabilities, existing technologies typically represent stored data in vector form and utilize vector indexes or similarity calculations for subsequent analysis and processing.

[0003] Existing methods for generating storage data vectors mostly use fixed-length data blocks, file fragments, or record units as the basic processing objects, primarily relying on statistical characteristics, byte distribution characteristics, or overall semantic features of the data content to generate vector representations. While these methods can reflect a certain degree of data content similarity, they often neglect the true semantic boundaries within the stored data stream, the contextual relationships between fragments, and the positional relationships of data units within the storage structure. When the original stored data stream contains consecutive records, nested fields, cross-block semantic relationships, or data fragments of different lengths, traditional fixed-granularity segmentation methods are prone to semantic truncation or semantic aliasing, making it difficult for the generated vectors to accurately express the semantic connotations of fine-grained data units. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for generating fine-grained storage data vectors based on semantic awareness.

[0005] In a first aspect, embodiments of the present invention provide a method for generating fine-grained stored data vectors based on semantic awareness, the method comprising:

[0006] Obtain a raw storage data stream with storage time stamps and storage location identifiers, the raw storage data stream containing multiple consecutively recorded storage data segment units with data block boundary markers;

[0007] The original stored data stream is subjected to semantic boundary recognition processing to generate semantic boundary combinations within each stored data segment unit, and each stored data segment unit is divided into multiple semantic atom units based on the semantic boundary combinations.

[0008] A semantic topology graph is constructed based on the offset position marker and length attribute carried by the semantic atomic unit, and the semantic topology graph uses the semantic atomic unit as a topology node;

[0009] Based on the semantic topology graph, semantic context aggregation processing is performed on each topology node to generate a fine-grained semantic context vector corresponding to each semantic atom unit. The fine-grained semantic context vector includes the local semantic environment features of the semantic atom unit and the global topology position features of the semantic atom unit in the semantic topology graph.

[0010] The fine-grained semantic context vector is granularized by using the length attribute of the semantic atom unit to generate a fine-grained storage data vector with a unified dimension representation. The fine-grained storage data vector is used to describe the semantic connotation and storage structure position of the semantic atom unit in the original storage data stream.

[0011] Secondly, embodiments of the present invention provide a semantically aware fine-grained storage data vector generation system, including at least one service node;

[0012] The service node includes a storage unit and a computing unit; the storage unit is used to store program code; the computing unit is used to run the program code to execute the method described in the first aspect.

[0013] Compared to existing technologies, the beneficial effects provided by this invention include: Using the semantically aware fine-grained storage data vector generation method and system disclosed in this invention, the original storage data stream carrying storage time tags, storage location identifiers, and data block boundary markers is acquired. Semantic boundaries are identified in continuously recorded storage data segments to form a set of semantic boundaries within the segments, which are then segmented into multiple semantic atom units. Subsequently, a semantic topology graph is constructed based on the offset position markers and length attributes of the semantic atom units. Each semantic atom unit is used as a topology node, and a fine-grained semantic context vector representing both local semantic environment features and global topological position features is obtained through semantic context aggregation. Further granular alignment transformation is performed using the length attribute to generate a uniform-dimensional fine-grained storage data vector, which significantly improves the accuracy and semantic integrity of the storage data vector representation. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating the steps of a semantically aware fine-grained storage data vector generation method provided in an embodiment of the present invention;

[0016] Figure 2 This is an interactive schematic diagram of spatial augmentation vector generation provided in an embodiment of the present invention;

[0017] Figure 3 A structural block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0019] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] In order to solve the technical problems mentioned in the background art Figure 1 This is a flowchart illustrating the semantically aware fine-grained storage data vector generation method provided in this embodiment. The following is a detailed description of the semantically aware fine-grained storage data vector generation method.

[0021] Step S201: Obtain the original storage data stream with storage time stamp and storage location identifier, wherein the original storage data stream contains multiple continuously recorded storage data segment units with data block boundary markers;

[0022] Step S202: Perform semantic boundary recognition processing on the original storage data stream to generate semantic boundary combinations within each storage data segment unit, and divide each storage data segment unit into multiple semantic atom units according to the semantic boundary combinations. Each semantic atom unit carries the offset position mark of the semantic atom unit in its respective storage data segment unit and the length attribute of the semantic atom unit.

[0023] Step S203: Construct a semantic topology graph based on the offset position marker and length attribute carried by the semantic atom unit. The semantic topology graph uses the semantic atom unit as the topology node and the co-occurrence relationship of the semantic atom unit in its respective storage data segment unit as the topology edge. The connection strength of the topology edge is determined based on the proximity of the offset position marker of the semantic atom unit and the similarity of the length attribute of the semantic atom unit.

[0024] Step S204: Based on the semantic topology graph, perform semantic context aggregation processing on each topology node to generate a fine-grained semantic context vector corresponding to each semantic atom unit. The fine-grained semantic context vector includes the local semantic environment features of the semantic atom unit and the global topology position features of the semantic atom unit in the semantic topology graph.

[0025] Step S205: The fine-grained semantic context vector is subjected to granular alignment transformation using the length attribute of the semantic atom unit to generate a fine-grained storage data vector with a unified dimension representation. The fine-grained storage data vector is used to describe the semantic connotation and storage structure position of the semantic atom unit in the original storage data stream.

[0026] In this embodiment of the invention, for example, a server is deployed in the metadata management node or vector generation service node of a distributed storage system to perform fine-grained semantic parsing and vectorization of business data streams entering the storage system. The server includes a data access module, a semantic boundary recognition module, a semantic topology construction module, a context aggregation module, and a granular alignment transformation module. The server receives raw storage data streams from business systems, log systems, object storage systems, or block storage systems, and performs unified semantic-aware vector generation processing on the raw storage data streams before data writing, archiving, retrieval index construction, or cold / hot data migration.

[0027] The server first acquires the raw storage data stream, which includes storage timestamps and storage location identifiers. This raw storage data stream consists of multiple consecutively recorded storage data fragment units, each with a data block boundary marker. Taking an enterprise-level log archiving scenario as an example, the business server continuously generates access logs, exception logs, user behavior logs, and system operation logs. The storage system encapsulates these logs into consecutive data blocks according to their write time. Each data block is appended with information such as write time, disk volume number, object bucket number, shard number, and file offset address when it enters the storage system. When the server reads this data stream, it can obtain the storage timestamp for each storage data fragment unit, such as "2026-04-30 10:15:23," and the storage location identifier, such as "volume-03 / shard-12 / block-00087." The data block boundary marker indicates the start and end positions of a storage data fragment unit, allowing the server to distinguish between the physical and logical boundaries between adjacent data fragments in a continuous data stream. It should be noted that, in this specification, the semantic boundary combination refers to an ordered set of boundaries formed by arranging multiple semantic boundary markers according to their offset addresses in the stored data segment unit. Each semantic boundary marker is used to indicate the position where semantic transformation, structural field switching, description object switching, or topic context transition occurs between two adjacent semantic content units. For ease of description, the semantic boundary combination can also be called a semantic boundary set, and both have the same meaning in this specification. The semantic atomic unit refers to the smallest semantic processing unit formed by the data content between two adjacent semantic boundary markers. The smallest semantic processing unit is not limited to a single character, a single word, or a data block of fixed byte length, but refers to a data content unit that has a relatively complete semantic meaning under the current semantic boundary recognition rule and can independently participate in topology construction and vector generation. The fine-grained semantic context vector includes local semantic environment feature components and global topological position feature components. Among them, the local semantic environment feature components are used to represent the content of the semantic atomic unit itself and its neighboring semantic content, and the global topological position feature components are used to represent the connection relationship, path position, centrality, and cross-segment association state of the semantic atomic unit in the semantic topology graph.

[0028] After acquiring the raw storage data stream, the server performs semantic boundary identification processing. The server first parses the data block boundary markers, forming a time-series chain based on storage time tags and storage location identifiers. This organizes each storage data segment unit according to the writing order, storage location order, and data block boundary order. Taking a user behavior log segment as an example, a storage data segment unit continuously records content such as "user login," "visit product page," "add to cart," "payment failed," and "re-payment successful." When the server performs semantic content scanning on this segment unit, it not only identifies explicit structures such as line breaks, field separators, JSON key-value boundaries, and protocol message field boundaries, but also identifies semantic transition breakpoints. For example, the transition between "user login" and "visit product page" represents a behavior stage switch, and the transition between "payment failed" and "re-payment successful" represents a state semantic change. The server records these semantic transition locations as semantic boundary points. Multiple semantic boundary points combine to form semantic boundary combinations within the storage data segment unit. The server uses these semantic boundary combinations to divide the original segment into multiple semantic atomic units.

[0029] In one optional implementation, the server performs semantic boundary recognition processing as follows: For any stored data fragment unit, the server first reads the byte sequence, character sequence, or field sequence of the stored data fragment unit and identifies the explicit structural delimiters therein. The explicit structural delimiters include at least one of the following: newline character, field delimiter, JSON key-value boundary, XML tag boundary, protocol message field boundary, database record boundary, log template delimiter, and preset business field delimiter. The server performs a sliding scan of the stored data fragment unit using a preset scanning window. The length of the preset scanning window can be set to 64 bytes, 128 bytes, 256 bytes, or 512 bytes depending on the data type, and the window sliding step can be set to half or a quarter of the scanning window length. For the p-th scanning window, the server extracts the content tokens, field types, character categories, structural delimiter types, proportion of numeric fields, proportion of string fields, binary entropy value, and business keywords within the window to form a window feature set. The server converts the window feature set into a window semantic feature vector. Specifically, the server can use hash vectorization to map content terms to fixed-dimensional term feature vectors; it can encode field types, delimiter types, and protocol field identifiers into structural feature vectors; and it can encode the proportion of numeric fields, the proportion of string fields, window length, information entropy, and the number of fields into statistical feature vectors. After concatenating and normalizing the term feature vectors, structural feature vectors, and statistical feature vectors, the window semantic feature vector Vp of the p-th scanning window is obtained. For two adjacent scanning windows, the server calculates a semantic change score Sp. The semantic change score Sp can be calculated using the following formula: Sp = λ1 × (1 - cos(Vp, Vp+1)) + λ2 × Jp + λ3 × Rp + λ4 × Ep; where cos(Vp, Vp+1) represents the cosine similarity between the semantic feature vectors of two adjacent windows; Jp represents the Jaccard difference between the keyword set or field name set of two adjacent windows; Rp represents the explicit structural type change value between two adjacent windows, taking the larger value when there is a field boundary, record boundary, or protocol field switching between the two windows, otherwise taking the smaller value; Ep represents the information entropy change value or field length distribution change value between two adjacent windows; λ1, λ2, λ3, and λ4 are preset weight parameters, and λ1 + λ2 + λ3 + λ4 = 1. For example, λ1 can be 0.4, λ2 can be 0.2, λ3 can be 0.25, and λ4 can be 0.15. The server compares the semantic change score Sp with the preset semantic boundary threshold. When Sp is greater than or equal to the preset semantic boundary threshold, and Sp is a local maximum value among its adjacent scanning positions, the position between the p-th scanning window and the (p+1)-th scanning window is determined as the candidate semantic boundary position.The preset semantic boundary threshold can be set to a value between 0.45 and 0.75 based on the data type. For example, it can be set to 0.55 in log data scenarios, 0.60 in structured message scenarios, and 0.50 in natural language document scenarios. For boundary positions directly indicated by explicit structure delimiters, the server uses this boundary position as a candidate semantic boundary position; for boundary positions determined by semantic change scores, the server uses this boundary position as an implicit semantic boundary position. The server merges the explicit and implicit semantic boundary positions and sorts them in ascending order of offset address to form a candidate semantic boundary set. The server further performs boundary merging processing on the candidate semantic boundary set. If the distance between two candidate semantic boundary positions is less than the preset minimum atomic length, the candidate semantic boundary position with the higher semantic change score is retained, and the other candidate semantic boundary position is deleted. The preset minimum atomic length can be set to 16 bytes, 32 bytes, or determined according to the minimum field length of the business data record. If the length of the data segment formed between two adjacent candidate semantic boundary positions is greater than the preset maximum atomic length, the server will perform sliding scan and semantic change score calculation on the data segment again until the length of the resulting data segment is less than or equal to the preset maximum atomic length, or there is no position within the data segment where the semantic change score exceeds the preset semantic boundary threshold.

[0030] The server combines the start and end positions of the stored data fragment units with the candidate semantic boundary positions after merging processing to form a semantic boundary combination. Along the increasing direction of the storage address, the server treats the data content between two adjacent semantic boundary markers as a semantic atom unit, and records the start offset address, end offset address, and length attribute of this semantic atom unit. Through the above processing, the semantic boundary recognition process can determine the segmentation position based on both explicit structural boundaries and implicit semantic change boundaries, enabling the segmented semantic atom units to have reproducible generation rules.

[0031] During the segmentation process, the server extracts an offset position marker and a length attribute for each semantic atomic unit. The offset position marker indicates the relative address position of the semantic atomic unit within its respective storage data segment unit, such as starting from byte 128; the length attribute indicates the data length occupied by the semantic atomic unit, such as 64 bytes, 128 bytes, or 512 bytes. Taking document data in object storage as an example, a storage data segment unit contains content such as contract number, contract subject, payment terms, liability for breach of contract, and signing time. The server segments the "contract subject" into one semantic atomic unit and the "payment terms" into another semantic atomic unit, recording their starting offset and content length within the data segment respectively. In this way, the semantic atomic unit maintains both the semantic independence of the original data and preserves its address position relationship in the storage structure. The server also associates the storage timestamp, storage location identifier, offset position marker, and length attribute with the semantic atomic unit, so that subsequent vectors not only represent semantic content but also reflect the spatiotemporal distribution of data in the storage system.

[0032] Subsequently, the server constructs a semantic topology graph based on the offset position markers and length attributes carried by the semantic atomic units. The server initializes each semantic atomic unit as a topology node, and uses its offset position marker, length attribute, storage timestamp, and storage location identifier as auxiliary descriptive information. For multiple semantic atomic units within the same stored data segment, the server establishes semantic adjacency paths based on the order and proximity of their offset positions. Taking "user login," "access product page," "add to cart," "payment failed," and "re-payment successful" in a log segment as an example, these semantic atomic units appear consecutively in the same data segment. The server establishes topology edges based on their co-occurrence relationships and calculates the connection strength based on adjacent offset distances. The closer the offset distance, the stronger the spatial association between the two semantic atomic units within the same storage segment, and the higher the connection strength of the topology edge.

[0033] The server also adjusts the connection strength of topological edges based on the similarity of length attributes. When two semantic atomic units have similar length attributes, it indicates that they are close in data granularity and semantic expression range, and the server increases the connection strength between them. When the lengths of two semantic atomic units differ significantly, the server decreases the connection strength between them. Taking monitoring metric data as an example, the semantic atomic units "CPU utilization = 82%" and "memory utilization = 76%" have similar lengths, similar field structures, and adjacent offset positions, so the server establishes a strong topological edge for them. The "exception stack details" content is longer, while the "error code" content is shorter. Although the server still establishes a topological edge based on co-occurrence relationships, the connection strength is lower than that of nodes with similar lengths. The server calculates a comprehensive connection strength value based on the proximity of offset positions and the similarity of length attributes, prunes topological edges with excessively low connection strength, and adds connections to nodes that have semantic associations but lack direct adjacency relationships, thereby forming a stable semantic topological graph.

[0034] In one optional implementation, the server determines the topological edges and their combined connection strength values ​​in the semantic topology graph as follows: Let Ni be the topological node corresponding to any semantic atomic unit Ai, oi be the offset position marker of Ai, li be the length attribute, Li be the length of the storage data segment unit to which it belongs, bi be the identifier of the storage data segment unit, and pi be the storage location identifier. For two semantic atomic units Ai and Aj within the same storage data segment unit, if they are adjacent after being sorted according to their offset position markers, the server establishes an intra-segment semantic adjacency edge between the corresponding topological nodes Ni and Nj. For two semantic atomic units Ai and Aj in different storage data segment units, if their normalized offset position difference is less than a preset offset matching threshold, the server establishes a cross-segment offset matching edge between the corresponding topological nodes; wherein, the normalized offset position difference can be expressed as: Do(i,j)=|oi / Li-oj / Lj|. The preset offset matching threshold can be set to a value between 0.03 and 0.15. For any two semantic atomic units Ai and Aj, the server calculates the length similarity Dl(i,j), which can be expressed as: Dl(i,j) = 1 - |li - lj| / max(li,lj,1). When Dl(i,j) is greater than or equal to a preset length similarity threshold, the server establishes a length-related topological edge between the corresponding topological nodes. The preset length similarity threshold can be set to 0.6, 0.7, or 0.8. The server further calculates the comprehensive connection strength value Wij of the topological edge. For topology nodes Ni and Nj, the comprehensive connection strength value Wij can be calculated according to the following formula: Wij=α×Cij+β×exp(-Do(i,j) / σo)+γ×Dl(i,j)+δ×Pij; where, Cij represents the co-occurrence relationship factor, which is 1 when Ai and Aj belong to the same storage data segment unit or the same service record, and 0 otherwise; Do(i,j) represents the normalized offset position difference; σo represents the offset attenuation parameter; Dl(i,j) represents the length similarity; Pij represents the storage location matching factor, which is 1 when Ai and Aj have the same storage location identifier or belong to adjacent storage location fragments, and 0 otherwise; α, β, γ and δ are preset weight parameters, and α+β+γ+δ=1. For example, α can be 0.35, β can be 0.30, γ can be 0.25, and δ can be 0.10. For adjacent semantic atomic units within a segment, the server can also calculate the intra-segment proximity strength Qo(i,j) based on the original offset distance, with the formula: Qo(i,j)=exp(-|oi-oj| / max(li+lj,1)). During the calculation of the comprehensive connection strength value of semantic adjacency edges within a segment, Qo(i,j) can be used to replace or modify exp(-Do(i,j) / σo) to improve the connection strength between adjacent semantic atomic units within the same stored data segment unit.

[0035] After obtaining the overall connection strength value of all candidate topological edges, the server normalizes the overall connection strength value so that the weight of each topological edge falls between 0 and 1. The server deletes topological edges with an overall connection strength value lower than a preset connection strength threshold, which can be set to 0.2, 0.3, or 0.4. For isolated topological nodes formed after pruning, the server calculates the temporary connection score between the isolated topological node and other topological nodes, and selects the topological node with the highest temporary connection score that exceeds a preset supplementary connection threshold to establish a supplementary topological edge. The temporary connection score can be obtained using the formula for calculating the overall connection strength value. If there is no topological node exceeding the preset supplementary connection threshold, the isolated topological node is connected to the topological node with the closest offset position in the same stored data segment unit. In this way, it can be ensured that the semantic topological graph both deletes low-value weak connections and avoids key semantic atomic units from being unable to participate in context aggregation due to excessive pruning.

[0036] After the semantic topology graph is constructed, the server performs semantic context aggregation processing on each topology node based on this graph. The server determines the target aggregation topology node and its neighboring topology nodes within the semantic topology graph. Neighboring topology nodes include adjacent nodes within the same stored data segment unit as the target node, as well as other nodes connected through co-occurrence, length similarity, and offset matching relationships. When the target node is the semantic atom unit "Payment Failure," the server includes its predecessor node "Add to Cart," its successor node "Re-payment Successful," and other nodes in the data segment with the same error code or similar payment status into the neighboring domain. The server constructs structural relationship information and adjacency weight information based on the offset position relationship, length attribute relationship, and topological edge connection strength between the target node and its neighboring nodes. Neighboring nodes with closer offset distances, similar lengths, and higher connection strengths are assigned higher aggregation weights.

[0037] The server parses the semantic atomic units corresponding to the target aggregated topology node and its neighboring topology nodes, and generates initialization vectors. For text data, the server encodes field names, field values, semantic keywords, business status words, and context identifiers into initial semantic vectors. For binary protocol data, the server generates initial vectors based on protocol field types, field value ranges, field order, and structural labels. For log data, the server generates initial vectors based on log levels, event types, object identifiers, error codes, and actions. Subsequently, the server aggregates these initialization vectors according to adjacency weight information and adjusts the aggregation results using structural relationship information, so that the vector of the target semantic atomic unit can simultaneously absorb the local semantic environment and the global topology position. Taking "payment failed" as an example, the fine-grained semantic context vector generated by the server not only includes local semantic environment features such as "payment," "failure," "error code," and "order status," but also the topology position features of the node after "added to cart" and before "successful re-payment," as well as the global correlation position features of the node in multiple payment error data segments.

[0038] In one optional implementation, the server generates a fine-grained semantic context vector as follows: For a target topology node Ni, the server first extracts the content features, structural features, and storage structure features of its corresponding semantic atomic unit Ai. The content features include at least one of lexical terms, field names, field values, log levels, error codes, protocol field values, business status words, and object identifiers; the structural features include at least one of field types, delimiter types, record levels, data format types, and protocol field order; the storage structure features include at least one of offset position markers, length attributes, storage timestamps, and storage location identifiers.

[0039] The server converts content features into a content initialization vector Ci through hash vectorization, dictionary encoding, or a preset semantic encoding model, encodes structural features into a structural initialization vector Si, and encodes storage structural features into a location initialization vector Pi. Subsequently, the server concatenates and normalizes Ci, Si, and Pi to form the initialization vector Xi for the target topology node Ni, i.e., Xi = Norm([Ci,Si,Pi]); where Norm represents normalization and [ ] represents vector concatenation. Starting from the target topology node Ni in the semantic topology graph, the server extends outward along the topology edges to a preset topology hop count K, obtaining a set of neighboring topology nodes Nei(i). The preset topology hop count K can be set to 1, 2, or 3. For each neighboring topology node Nj, the server extracts the comprehensive connection strength value Wij between Ni and Nj, and extracts the structural relationship feature Rij between them. The structural relationship feature Rij includes normalized offset difference, length similarity, topology edge type, topology hop count, whether they belong to the same storage data segment unit, and whether they belong to the same storage location fragment. The server calculates the adjacency aggregation weight Aij based on the comprehensive connection strength value Wij and the structural relationship feature Rij. The adjacency aggregation weight Aij can be calculated using the following formula: Aij = exp(Wij + uT × Rij) / Σj∈Nei(i)exp(Wij + uT × Rij); where u is the structural relationship adjustment vector, and uT × Rij represents the adjustment term of the structural relationship feature on the adjacency aggregation weight. The structural relationship adjustment vector u can be obtained through training on historical samples or pre-set according to empirical rules for the field. For example, a higher positive adjustment value is set for neighboring nodes with small offset differences, high length similarity, and topological edge types that are intra-segment semantic adjacency edges. The server performs weighted aggregation on the initial vectors of the neighboring topological nodes to obtain the aggregated environment semantic vector Mi: Mi = Σj∈Nei(i)Aij × Xj. Simultaneously, the server generates a global topological position feature vector Gi based on the global structural state of the target topological node Ni in the semantic topological graph. The global topological location feature vector Gi can include the weighted degree of Ni, normalized degree centrality, PageRank value, shortest path distance to the preset anchor node, identifier of the connected component, and number of cross-segment connections. The PageRank value can be calculated iteratively as follows: PRi(r+1)=(1-d) / N+d×Σj∈In(i)PRj(r)×Wji / Σk∈Out(j)Wjk; where d is the damping coefficient, N is the total number of topological nodes, In(i) represents the set of adjacent nodes pointing to or connected to Ni, Out(j) represents the set of adjacent nodes of Nj, and Wji and Wjk are the weights of the corresponding topological edges. The damping coefficient d can be set to 0.85.The server inputs the initialization vector Xi of the target topology node, the aggregated environment semantic vector Mi, and the global topology location feature vector Gi into the context fusion function to generate a fine-grained semantic context vector Yi. The context fusion function can be expressed as: Yi = φ(Wx×Xi + Wm×Mi + Wg×Gi + b); where Wx, Wm, and Wg are dimension matching matrices, b is a bias vector, and φ is a non-linear activation function. The dimension matching matrix can be obtained through training on historical data, or it can be generated using a fixed random projection matrix, an identity expansion matrix, or an orthogonal transformation matrix. If a fixed random projection matrix is ​​used, the server generates matrix elements according to a preset random seed to ensure consistent vector results for the same input data in different runs. When generating Yi, the server defines the first part of Yi's dimensions as local semantic environment feature components and the second part of Yi's dimensions as global topology location feature components. For example, when the dimension of Yi is 192, the first 128 dimensions can be used as local semantic environment feature components, and the last 64 dimensions as global topology location feature components. Therefore, subsequent granular alignment transformation processing can clearly identify and maintain the proportional relationship between local semantic environment features and global topological location features.

[0040] After generating fine-grained semantic context vectors, the server further performs granular alignment transformation on these vectors using the length attribute of the semantic atomic units. Different semantic atomic units may have different expression scales in their initial context vectors due to variations in content length, field structure, and semantic information density. The server constructs length-granularity grouping intervals based on the length attribute distribution of all semantic atomic units, such as short field groups, medium field groups, and long text groups. Short field groups correspond to shorter data such as error codes, status codes, and timestamps; medium field groups correspond to structured content such as user behavior records and order status records; and long text groups correspond to longer content such as exception stack traces, contract terms, and audit instructions. Within each length-granularity grouping interval, the server statistically analyzes the dimensional distribution pattern of the fine-grained semantic context vectors and determines the baseline transformation dimension corresponding to that group.

[0041] For each semantic atom unit, the server calculates the offset between its length attribute and the center length of its corresponding length-granularity grouping interval, and calculates a granularity scaling adjustment factor based on this offset. Semantic atom units with lengths close to the center of the interval maintain a smaller scaling magnitude; semantic atom units with lengths significantly deviating from the center of the interval undergo more pronounced stretching or shrinking. The server uses this granularity scaling adjustment factor to perform dimensional scaling on the fine-grained semantic context vector, forming a granularity-aligned intermediate vector. Subsequently, the server constructs a granularity-alignment transformation matrix, where the row dimensions correspond to the baseline transformation dimensions, and the column dimensions correspond to the original dimensions before dimensionality scaling. The matrix elements are determined based on the granularity scaling adjustment factor and the feature distribution of each dimension of the original vector. The server uses this granularity-alignment transformation matrix to perform a linear mapping on the fine-grained semantic context vector, mapping context vectors of different lengths and granularity sources to a unified dimensional space.

[0042] During granular alignment, the server performs semantic integrity preservation processing. The server identifies the components representing local semantic environment features and the components representing global topological location features in the fine-grained semantic context vector, and maintains the relative proportion between these two parts in the granular alignment normalization vector. Taking phrase semantic atomic units like "error code" as an example, the server does not simply copy field value features during dimensional expansion, but retains its topological association proportion with surrounding nodes such as "error description," "trigger interface," and "recovery action." Taking long semantic atomic units like "exception stack details" as an example, the server retains key exception types, call chain nodes, triggering modules, and the global topological center position during dimensional contraction, avoiding the loss of core semantics after long text is compressed. The server further encodes storage timestamps and storage location identifiers into time-location description components and appends them to the end of the granular alignment normalization vector, so that the final generated fine-grained storage data vector simultaneously carries semantic information, structural location information, and storage spatiotemporal information.

[0043] Through the above processing, the server performs a uniform granularity alignment transformation on all semantic atomic units, generating a set of fine-grained storage data vectors with a unified dimensional representation. This vector set can be used for subsequent similar data retrieval, semantic index construction, hot and cold data identification, cross-shard semantic propagation, storage medium hierarchical management, and data reconstruction priority calculation. Taking a hierarchical storage system as an example, the server can determine whether a semantic atomic unit belongs to high-value, high-frequency access data based on the local semantic environment features, global topological location features, access frequency attributes, and topological centrality measures in the fine-grained storage data vectors. For semantic atomic units with high access frequency and strong topological centrality, the server can migrate their vectors to the first storage medium layer with higher access rates; for semantic atomic units with low access frequency, the server can retain them in the second storage medium layer with larger capacity. Thus, the final generated fine-grained storage data vectors not only describe the semantic connotation of semantic atomic units in the original storage data stream, but also accurately express the offset position, length granularity, topological relationship, timestamp, and storage location of the semantic atomic unit in the storage data fragment unit, providing a unified data representation foundation for semantic retrieval, data governance, shard management, and hierarchical scheduling in the storage system.

[0044] In this embodiment of the invention, the semantic boundary recognition process performed on the original stored data stream to generate semantic boundary combinations within each stored data segment unit, and the segmentation of each stored data segment unit into multiple semantic atomic units based on the semantic boundary combinations, can be implemented through the following example.

[0045] Parse the data block boundary markers of each storage data segment unit in the original storage data stream, and construct a time series chain of storage data segment units according to the order relationship of the data block boundary markers. Each node in the time series chain records the storage time tag and storage location identifier of the corresponding storage data segment unit.

[0046] Semantic content scanning is performed on each storage data segment unit in the time series chain to identify semantic transition discontinuities within the storage data segment unit. These semantic transition discontinuities include data locations in the storage data where the context topic changes and data locations in the storage data where the descriptive object changes.

[0047] A semantic boundary combination is constructed based on the location distribution of the identified semantic transition discontinuities. Each segmentation boundary marker in the semantic boundary combination corresponds to the offset address of a semantic transition discontinuity in the stored data segment unit.

[0048] The storage data segment unit is recursively segmented using the semantic boundary combination, and the data content located between two adjacent segmentation boundary markers is extracted segment by segment along the increasing direction of the storage address of the storage data segment unit to form semantic atomic units.

[0049] In the process of segmenting to form each semantic atom unit, the offset position of the starting boundary of the semantic atom unit in the storage data segment unit is extracted as the offset position mark, and the address span between the ending boundary and the starting boundary of the semantic atom unit is calculated as the length attribute.

[0050] The offset position marker and the length attribute are encapsulated with the corresponding generated semantic atom unit to establish a reference association between the offset position marker and the length attribute and the semantic atom unit;

[0051] The encapsulated semantic atom units are subjected to semantic purity screening, semantic atom units with content duplication exceeding a preset duplication threshold are removed, and the retained semantic atom units are stored in a semantic atom unit temporary storage set.

[0052] In the semantic atom unit temporary storage set, the semantic atom units are sorted and reorganized according to their offset positions to form a semantic atom unit sequence that is consistent with the original storage order of the stored data fragment units;

[0053] The semantic atom unit sequence is associated and bound with the storage time tag and storage location identifier of the storage data segment unit to generate a semantic atom unit descriptor carrying time and location attributes.

[0054] In this embodiment of the invention, the construction of a semantic topology graph based on the offset position marker and length attribute carried by the semantic atom unit can be implemented through the following example.

[0055] Obtain the set of semantic atomic units generated by the segmentation of all stored data fragments, and initialize each semantic atomic unit in the set of semantic atomic units as an independent topology node. Each topology node holds the offset position mark, length attribute, storage time tag and storage location identifier of the semantic atomic unit as node attached description information.

[0056] The topological nodes belonging to the same storage data segment unit are sorted according to the numerical value of the offset position mark. The adjacent topological nodes in the sorting result are connected by bidirectional co-occurrence topological edges to form semantic adjacency paths within the same storage data segment unit.

[0057] Cross-fragment association analysis is performed on topological nodes belonging to different storage data fragment units. Semantic atomic units with the same offset position mark in different storage data fragment units are extracted, and topological nodes corresponding to semantic atomic units with the same offset position mark are connected by cross-fragment co-occurrence topological edges.

[0058] Cross-fragment length matching is performed on topology nodes belonging to different storage data fragment units. Semantic atomic units with length attribute differences less than a preset length deviation threshold are extracted from different storage data fragment units. The topology nodes corresponding to the semantic atomic units that satisfy the length attribute difference condition are connected by length-related topology edges.

[0059] The offset position mark difference between the topological nodes connected at both ends of each topological edge is calculated, and the offset position mark difference is used as the first connection strength adjustment factor of the topological edge. The first connection strength adjustment factor is negatively correlated with the offset position mark difference.

[0060] The similarity of length attributes between the topological nodes connected at both ends of each topological edge is statistically analyzed, and the similarity of length attributes is used as the second connection strength adjustment factor of the topological edge. The second connection strength adjustment factor is negatively correlated with the relative deviation value of the length attribute.

[0061] By fusing the first connection strength adjustment factor and the second connection strength adjustment factor, a comprehensive connection strength value for each topological edge is generated, and the comprehensive connection strength value is written into the semantic topology graph as the weight description field of the topological edge.

[0062] Traverse all topological nodes in the semantic topology graph, remove all topological edges whose comprehensive connection strength value is lower than the preset connection strength threshold, and generate a semantic topology graph after edge pruning.

[0063] The semantic topology graph after edge pruning is subjected to topological connectivity analysis to identify isolated topological nodes in the graph and reconnect the isolated topological nodes with the nearest connected topological nodes by supplementing topological edges.

[0064] In an embodiment of the invention, for example, after completing the semantic atomic unit segmentation, the server obtains a set of semantic atomic units generated from all stored data fragment units and initializes each semantic atomic unit as an independent topology node. Each topology node records the offset position marker, length attribute, storage time tag, and storage location identifier of the semantic atomic unit. For example, the server extracts semantic atomic units such as "order number," "payment status," "error code," and "inventory result" from order log fragments and generates corresponding topology nodes for each.

[0065] The server sorts the topology nodes belonging to the same storage data segment unit in ascending order according to their offset position markers, and connects adjacent nodes through bidirectional co-occurring topology edges. For example, in the payment processing segment, "order amount confirmation" is located in 128 bytes, "payment channel call" is located in 384 bytes, and "payment failure return" is located in 768 bytes. The server connects these nodes sequentially to form semantic adjacency paths within the segment.

[0066] The server further performs cross-segment association analysis on the topology nodes in different stored data segments. When the "payment error code field" appears at the same offset position in multiple order log segments, the server connects these nodes through cross-segment co-occurrence topology edges, indicating that they have the same structural role in different segments. The server also performs length matching processing, connecting nodes whose length attribute difference is less than a preset length deviation threshold as length-related topology edges. For example, if multiple "error code fields" are all about 32 bytes in length, the server determines them as semantic nodes with similar granularity and establishes a connection.

[0067] The server calculates the connection strength for each topological edge. It calculates the difference in offset markers between the nodes at both ends of the edge and uses this difference as the first connection strength adjustment factor; the smaller the offset difference, the higher the connection strength. The server then calculates the similarity of the length attributes of the nodes at both ends and uses this as the second connection strength adjustment factor; the smaller the relative length deviation, the higher the connection strength. Finally, the server combines the two adjustment factors to obtain the comprehensive connection strength value of the topological edge and writes this value into the semantic topology graph as the edge weight.

[0068] After writing the edge weights, the server traverses the semantic topology graph and deletes topological edges whose overall connection strength value is lower than a preset connection strength threshold. For example, if a debugging placeholder field and an order amount field are only connected due to accidental co-occurrence, and their offset distances are large and their lengths differ significantly, the server will delete this weak connection. After pruning, the server performs topology connectivity analysis to identify isolated topological nodes and reconnects these isolated nodes with the nearest connected node in terms of offset distance and length attribute by adding topological edges. For example, after the "Inventory Status Field" is pruned and becomes an isolated node, the server will add it to the nearest "Inventory Rollback Result Field" within the same segment. Thus, the server generates a semantic topology graph that retains valid semantic relationships while maintaining overall connectivity.

[0069] In this embodiment of the invention, the step of performing semantic context aggregation processing on each topology node based on the semantic topology graph to generate a fine-grained semantic context vector corresponding to each semantic atom unit can be implemented through the following example.

[0070] In the semantic topology graph, a topology node is selected as the target aggregation topology node. Starting from the target aggregation topology node, the topology is expanded outward along the topology edge to a preset topology hop count range. All topology nodes reachable within the preset topology hop count range are marked as neighboring domain topology nodes.

[0071] The offset position markers and length attributes of the semantic atomic units corresponding to the target aggregated topology node are collected, and the offset position markers and length attributes of the semantic atomic units corresponding to each neighboring domain topology node are also collected to construct a structural relationship matrix between the target aggregated topology node and the neighboring domain topology nodes.

[0072] Extract the comprehensive connection strength value of the topological edges between the target aggregated topological node and each neighboring domain topological node, and generate an adjacency weight vector with the comprehensive connection strength value as the element. The dimension of the adjacency weight vector is consistent with the number of neighboring domain topological nodes.

[0073] The semantic atom units corresponding to the target aggregation topology node are subjected to content parsing processing to extract the local semantic description information inside the semantic atom units corresponding to the target aggregation topology node, and the local semantic description information is converted into a local semantic initialization vector.

[0074] Content parsing processing is performed on the semantic atom unit corresponding to each neighboring domain topology node to extract the environmental semantic description information inside the semantic atom unit corresponding to each neighboring domain topology node, and the environmental semantic description information is converted into an environmental semantic initialization vector;

[0075] The environmental semantic initialization vector is weighted and aggregated using the adjacency weight vector. The environmental semantic initialization vector corresponding to each neighboring topological node is multiplied by the comprehensive connection strength value of the corresponding topological edge and then summed to generate an aggregated environmental semantic vector.

[0076] Based on the structural relationship matrix, the aggregated environment semantic vector is subjected to structure-aware filtering. The relative difference information of offset position markers and length attributes reflected in the structural relationship matrix is ​​used to adjust the response intensity of each dimension in the aggregated environment semantic vector, thereby generating a structure-aware aggregated environment semantic vector.

[0077] The local semantic initialization vector and the structure-aware aggregated environment semantic vector are concatenated dimensionally to generate a fine-grained semantic context vector containing local semantic environment features and global topological location features.

[0078] The fine-grained semantic context vector is associated with the semantic atom unit corresponding to the target aggregated topology node and stored, and the same semantic context aggregation processing operation is performed on each topology node in the semantic topology graph.

[0079] In this embodiment of the invention, for example, the server selects topology nodes one by one as target aggregation topology nodes in the constructed semantic topology graph. When processing order log data, the server takes the "payment failure return" node as the current target aggregation topology node, and expands outward along the topology edge from this node to a preset topology hop range. For example, the server expands the range by two hops: the first hop obtains adjacent nodes such as "payment channel call" and "failure reason description", and the second hop obtains indirectly related nodes such as "order amount confirmation" and "inventory rollback result", and marks these reachable nodes as neighboring domain topology nodes.

[0080] The server collects the offset position markers and length attributes of the semantic atomic units corresponding to the target aggregate topology node, and simultaneously collects the offset position markers and length attributes of each neighboring topology node, constructing a structural relationship matrix. For example, "Payment failure return" is located in 768 bytes with a length of 128 bytes, "Failure reason description" is located in 896 bytes with a length of 224 bytes, and "Payment channel call" is located in 384 bytes with a length of 96 bytes. The server writes the offset difference, length difference, and relative position relationship between the target node and each neighboring node into the structural relationship matrix to express the context position of the target node in the storage structure.

[0081] The server further extracts the comprehensive connection strength value of the topological edges between the target aggregation topology node and each neighboring domain topology node, generating an adjacency weight vector. For example, "Payment Failure Return" has a close offset distance and strong business correlation with "Failure Reason Description," resulting in a higher weight; while its indirect correlation with "Order Amount Confirmation" is weaker, leading to a lower weight. The dimension of the adjacency weight vector is consistent with the number of neighboring domain topology nodes, ensuring that each neighboring node has a corresponding weight in subsequent aggregations.

[0082] Subsequently, the server performs content parsing on the semantic atomic units corresponding to the target aggregated topology nodes. When parsing "payment failure return", the server extracts local semantic description information such as "payment", "failure", "return code", and "channel response", and converts this information into local semantic initialization vectors. The server also performs content parsing on each neighboring domain topology node to extract environmental semantic description information. For example, it extracts "channel number" and "call interface" from "payment channel call", "insufficient balance", "timeout", and "risk control rejection" from "failure reason description", and "inventory recovery" and "status write-back" from "inventory rollback result", and converts them into environmental semantic initialization vectors respectively.

[0083] The server uses adjacency weight vectors to perform weighted aggregation of the environmental semantic initialization vector. The server multiplies the environmental semantic initialization vector of each neighboring topological node by the comprehensive connection strength value of the corresponding topological edge and then sums the results to generate the aggregated environmental semantic vector. Therefore, strongly related nodes such as "failure reason description" contribute more to the aggregation result, while weakly related nodes such as "order amount confirmation" contribute less, enabling the aggregation result to highlight the contextual information most relevant to the target node.

[0084] The server performs structure-aware filtering on the aggregated environment semantic vector based on the structure relation matrix. The offset position differences and length attribute differences recorded in the structure relation matrix are used to adjust the response intensity of each dimension of the aggregated environment semantic vector. For neighboring nodes with similar offset positions and lengths, the server enhances the response of their corresponding semantic dimensions; for nodes that are far away or have significant length differences, the server reduces their semantic impact. After filtering, the server generates a structure-aware aggregated environment semantic vector, ensuring that the environment semantics reflect not only content similarity but also proximity relationships in the storage structure.

[0085] Finally, the server concatenates the local semantic initialization vector of the target node with the structure-aware aggregated environment semantic vector to generate a fine-grained semantic context vector. This vector contains both the local semantic environment features of "payment failure return" itself and its global topological position features in the semantic topology graph. The server associates and stores the generated fine-grained semantic context vector with the target semantic atomic unit, and continues to traverse each topological node in the semantic topology graph, performing the same aggregation process on nodes such as "order number," "payment channel call," and "inventory rollback result," ultimately obtaining a set of fine-grained semantic context vectors corresponding to all semantic atomic units.

[0086] In this embodiment of the invention, the step of using the length attribute of the semantic atom unit to perform granular alignment transformation on the fine-grained semantic context vector to generate a fine-grained storage data vector with a unified dimensional representation can be implemented through the following example.

[0087] Obtain the fine-grained semantic context vector and the length attribute of each semantic atom unit. Construct a length granularity grouping interval based on the numerical distribution range of the length attribute. The length granularity grouping interval will group semantic atom units with similar length attributes into the same granularity group.

[0088] Within each length granularity grouping interval, the dimensional distribution pattern of the fine-grained semantic context vectors of all semantic atom units in that group is statistically analyzed, and the baseline transformation dimension corresponding to that length granularity grouping interval is determined based on the dimensional distribution pattern.

[0089] For each semantic atom unit, a granularity scaling adjustment factor is calculated based on the offset between the length attribute of the semantic atom unit and the center length of the interval to which it belongs in the length granularity grouping interval. The granularity scaling adjustment factor changes in correlation with the offset.

[0090] The fine-grained semantic context vector of the semantic atom unit is scaled using the granularity scaling adjustment factor. The dimension of the fine-grained semantic context vector is stretched or shrunk to the reference transformation dimension corresponding to the length granularity grouping interval, so as to obtain the granularity aligned intermediate vector.

[0091] Construct a granularity alignment transformation matrix, wherein the row dimension of the granularity alignment transformation matrix is ​​equal to the baseline transformation dimension, and the column dimension is equal to the original dimension of the fine-grained semantic context vector before performing dimension scaling. The elements of the granularity alignment transformation matrix are determined based on the granularity scaling adjustment factor and the feature distribution on the original dimension of the fine-grained semantic context vector.

[0092] The fine-grained semantic context vector is linearly mapped using the granularity alignment transformation matrix, mapping the fine-grained semantic context vector from the original dimension space to the baseline transformation dimension space, thereby generating a granularity alignment normalized vector.

[0093] In the process of generating granular alignment normalized vectors, semantic integrity preservation processing is performed on the granular alignment normalized vectors. The component parts representing local semantic environment features and the component parts representing global topological location features in the fine-grained semantic context vectors are extracted, and the relative proportional relationship between the two components is preserved in the granular alignment normalized vectors.

[0094] The granularity alignment normalization vector is fused with the storage time tag and storage location identifier of the semantic atom unit. The storage time tag and storage location identifier are encoded as time location description components and appended to the tail of the granularity alignment normalization vector to generate a fine-grained storage data vector carrying time location information.

[0095] Perform the same granularity alignment transformation operation on all semantic atomic units to generate a set of fine-grained storage data vectors with a unified dimensional representation corresponding to all the semantic atomic units.

[0096] In this embodiment of the invention, for example, after generating the fine-grained semantic context vector, the server obtains the fine-grained semantic context vector and its length attribute corresponding to each semantic atom unit. The server statistically analyzes the length attributes of all semantic atom units and constructs length granularity grouping intervals according to the length value distribution. For example, in the order log storage scenario, fields such as "error code" and "status code" have lengths concentrated between 16 bytes and 64 bytes, and the server classifies them into a short-granularity group; fields such as "payment channel return summary" and "order amount calculation result" have lengths concentrated between 128 bytes and 512 bytes, and the server classifies them into a medium-granularity group; content such as "exception stack details" and "risk control rejection explanation" have lengths exceeding 512 bytes, and the server classifies them into a long-granularity group. Through this processing, semantic atom units with similar lengths and similar semantic carrying granularity are divided into the same granularity group.

[0097] The server statistically analyzes the dimensionality distribution pattern of the fine-grained semantic context vectors within each granularity group. For short-granularity groups, the server finds that their vectors primarily express field values, state types, and structural positions, with low dimensionality requirements. For medium-granularity groups, the vectors simultaneously express business actions, field relationships, and adjacent context, with moderate dimensionality requirements. For long-granularity groups, the vectors need to express multiple layers of semantic content and topological positional relationships, resulting in high dimensionality requirements. Based on this, the server determines the corresponding baseline transformation dimension for each granularity group, enabling the vectors of semantic atomic units within the same group to be mapped to a unified expression space.

[0098] For each semantic atom unit, the server calculates the offset between its length attribute and the center length of its corresponding length-granularity group interval, and then calculates a granularity scaling adjustment factor based on this offset. For example, the center length of a short granularity group is 32 bytes, and the "error code" unit is exactly 32 bytes long, so the server generates a scaling adjustment factor close to the standard value; the "status description" unit is 60 bytes long, and the server generates a larger adjustment factor based on its deviation from the center length. This adjustment factor is used to control the magnitude of subsequent vector dimension stretching or shrinking.

[0099] The server uses a granular scaling adjustment factor to perform dimensional scaling on the fine-grained semantic context vector. For the shorter but semantically important "payment error code" unit, the server enhances its error category, payment status, and topological location-related components when stretching the dimension, ensuring that the short field is not insufficiently expressive despite its limited content. For the longer "exception stack details" unit, the server retains the exception type, call chain nodes, triggering module, and adjacent semantic relationships when shrinking the dimension, while removing low-value responses such as duplicate stack frames, resulting in a granular-aligned intermediate vector.

[0100] Subsequently, the server constructs a granularity alignment transformation matrix. The row dimension of this matrix equals the baseline transformation dimension of the corresponding granularity group, and the column dimension equals the original dimension of the fine-grained semantic context vector. The server determines the matrix elements based on the granularity scaling adjustment factor and the feature distribution of each dimension of the original vector. For example, dimensions with a higher concentration of local semantic features receive higher mapping weights, while noisy or repetitive semantic components receive lower mapping weights. The server uses this granularity alignment transformation matrix to linearly map the original fine-grained semantic context vector from its original dimension space to the baseline transformation dimension space, generating a granularity-aligned normalized vector.

[0101] During the generation of granular alignment normalized vectors, the server performs semantic integrity preservation processing. The server extracts the components representing local semantic environment features and the components representing global topological location features from the original vector, preserving their relative proportions in the normalized vector. For example, the local semantics of the "payment failure return" unit include the failure status, return code, and reason summary, while the global topological location includes its position after the payment call and before the inventory rollback. The server retains both types of information after transformation, avoiding the loss of structural relationships by only retaining field content.

[0102] Finally, the server fuses the granular alignment normalized vector with the storage timestamps and storage location identifiers of the semantic atomic units. The server encodes the write time, disk volume number, fragment number, and data block number as time location description components and appends them to the end of the normalized vector, generating a fine-grained storage data vector carrying time location information. The server performs the same processing on all semantic atomic units, ultimately forming a unified set of fine-grained storage data vectors, providing a standardized vector foundation for subsequent semantic retrieval, topology analysis, hot / cold tiering, and storage scheduling.

[0103] In this embodiment of the invention, the method further includes:

[0104] Obtain the storage time tag of each storage data segment unit in the original storage data stream, and construct a time evolution sequence of the storage data segment unit according to the order of the storage time tags. The time evolution sequence describes the temporal evolution of the semantic content in the original storage data stream.

[0105] Extract the differences in semantic atom unit composition between adjacent storage data segment units in the time evolution sequence, and identify semantic atom units added in subsequent storage data segment units and semantic atom units that appeared in previous storage data segment units but disappeared in subsequent storage data segment units;

[0106] The newly added semantic atom units are subjected to semantic source tracing processing. The semantic topology graph is searched for the preceding semantic atom units that are connected to the newly added semantic atom units through cross-segment co-occurrence topology edges or length-related topology edges. The searched preceding semantic atom units are marked as the evolutionary predecessor nodes of the newly added semantic atom units.

[0107] The semantic destination tracking process is performed on the missing semantic atom units. In the semantic topology graph, the subsequent semantic atom units connected to the missing semantic atom units through cross-segment co-occurrence topology edges or length-related topology edges are searched. The searched subsequent semantic atom units are marked as the evolution successor nodes of the missing semantic atom.

[0108] Based on the combined connection strength value of the topological edges between the newly added semantic atom's evolutionary predecessor node and the newly added semantic atom unit, an inheritance strength parameter is generated, which reflects the degree to which the newly added semantic atom inherits the semantic connotation of the preceding semantic atom.

[0109] Based on the combined connection strength value of the topological edges between the evolved successor node of the vanished semantic atom and the vanished semantic atom unit, a transfer strength parameter is generated, which reflects the degree to which the semantic connotation of the vanished semantic atom is preserved in the successor semantic atom.

[0110] The inheritance strength parameter and the transfer strength parameter are encoded as evolutionary association description fields, and the evolutionary association description fields are appended to the fine-grained storage data vector of the corresponding semantic atom unit to generate a fine-grained storage data vector carrying evolutionary information.

[0111] Based on the fine-grained storage data vector carrying evolution information, a semantic evolution trajectory description structure is constructed for the original storage data stream. The semantic evolution trajectory description structure records the association links of the birth, inheritance, transfer and extinction of semantic atomic units in the time dimension.

[0112] Evolutionary pattern recognition processing is performed on the semantic evolution trajectory description structure to extract the topic convergence direction information and semantic drift amplitude information of the semantic connotation evolution over time in the original stored data stream.

[0113] In an embodiment of the invention, for example, after generating semantic atomic units and fine-grained storage data vectors, the server further obtains the storage timestamp of each storage data segment unit in the original storage data stream, and constructs a temporal evolution sequence of the storage data segment units in chronological order. For example, in an order business log scenario, the server arranges "10:00 Order Creation Segment", "10:01 Payment Processing Segment", "10:02 Payment Failure Segment", "10:03 Inventory Rollback Segment", and "10:04 Successful Repayment Segment" in sequence to form a temporal evolution context describing the changes in business semantics over time.

[0114] The server extracts semantic atomic unit composition differences between adjacent stored data segments in the time evolution sequence. When comparing "payment processing segment" and "payment failure segment," the server identifies newly added semantic atomic units such as "payment error code," "failure reason description," and "channel abnormal response" in subsequent segments. Simultaneously, it identifies that "payment request sending status" from previous segments no longer appears in subsequent segments. Through this processing, the server can clearly identify newly emerging and disappeared business semantics at each time stage.

[0115] For newly added semantic atomic units, the server performs semantic source tracing processing. The server searches the semantic topology graph for preceding semantic atomic units that are connected to the "payment error code" through cross-segment co-occurrence topological edges or length-related topological edges. For example, if the server finds that the "payment channel call result" in the preceding segment has the same offset structure or similar length attributes as the newly added "payment error code," and there is a high composite connection strength value between the two, the server marks the "payment channel call result" as the evolutionary predecessor node of the "payment error code," indicating that the semantics of the new error code originates from the state change of the preceding payment call result.

[0116] For missing semantic atomic units, the server performs semantic destination tracking. The server searches the semantic topology graph for subsequent semantic atomic units that are connected to the missing "payment request sending status" through cross-segment co-occurrence topology edges or length-related topology edges. For example, if a subsequent segment contains "payment failure return status" and "failure reason description," the server marks it as the evolved successor node of the missing semantic atom based on the topology edge connection relationship, indicating that the preceding request sending semantics has been transferred to failure response semantics in the subsequent stage.

[0117] The server generates an inheritance strength parameter based on the overall connection strength value of the topological edges between newly added semantic atoms and their evolving predecessor nodes. A higher overall connection strength indicates a stronger semantic inheritance of the preceding semantic atoms by the newly added semantic atom. For example, "payment error code" and "payment channel call result" have similar offset positions, similar length attributes, and consistent cross-segment structural roles, resulting in a higher inheritance strength parameter generated by the server. The server also generates a transition strength parameter based on the overall connection strength value between disappearing semantic atoms and their evolving successor nodes, reflecting the degree to which disappearing semantics are preserved in subsequent semantics. For example, "payment request sending status" retains payment stage and status change information in the subsequent "payment failure return status," resulting in a higher transition strength parameter generated by the server.

[0118] The server encodes the inheritance strength parameters and transition strength parameters into evolutionary association description fields and appends them to the fine-grained storage data vector of the corresponding semantic atom unit. The resulting vector not only contains local semantic environment features, global topological location features, storage time, and storage location, but also carries information about the source and destination of the semantic atom in the time series. For example, the vector for "payment error code" is appended with an inheritance description indicating its origin from "payment channel call result," and the vector for "payment request sending status" is appended with a destination description indicating its transition to "payment failure return status."

[0119] Based on these fine-grained storage data vectors carrying evolutionary information, the server constructs a semantic evolution trajectory description structure for the original stored data stream. This structure records the birth, inheritance, transfer, and demise of each semantic atom unit in the time dimension. For example, a link evolves from "order creation" to "payment request," then to "payment failure," and finally to "inventory rollback" or "successful re-payment." The server performs evolutionary pattern recognition on this semantic evolution trajectory description structure, extracting the theme convergence direction and semantic drift magnitude. If the server identifies a large number of segments gradually converging from "normal payment" to "channel timeout anomaly," it can output the theme convergence direction; if the server identifies a significant semantic drift from "user operation" to "system rollback," it can calculate a large semantic drift magnitude, providing a basis for anomaly detection, data governance, and storage scheduling.

[0120] In this embodiment of the invention, the method further includes:

[0121] Extract the content feature description information of the semantic atom unit corresponding to each topological node in the semantic topology graph, calculate the semantic content correlation degree between different topological nodes based on the content feature description information, and write the semantic content correlation degree as the weight of the content semantic topological edge into the semantic topology graph to form a hybrid semantic correlation topology graph.

[0122] In the hybrid semantic association topology graph, a high-density semantic association subgraph composed of content semantic topology edges and co-occurrence topology edges is identified, and the semantic atom unit group corresponding to the high-density semantic association subgraph is marked as a semantic atom unit community.

[0123] Define the core semantic atom unit in each semantic atom unit community, calculate the sum of the topological edge weights of each semantic atom unit in the semantic atom unit community and other semantic atom units in the community, and determine the semantic atom unit with the largest sum of topological edge weights as the core semantic atom of the community.

[0124] Extract the fine-grained storage data vector corresponding to the core semantic atom of the community as the community representative vector, and perform semantic projection processing on the fine-grained storage data vector of all semantic atom units in the semantic atom unit community and the community representative vector to generate the semantic membership degree distribution of each semantic atom unit relative to the core semantic atom of the community.

[0125] Based on the semantic membership distribution, the semantic atom unit community is subjected to semantic hierarchical reorganization processing, and semantic atom units with similar semantic membership are grouped into the same semantic level. A semantic level label field is added to the fine-grained storage data vector to record the semantic level information to which each semantic atom unit belongs.

[0126] The community representative vector and the semantic membership distribution are written as community description metadata into the fine-grained storage data vector of each semantic atom unit in the semantic atom unit community, thereby generating a fine-grained storage data vector carrying community structure information.

[0127] A semantic interaction topology edge between the core semantic atoms of different semantic atom units is established between the communities. The weight of the semantic interaction topology edge between the communities is determined based on the semantic projection distance of the fine-grained storage data vectors of the two community core semantic atoms.

[0128] The semantic interaction topology network between communities is constructed using the semantic interaction topology edges between communities. Community-level semantic interaction topology network is then used to perform community-level semantic propagation processing to generate community-level semantic context vectors for each semantic atom community.

[0129] The community-level semantic context vector is distributed to each semantic atom unit in the corresponding semantic atom unit community. The community-level semantic context vector is then concatenated with the fine-grained storage data vector of each semantic atom unit to generate a fine-grained storage data vector with multi-granularity semantic awareness capability.

[0130] In this embodiment of the invention, the method further includes:

[0131] The original storage data stream is segmented into storage data segment units, and the access frequency of each storage data segment unit is calculated by external storage operations. Based on the access frequency, an access heat attribute is added to each storage data segment unit.

[0132] Obtain the storage media stack-up structure information of the storage system to which the original storage data stream belongs. The storage media stack-up structure information describes the access rate level and storage capacity limit of the first storage media layer and the access rate level and storage capacity limit of the second storage media layer. The access rate level of the first storage media layer is greater than the access rate level of the second storage media layer. Construct a storage media stack-up model based on the storage media stack-up structure information.

[0133] Extract the component representing the local semantic environment features from the fine-grained storage data vector of each semantic atom unit as an abstract semantic feature component subset, and bind the abstract semantic feature component subset with the offset position marker and length attribute of the semantic atom unit to form a lightweight summary descriptor of the semantic atom unit.

[0134] Based on the access popularity attribute of the storage data segment unit to which the semantic atom unit belongs and the topological centrality measure of the semantic atom unit in the semantic topology graph, a reconstruction priority score parameter is calculated for each semantic atom unit. The reconstruction priority score parameter reflects the probability density of the semantic atom unit being requested in storage data access.

[0135] Based on the reconstruction priority scoring parameters and the storage medium stacking model, the fine-grained storage data vectors corresponding to semantic atomic units whose reconstruction priority scoring parameters exceed a preset migration threshold are migrated to the first storage medium layer, while the fine-grained storage data vectors corresponding to semantic atomic units whose reconstruction priority scoring parameters do not exceed the preset migration threshold are retained in the second storage medium layer.

[0136] The fine-grained storage data vector migrated to the first storage medium layer is compressed and encoded to generate a compressed fine-grained storage data vector. The fine-grained storage data vector retained in the second storage medium layer is fully retained to preserve all semantic information of the fine-grained storage data vector.

[0137] When an access request for a fine-grained storage data vector for a specific semantic atom unit is received, the storage medium layer where the semantic atom unit is currently located is located according to the lightweight summary descriptor of the semantic atom unit. If the semantic atom unit is located in the first storage medium layer and its storage form is a compressed fine-grained storage data vector, then decompression and reconstruction processing is performed on the compressed fine-grained storage data vector.

[0138] The fine-grained storage data vector obtained by decompression and reconstruction is compared with the lightweight summary descriptor of the semantic atom unit to verify whether the fine-grained storage data vector after decompression and reconstruction retains the local semantic environment features and global topological location features of the semantic atom unit.

[0139] For fine-grained storage data vectors that have undergone semantic consistency comparison and whose semantic feature retention reaches a preset semantic fidelity threshold, the storage location identifier is updated. The current resident layer information of the fine-grained storage data vector in the storage medium stack-up structure is recorded, and the current resident layer information is appended to the data structure of the semantic atom unit as a storage state description field.

[0140] During the operation of the storage system, the storage space occupancy status of the first storage medium layer is continuously monitored. When the storage space occupancy ratio of the first storage medium layer reaches the preset storage space pressure threshold, the fine-grained storage data vector of the semantic atomic unit with the lowest reconstruction priority score parameter in the first storage medium layer is migrated back to the second storage medium layer and restored to the full retention form.

[0141] In this embodiment of the invention, for example, the server performs segmented frequency statistics on each storage data segment unit in the original storage data stream. The server records the number of times each segment is accessed by storage operations such as external queries, index hits, vector retrieval, and data reconstruction, and attaches an access popularity attribute based on the access frequency. For example, if a payment anomaly segment is accessed multiple times by the risk control system and operations and maintenance retrieval within one hour, the server marks it as a high-popularity segment; if a normal heartbeat log segment is accessed less frequently, the server marks it as a low-popularity segment.

[0142] The server further obtains information about the storage media stack-up structure of the storage system and constructs a storage media stack-up model. This model records the access rate levels and capacity limitations of the first and second storage media layers. For example, the first storage media layer is a high-speed solid-state cache layer with high access rates but limited capacity; the second storage media layer is a large-capacity disk layer with lower access rates but can store complete vector data. Based on this, the server determines the residency strategy for different fine-grained storage data vectors.

[0143] The server extracts components representing local semantic environment features from the fine-grained stored data vector of each semantic atom unit, forming an abstract semantic feature subset. This subset is then bound to the offset position marker and length attribute of the semantic atom unit to generate a lightweight summary descriptor. For example, the summary descriptor of the "payment error code" unit records the error code semantic summary, the starting offset of 768 bytes, and the length of 32 bytes, allowing the server to locate its basic semantic and structural position without reading the complete vector.

[0144] The server calculates reconstruction priority scoring parameters based on the access popularity attribute of the segment to which the semantic atomic unit belongs, and the topological centrality measure of that semantic atomic unit in the semantic topology graph. Payment anomaly segments have high access popularity, and the "payment failure return" node connects to error codes, failure reasons, and inventory rollback nodes, indicating high topological centrality; therefore, the server generates a higher reconstruction priority score for them. Duplicate states indicate fewer node visits and weaker connections, resulting in a lower score. This score represents the probability density of the corresponding semantic atomic unit being requested in subsequent accesses.

[0145] The server performs vector migration based on reconstruction priority scoring parameters and a storage media stacking model. Semantic atomic units with scores exceeding a preset migration threshold have their fine-grained storage data vectors migrated to the first storage media layer for fast access; vectors with scores below the threshold are retained in the second storage media layer. For vectors migrated to the first storage media layer, the server performs compression encoding to generate compressed fine-grained storage data vectors to reduce space usage in the high-speed layer; for vectors retained in the second storage media layer, the server performs full retention, preserving complete local semantics, topological location, temporal location, and community structure information.

[0146] When the server receives an access request for a specific semantic atom, it first reads the lightweight summary descriptor of that semantic atom and uses it to locate the current storage medium layer. For example, if the retrieval request hits the "payment failed return" unit, the server determines from the summary descriptor that it is located in the first storage medium layer and is in a compressed state. Therefore, it performs decompression and reconstruction processing to restore the corresponding fine-grained storage data vector.

[0147] The server performs a semantic consistency comparison between the decompressed vector and the lightweight summary descriptor. The server checks whether the reconstructed vector still retains the local semantics of "payment failure," error state features, and topological relationships with error codes and inventory rollback nodes. Once the comparison result reaches a preset semantic fidelity threshold, the server updates the storage location identifier of the vector, recording its current residence in the first storage medium layer, and appends this residence layer information as a storage state description field to the data structure of the semantic atomic unit.

[0148] During continuous operation of the storage system, the server monitors the space occupancy status of the first storage media layer in real time. When the space occupancy ratio of the high-speed layer reaches a preset storage space pressure threshold, the server selects the semantic atomic units with the lowest reconstruction priority score from the first storage media layer for migration back. For example, the "historical payment error description" vector, which was frequently accessed in the early stages but has recently experienced a decrease in access, is migrated back to the second storage media layer and restored to its full-retention form. In this way, the server maintains stable capacity of the high-speed storage layer while ensuring fast access to high-value semantic atomic units.

[0149] In one alternative implementation, please refer to [reference needed]. Figure 2 , Figure 2This is an interactive schematic diagram illustrating the generation of spatial augmentation vectors in an embodiment of the present invention. The server generates spatial augmentation vectors as follows: The server extracts a component representing the offset position marker from each fine-grained stored data vector and uses this component as a spatial coordinate field. For multiple semantic atomic units belonging to the same stored data segment unit, the server sorts them according to the offset position marker in ascending order and groups semantic atomic units with an offset position difference less than a preset spatial proximity threshold into the same spatial collaborative semantic atomic unit group. The preset spatial proximity threshold can be determined according to the proportion of the length of the stored data segment unit, for example, set to 5% to 20% of the length of the stored data segment unit. For any spatial collaborative semantic atomic unit group, the server concatenates the semantic atomic units in the group into a continuous semantic segment according to the offset position marker in ascending order, and calculates the overall semantic topic vector Tg based on the fine-grained stored data vectors of each semantic atomic unit in the continuous semantic segment. The overall semantic topic vector Tg can be a weighted average vector of all fine-grained stored data vectors in the group, and the weights can be determined according to the semantic atomic unit length attribute, the topological edge comprehensive connection strength value, or the semantic quality factor. The server calculates the topic deviation parameter Bi for each semantic atomic unit within the group: Bi = 1 - cos(Fi, Tg); where Fi represents the fine-grained storage data vector of the semantic atomic unit Ai, and Tg represents the overall semantic topic vector of the group. When Bi is greater than a preset topic deviation tolerance threshold, the server removes Ai from the spatial collaborative semantic atomic unit group. The preset topic deviation tolerance threshold can be set to 0.35, 0.40, or 0.50. For the filtered topic-purified collaborative semantic atomic unit groups, the server constructs a spatially ordered semantic transmission path. Let the semantic atomic units arranged in ascending order of offset position in this path be A1, A2, ..., Am, and the corresponding fine-grained storage data vectors be F1, F2, ..., Fm. The server performs forward semantic recursion processing: PF1 = F1; PFi = Norm((1-η)×Fi + η×PFi-1), i = 2, 3, ..., m; where PFi represents the pre-order enhanced fine-grained storage data vector of the i-th semantic atom unit, and η is the forward recursion weight, which can be set to a value between 0.2 and 0.6. The server performs backward semantic recursion processing: BFm = Fm; BFi = Norm((1-η)×Fi + η×BFi+1), i = m-1, m-2, ..., 1; where BFi represents the post-order enhanced fine-grained storage data vector of the i-th semantic atom unit. The server fuses the original fine-grained storage data vector Fi, the pre-order enhanced fine-grained storage data vector PFi, and the post-order enhanced fine-grained storage data vector BFi to obtain the spatial enhancement vector Ei: Ei = Norm(We×[Fi,PFi,BFi]); where We is the fusion matrix, and [ ] indicates vector concatenation.The fusion matrix We can be obtained through training on historical samples or generated using a fixed projection matrix. The spatial augmentation vector Ei simultaneously contains the local semantic environment features, global topological location features, preceding semantic background features, and subsequent semantic expectation features of the semantic atom unit itself.

[0150] In an embodiment of the invention, for example, after generating fine-grained storage data vectors, the server further extracts the component representing the offset position marker in each vector and defines this component as a spatial coordinate field. This spatial coordinate field is used to represent the relative address position of a semantic atomic unit within its respective storage data segment unit. For example, in a payment processing segment, "order amount confirmation" is located at 128 bytes, "payment channel call" at 384 bytes, "payment failure return" at 768 bytes, and "failure reason description" at 896 bytes. The server encodes this offset information into a spatial coordinate field, so that the vector not only expresses semantics but also preserves the spatial order of data within the segment.

[0151] The server retrieves the spatial coordinate fields of all semantic atomic units in the vector space and identifies areas where spatial coordinates cluster. In the order log, multiple semantic atomic units related to payment processing are concentrated in the middle and later address ranges of the same segment. The server marks units such as "payment channel call," "payment failure return," "error code," and "failure reason description" within this area as spatially coordinated semantic atomic unit groups. This group indicates that these semantic atomic units are adjacent to each other in storage address and usually jointly describe a continuous business process.

[0152] Subsequently, the server performs spatial sequence reconstruction on the spatially coordinated semantic atomic unit groups. The server concatenates the semantic atomic units within the group into continuous semantic fragments according to the ascending order of their offset positions. For example, the server reconstructs "payment channel call—payment failure return—error code—failure reason description" into a continuous semantic fragment and extracts the overall semantic theme description information from this fragment, namely "payment channel call failure and reason explanation." This overall theme is used to determine whether each semantic atomic unit within the group truly belongs to the same semantic process.

[0153] The server performs topic consistency analysis on the overall semantic topic description information of continuous semantic fragments and the fine-grained semantic context vectors of each semantic atomic unit within the group. For "error code" and "failure reason description," their vectors are highly consistent with the "payment failure" topic, and the server generates a low topic deviation parameter. For "debugging placeholder field" or "no-follow-click information" mixed into this address area, their vectors differ significantly from the overall topic, and the server generates a high topic deviation parameter. Based on the topic deviation parameter, the server performs topic cohesion filtering, removing semantic atomic units that exceed the preset topic deviation tolerance threshold, retaining units with consistent topics, and forming a topic-purified collaborative semantic atomic unit group.

[0154] Within a group of topic-purified collaborative semantic atomic units, the server constructs a spatially ordered semantic transmission path. This path connects adjacent semantic atomic units sequentially along the increasing direction of the offset position marker, with the connection direction consistent with the growth direction of the storage address. For example, the path sequentially points from "Payment Channel Invocation" to "Payment Failure Return," then to "Error Code," and finally to "Failure Reason Description." This path reflects the actual arrangement order of data in the storage segment and also reflects the unfolding order of business semantics.

[0155] The server performs forward semantic recursion processing along the spatially ordered semantic transmission path. The server uses the semantic atomic unit vectors of preceding positions as semantic background information for subsequent positions. For example, the vector for "payment channel call" is passed to "payment failure return," enhancing the "payment failure return" vector with its background originating from the channel call process; the "payment failure return" vector is then passed to "error code," making the "error code" not only represent a numerical field but also indicate that the error code belongs to a payment failure scenario. Based on this, the server generates a fine-grained, pre-order enhanced storage data vector carrying the pre-order semantic background.

[0156] The server also performs backward semantic recursion processing on the same path. The server uses the semantic atomic unit vector of the subsequent position as the semantic expectation information of the preceding position. For example, the "failure reason description" is passed forward to the "error code", so that the "error code" vector obtains the subsequent reason explanation; the "error code" is then passed forward to the "payment failure return", so that the node obtains the semantic expectation of the subsequent failure details. Based on this, the server generates a backward enhanced fine-grained storage data vector carrying the backward semantic expectation.

[0157] Finally, the server performs a three-dimensional semantic fusion of the preceding enhanced fine-grained storage data vector, the following enhanced fine-grained storage data vector, and the original fine-grained storage data vector. The fused spatially enhanced vector simultaneously includes the local semantic environment features of the semantic atom unit itself, the global topological position features in the semantic topology graph, the semantic background features from the preceding node, and the semantic expectation features from the following node. The server replaces the original fine-grained storage data vector with this spatially enhanced vector as the final vectorized representation of the semantic atom unit in the original storage data stream, thereby improving the completeness and order awareness of semantic relationship expression in consecutive storage segments.

[0158] In this embodiment of the invention, the method further includes:

[0159] Obtain the length attribute of the semantic atom unit corresponding to each topological node in the semantic topology graph, and divide all semantic atom units into multiple semantic granularity levels according to the numerical distribution of the length attribute. Semantic atom units within the same semantic granularity level have similar length attribute numerical ranges.

[0160] Within each semantic granularity level, fine-grained storage data vectors of all semantic atomic units within that semantic granularity level are extracted, and a hierarchical semantic vector space for that semantic granularity level is constructed based on the fine-grained storage data vectors. The dimension of the hierarchical semantic vector space is consistent with the unified dimension of the fine-grained storage data vectors.

[0161] Perform inter-level semantic mapping processing between different semantic granularity levels. Take the hierarchical semantic vector space of the semantic granularity level where the fine-grained semantic atom unit is located as the source hierarchical semantic vector space, and take the hierarchical semantic vector space of the semantic granularity level where the coarse-grained semantic atom unit is located as the target hierarchical semantic vector space. Construct a hierarchical semantic mapping function from the source hierarchical semantic vector space to the target hierarchical semantic vector space.

[0162] The hierarchical semantic mapping function takes the fine-grained storage data vector of the fine-grained semantic atom unit in the source hierarchical semantic vector space as the mapping input, and takes the fine-grained storage data vector of the coarse-grained semantic atom unit corresponding to the upper-level semantic structure to which the fine-grained semantic atom unit belongs in the storage data fragment unit as the mapping target. It learns the semantic aggregation mapping rules between the source hierarchical semantic vector space and the target hierarchical semantic vector space through the semantic mapping relationship between the hierarchical levels.

[0163] The hierarchical semantic mapping function is used to perform hierarchical upward mapping on the fine-grained storage data vector of each fine-grained semantic atom unit to generate the upward-mapped virtual semantic vector of the fine-grained semantic atom unit in the target hierarchical semantic vector space. The upward-mapped virtual semantic vector represents the semantic vector expression when the semantic connotation of the fine-grained semantic atom unit is projected to a coarser semantic granularity level.

[0164] Aggregate and align the upward-mapped virtual semantic vectors of all fine-grained semantic atomic units contained in the same coarse-grained semantic atomic unit, calculate the semantic distribution center vector of the upward-mapped virtual semantic vector, and compare the semantic distribution center vector with the original fine-grained storage data vector generated by the coarse-grained semantic atomic unit to obtain the hierarchical mapping semantic fidelity parameter.

[0165] The mapping parameters of the hierarchical semantic mapping function are adjusted in reverse according to the hierarchical mapping semantic fidelity parameter, so that the semantic deviation between the semantic distribution center vector of the upward mapping virtual semantic vector generated by the hierarchical semantic mapping function after the mapping parameter adjustment and the fine-grained storage data vector of the coarse-grained semantic atom unit gradually converges.

[0166] After parameter adjustment, the hierarchical semantic mapping function is extended to construct the semantic mapping relationship between all adjacent semantic granularity levels in the semantic topology graph, forming a hierarchical semantic mapping function chain covering all semantic granularity levels;

[0167] Between any two non-adjacent semantic granularity levels in the hierarchical semantic mapping function chain, a cross-level semantic mapping function is generated through a layer-by-layer composite mapping method. The cross-level semantic mapping function supports bidirectional semantic mapping transformation of fine-grained stored data vectors of semantic atom units between any two semantic granularity levels.

[0168] The hierarchical semantic mapping function chain and the cross-level semantic mapping function are appended as semantic granularity hierarchical transformation description metadata to the fine-grained storage data vector of each semantic atom unit, so that the fine-grained storage data vector of each semantic atom unit has both the original semantic expression capability at this semantic granularity level and the transformable semantic expression capability at any semantic granularity level.

[0169] In this embodiment of the invention, for example, the server obtains the length attribute of the semantic atomic unit corresponding to each topological node in the semantic topology graph, and divides it into multiple semantic granularity levels according to the numerical distribution of the length attribute. When processing order storage data, the server classifies short fields such as "error code," "status code," and "order number" into the fine-grained level, medium-length fields such as "payment return summary" and "inventory rollback result" into the medium-grained level, and long content such as "exception reason description" and "risk control audit description" into the coarse-grained level. The semantic atomic unit lengths within each level are similar, indicating that their semantic carrying range and data granularity are similar.

[0170] Within each semantic granularity level, the server extracts fine-grained storage data vectors of all semantic atomic units within that level and constructs the corresponding hierarchical semantic vector space. The dimension of this vector space remains consistent with the aforementioned unified dimension. For example, the fine-grained level stores vectors such as "error codes" and "status codes," while the medium-grained level stores vectors such as "payment status description" and "channel return summary." This allows the server to compare similar fields within the same granularity level and establish mapping relationships between different granularity levels.

[0171] Subsequently, the server performs inter-level semantic mapping processing between different semantic granularity levels. The server uses the vector space containing the fine-grained semantic atoms as the source-level semantic vector space, and the coarse-grained vector space containing their superior semantic structures as the target-level semantic vector space, constructing a hierarchical semantic mapping function. For example, "error code," "channel response code," and "failure status code" all belong to the superior semantic structure "payment failure return." The server uses these fine-grained vectors as mapping input and the vector of "payment failure return" as the mapping target, learning the aggregation rules from multiple fine-grained semantics to coarse-grained business semantics.

[0172] The server utilizes a hierarchical semantic mapping function to perform hierarchical up-mapping on the fine-grained stored data vector of each fine-grained semantic atom unit, generating an up-mapped virtual semantic vector. For example, mapping the vector of "error code 504" to a medium-grained level yields a virtual semantic vector representing "payment channel timeout exception"; mapping the vector of "inventory status rollback completed" to a coarser level yields a virtual semantic vector representing "exception handling completed." This virtual vector represents the expression of fine-grained semantics at a coarser semantic granularity.

[0173] For multiple fine-grained semantic atomic units contained within the same coarse-grained semantic atomic unit, the server aggregates and aligns their upward-mapped virtual semantic vectors. The server calculates the semantic distribution center vector of these virtual vectors and compares this center vector with the original fine-grained stored data vector generated by the coarse-grained semantic atomic unit for semantic consistency. For example, when the center vectors mapped to "error code," "failure reason," and "channel return summary" are highly close to the original vector of "payment failure return," the server obtains a higher hierarchical mapping semantic fidelity parameter; when the deviation between the two is large, the server obtains a lower fidelity parameter.

[0174] The server adjusts the mapping parameters of the hierarchical semantic mapping function in reverse based on the hierarchical mapping semantic fidelity parameter, so that the center of the mapped semantic distribution gradually approaches the original vector of the corresponding coarse-grained semantic atom. After parameter adjustment, the server extends the hierarchical semantic mapping function to all adjacent semantic granularity levels in the semantic topology graph, forming a hierarchical semantic mapping function chain covering short fields, medium fields, and long content.

[0175] Between any two non-adjacent semantic granularity levels, the server generates cross-level semantic mapping functions through layer-by-layer composite mapping. For example, the server first maps "error code" from the fine-grained level to the medium-grained level where "payment failure return" is located, and then maps it to the coarse-grained level where "payment anomaly event" is located, thus supporting cross-level semantic transformation. The server also retains the reverse mapping relationship, allowing coarse-grained semantics to fall back to the corresponding fine-grained semantic components. Finally, the server appends the hierarchical semantic mapping function chain and the cross-level semantic mapping function as semantic granularity level transformation description metadata to the fine-grained storage data vector of each semantic atom unit, so that each vector retains the original semantic expression of its own level and has the ability to transform its expression between any semantic granularity level.

[0176] In an optional implementation, the server can also trigger vector generation processing via file change awareness. Specifically, the server utilizes the event listening function of the distributed file system to monitor file change operations such as file creation, modification, and deletion in real time. When a file change is detected, the file change awareness module generates a file change event and sends it to the vector generation module through a notification mechanism. This notification mechanism can be implemented based on message queues, such as Kafka or RabbitMQ, to ensure reliable transmission of file change events. Upon receiving a file change event, the vector generation module reads the corresponding file content and uses it as the raw storage data stream for semantic boundary identification, semantic atomic unit segmentation, semantic topology graph construction, semantic context aggregation, and granular alignment transformation. When generating initialization vectors or semantic representations, the vector generation module can use an embedding model to convert the file content or semantic atomic units into vector representations. This embedding model can be a Sentence Transformers model or other text embedding models. The generated fine-grained storage data vectors are stored in a vector database, which can be Elasticsearch, Milvus, or other database systems that support vector storage and retrieval, thereby achieving automatic updates and reliable storage of vector data after file changes.

[0177] This invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned semantically aware fine-grained stored data vector generation method. For example... Figure 3 As shown, Figure 3 This is a structural block diagram of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or interaction, the memory 111, processor 112, and communication unit 113 are electrically connected to each other directly or indirectly. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0178] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the foregoing illustrative discussions are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in accordance with the foregoing teachings. These embodiments were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and to employ various embodiments with different modifications to suit a particular intended application.

Claims

1. A method for generating fine-grained storage data vectors based on semantic awareness, characterized in that, The method includes: Obtain a raw storage data stream with storage time stamps and storage location identifiers, the raw storage data stream containing multiple consecutively recorded storage data segment units with data block boundary markers; The original stored data stream is subjected to semantic boundary recognition processing to generate semantic boundary combinations within each stored data segment unit, and each stored data segment unit is divided into multiple semantic atom units based on the semantic boundary combinations. A semantic topology graph is constructed based on the offset position marker and length attribute carried by the semantic atomic unit, and the semantic topology graph uses the semantic atomic unit as a topology node; Based on the semantic topology graph, semantic context aggregation processing is performed on each topology node to generate a fine-grained semantic context vector corresponding to each semantic atom unit. The fine-grained semantic context vector includes the local semantic environment features of the semantic atom unit and the global topology position features of the semantic atom unit in the semantic topology graph. The fine-grained semantic context vector is granularized by using the length attribute of the semantic atom unit to generate a fine-grained storage data vector with a unified dimension representation. The fine-grained storage data vector is used to describe the semantic connotation and storage structure position of the semantic atom unit in the original storage data stream.

2. The method according to claim 1, characterized in that, The step of performing semantic boundary recognition processing on the original stored data stream to generate semantic boundary combinations within each stored data segment unit, and then dividing each stored data segment unit into multiple semantic atomic units based on the semantic boundary combinations, includes: Parse the data block boundary markers in the original stored data stream to form a time series chain that records the storage time tag and the storage location identifier; Semantic content scanning is performed on the stored data segment units in the time series chain to identify semantic transition discontinuities; The semantic boundary combination is constructed based on the positional distribution of the semantic transformation discontinuities. The stored data segment unit is segmented using the semantic boundary combination to form the semantic atom unit; Extract the offset position marker and length attribute of the semantic atom unit, and associate the offset position marker, the length attribute, the storage time tag, and the storage location identifier with the semantic atom unit.

3. The method according to claim 1, characterized in that, The step of constructing a semantic topology graph based on the offset position marker and length attribute carried by the semantic atom unit includes: The semantic atomic unit is initialized as a topological node, and the offset position marker, the length attribute, the storage time tag, and the storage location identifier are used as node-related description information. Semantic adjacency paths are established based on the offset position relationships within the same storage data segment unit; topological edges are established based on the offset position matching relationships and length attribute similarity relationships between different storage data segment units. The overall connection strength value of the topological edge is determined based on the offset position marker and the length attribute; The semantic topology graph is obtained by pruning and supplementing the topological edges based on the comprehensive connection strength value.

4. The method according to claim 1, characterized in that, The step of performing semantic context aggregation processing on each topology node based on the semantic topology graph to generate a fine-grained semantic context vector corresponding to each semantic atom unit includes: In the semantic topology graph, the target aggregation topology node and the neighboring domain topology node are determined; Based on the offset position relationship, length attribute relationship and topological edge connection strength between the target aggregate topological node and the neighboring domain topological node, structural relationship information and adjacency weight information are constructed. Content parsing is performed on the semantic atomic units corresponding to the target aggregation topology node and the neighboring domain topology node, and initialization vectors are generated. The initialization vector is aggregated based on the adjacency weight information, and the aggregation result is adjusted using the structural relationship information to generate the fine-grained semantic context vector.

5. The method according to claim 1, characterized in that, The step of performing granular alignment transformation on the fine-grained semantic context vector using the length attribute of the semantic atom unit to generate a fine-grained stored data vector with a unified dimensional representation includes: Obtain the fine-grained semantic context vector and the length attribute of each semantic atom unit. Construct a length granularity grouping interval based on the numerical distribution range of the length attribute. The length granularity grouping interval will group semantic atom units with similar length attributes into the same granularity group. Within each length granularity grouping interval, the dimensional distribution pattern of the fine-grained semantic context vectors of all semantic atom units in that group is statistically analyzed, and the baseline transformation dimension corresponding to that length granularity grouping interval is determined based on the dimensional distribution pattern. For each semantic atom unit, a granularity scaling adjustment factor is calculated based on the offset between the length attribute of the semantic atom unit and the center length of the interval to which it belongs in the length granularity grouping interval. The granularity scaling adjustment factor changes in correlation with the offset. The fine-grained semantic context vector of the semantic atom unit is scaled using the granularity scaling adjustment factor. The dimension of the fine-grained semantic context vector is stretched or shrunk to the reference transformation dimension corresponding to the length granularity grouping interval, so as to obtain the granularity aligned intermediate vector. Construct a granularity alignment transformation matrix, wherein the row dimension of the granularity alignment transformation matrix is ​​equal to the baseline transformation dimension, and the column dimension is equal to the original dimension of the fine-grained semantic context vector before performing dimension scaling. The elements of the granularity alignment transformation matrix are determined based on the granularity scaling adjustment factor and the feature distribution on the original dimension of the fine-grained semantic context vector. The fine-grained semantic context vector is linearly mapped using the granularity alignment transformation matrix, mapping the fine-grained semantic context vector from the original dimension space to the baseline transformation dimension space, thereby generating a granularity alignment normalized vector. In the process of generating granular alignment normalized vectors, semantic integrity preservation processing is performed on the granular alignment normalized vectors. The component parts representing local semantic environment features and the component parts representing global topological location features in the fine-grained semantic context vectors are extracted, and the relative proportional relationship between the two components is preserved in the granular alignment normalized vectors. The granularity alignment normalization vector is fused with the storage time tag and storage location identifier of the semantic atom unit. The storage time tag and storage location identifier are encoded as time location description components and appended to the tail of the granularity alignment normalization vector to generate a fine-grained storage data vector carrying time location information. Perform the same granularity alignment transformation operation on all semantic atomic units to generate a set of fine-grained storage data vectors with a unified dimensional representation corresponding to all the semantic atomic units.

6. The method according to claim 1, characterized in that, The method further includes: The storage data fragment unit is divided into storage location fragments according to the storage location identifier, and a storage location fragment semantic sub-topology graph is constructed based on the semantic atomic units within the storage location fragments; Establish cross-shard semantic bridge edges based on the offset position matching relationship and length attribute similarity relationship between shards in different storage locations; The storage location shard semantic sub-topology graph is connected into a globally unified storage semantic topology network through the cross-shard semantic bridge edge; In the globally unified storage semantic topology network, storage location-aware semantic propagation and semantic fusion are performed to generate storage location-aware fine-grained storage data vectors.

7. The method according to claim 1, characterized in that, The method further includes: The original storage data stream is segmented into storage data segment units, and the access frequency of each storage data segment unit is calculated by external storage operations. Based on the access frequency, an access heat attribute is added to each storage data segment unit. Obtain the storage media stack-up structure information of the storage system to which the original storage data stream belongs. The storage media stack-up structure information describes the access rate level and storage capacity limit of the first storage media layer and the access rate level and storage capacity limit of the second storage media layer. The access rate level of the first storage media layer is greater than the access rate level of the second storage media layer. Construct a storage media stack-up model based on the storage media stack-up structure information. Extract the component representing the local semantic environment features from the fine-grained storage data vector of each semantic atom unit as an abstract semantic feature component subset, and bind the abstract semantic feature component subset with the offset position marker and length attribute of the semantic atom unit to form a lightweight summary descriptor of the semantic atom unit. Based on the access popularity attribute of the storage data segment unit to which the semantic atom unit belongs and the topological centrality measure of the semantic atom unit in the semantic topology graph, a reconstruction priority score parameter is calculated for each semantic atom unit. The reconstruction priority score parameter reflects the probability density of the semantic atom unit being requested in storage data access. Based on the reconstruction priority scoring parameters and the storage medium stacking model, the fine-grained storage data vectors corresponding to semantic atomic units whose reconstruction priority scoring parameters exceed a preset migration threshold are migrated to the first storage medium layer, while the fine-grained storage data vectors corresponding to semantic atomic units whose reconstruction priority scoring parameters do not exceed the preset migration threshold are retained in the second storage medium layer. The fine-grained storage data vector migrated to the first storage medium layer is compressed and encoded to generate a compressed fine-grained storage data vector. The fine-grained storage data vector retained in the second storage medium layer is fully retained to preserve all semantic information of the fine-grained storage data vector. When an access request for a fine-grained storage data vector for a specific semantic atom unit is received, the storage medium layer where the semantic atom unit is currently located is located according to the lightweight summary descriptor of the semantic atom unit. If the semantic atom unit is located in the first storage medium layer and its storage form is a compressed fine-grained storage data vector, then decompression and reconstruction processing is performed on the compressed fine-grained storage data vector. The fine-grained storage data vector obtained by decompression and reconstruction is compared with the lightweight summary descriptor of the semantic atom unit to verify whether the fine-grained storage data vector after decompression and reconstruction retains the local semantic environment features and global topological location features of the semantic atom unit. For fine-grained storage data vectors that have undergone semantic consistency comparison and whose semantic feature retention reaches a preset semantic fidelity threshold, the storage location identifier is updated. The current resident layer information of the fine-grained storage data vector in the storage medium stack-up structure is recorded, and the current resident layer information is appended to the data structure of the semantic atom unit as a storage state description field. During the operation of the storage system, the storage space occupancy status of the first storage medium layer is continuously monitored. When the storage space occupancy ratio of the first storage medium layer reaches the preset storage space pressure threshold, the fine-grained storage data vector of the semantic atomic unit with the lowest reconstruction priority score parameter in the first storage medium layer is migrated back to the second storage medium layer and restored to the full retention form.

8. The method according to claim 1, characterized in that, The method further includes: Extract the component portion of the fine-grained storage data vector corresponding to each semantic atom unit, which represents the offset position marker of the semantic atom unit within its respective storage data segment unit, and define the component portion as a spatial coordinate field, which reflects the relative address position of the semantic atom unit within the storage data segment unit; Identify spatial collaborative semantic atomic unit groups based on the spatial coordinate field; The spatial sequence of the spatial cooperative semantic atomic unit group is reconstructed according to the offset position marker, and topic consistency screening is performed. A spatially ordered semantic transmission path is established between the selected semantic atomic units; forward semantic recursion and backward semantic recursion are performed along the spatially ordered semantic transmission path, and the recursion results are fused with the original fine-grained stored data vector to generate a spatially enhanced vector.

9. The method according to claim 1, characterized in that, The method further includes: The semantic atom unit is divided into multiple semantic granularity levels according to the length attribute, and a hierarchical semantic vector space is constructed based on the fine-grained storage data vectors within the semantic granularity levels. Establish a hierarchical semantic mapping function between different semantic granularity levels; use the hierarchical semantic mapping function to perform inter-level mapping on fine-grained stored data vectors to generate upward-mapped virtual semantic vectors; The hierarchical semantic mapping function is adjusted based on the semantic consistency between the upward-mapped virtual semantic vector and the fine-grained storage data vector in the corresponding semantic granularity level; The adjusted hierarchical semantic mapping functions are combined to form a hierarchical semantic mapping function chain.

10. A fine-grained data vector generation system based on semantic awareness, characterized in that, Includes at least one service node; The service node includes a storage unit and a computing unit; the storage unit is used to store program code; the computing unit is used to run the program code to perform the method according to any one of claims 1 to 9.

Images