Network security slice data storage method and system in enterprise operation

Abstract

The invention provides a network security slice data storage method and system in enterprise operation, and relates to the technical field of computer data storage and management.The method comprises the steps that a data stream generated by enterprise operation is received, the service type and priority information in the data stream are recognized, service associated data packets are aggregated into a data slice unit, and the data slice unit is stored; and adding a feature identifier containing a service feature and an expected processing path to each unit. And then obtaining a load and network connection state of the distributed storage cluster, and evaluating a storage position of each data slice unit in combination with an expected processing path and service characteristics of the characteristic identifier. The method comprises the following steps: firstly, evaluating storage positions of all units, then storing all the units to corresponding nodes according to the evaluated storage positions, establishing a distributed index directory for the data slicing units of all the storage nodes, and adjusting the classification hierarchy and the detailed degree of the index directory according to the data access frequency, so that distributed storage and efficient management of enterprise operation data can be realized.

Description

Technical Field

[0001] This application relates to the field of computer data storage and management technology, and in particular to a method and system for storing network security slices in enterprise operations. Background Technology

[0002] Enterprise operations continuously generate high-throughput, low-latency real-time data streams. This real-time data encompasses user interaction clickstreams, IoT device time-series data, and real-time transaction records, among other information. These operational interaction clickstreams, IoT device time-series data, and real-time transaction records constitute fundamental data related to enterprise network security. Enterprise storage systems must support hundreds of thousands of write throughput per second with millisecond-level write latency; they must organize data systematically according to business logic, supporting multi-dimensional sub-second queries; and simultaneously maintain system stability under tens of thousands of concurrent requests to prevent data loss or service interruptions.

[0003] Currently, a common approach is a centralized storage architecture combined with static sharding. This approach manages data distribution through a single metadata server, pre-dividing the data stream into static shards according to fixed rules and allocating them to different storage nodes; it uses a simple linear index, where data is directly appended to the corresponding shard after being written, and queries locate the shard through the metadata server before scanning.

[0004] However, this solution has obvious limitations: centralized metadata servers are prone to becoming performance bottlenecks, and metadata query latency increases sharply in high-throughput scenarios; static sharding cannot adapt to dynamic loads, and some nodes are overloaded due to data skew, resulting in large fluctuations in write latency; sharding lacks business relevance, and cross-business queries require traversing multiple shards, which is inefficient; the index structure is fixed, and the mixed storage of high-frequency and low-frequency data slows down queries; under high concurrency, metadata node failures can easily trigger system avalanches, making it difficult to guarantee stability. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for storing network security slices in enterprise operations, so as to solve the problems of low system efficiency and poor stability in the existing technology.

[0006] To address the aforementioned technical problems, in a first aspect, this application provides a method for storing network security slice data in enterprise operations, comprising:

[0007] Receive data streams generated during enterprise operations;

[0008] Identify the business type and priority information in the data stream, aggregate data packets with business association into data slice units, and attach a feature identifier to each data slice unit, the feature identifier containing business characteristics and expected processing path;

[0009] Obtain the load status and network connectivity status of the distributed storage cluster, and evaluate the storage location corresponding to each data slice unit by combining the expected processing path and business characteristics of the feature identifier.

[0010] Each data slice unit is stored in its corresponding storage node according to the storage location, and a distributed index directory is established for each data slice unit in the storage node. At the same time, the classification level and level of detail of the index directory are adjusted according to the frequency of data access in the data slice unit.

[0011] Optionally, it also includes:

[0012] When a data query request is received, the intent of the query request is parsed and the associated data slice range is determined. The index directory is then queried based on the data slice range to locate the storage node of the target data.

[0013] The task of filtering and summarizing specific data in the query request is distributed to each storage node for execution. After each storage node completes the data processing locally, the results obtained by all storage nodes are summarized to form the query result.

[0014] Optionally, the load status and network connectivity of the distributed storage cluster are obtained, and the expected processing path and business characteristics of the feature identifier are combined to evaluate the storage location corresponding to each data slice unit, including:

[0015] Periodically monitor the load status and network connectivity of each storage node in the distributed storage cluster;

[0016] Based on the expected processing path of the feature identifier corresponding to each data slice unit, candidate storage nodes located on the network nodes traversed by the expected processing path are selected.

[0017] When the current remaining storage capacity of the candidate storage node is found to be lower than a preset security capacity threshold, or the network latency between the candidate storage node and the data request initiation location is continuously higher than a preset threshold, the candidate storage node is temporarily removed from the candidate list currently being evaluated.

[0018] Based on the load status and network connectivity, and combining the matching degree between the service characteristics of the data slicing unit and the current load mode of the candidate storage nodes, the adaptation score of each candidate storage node corresponding to the data slicing unit is calculated, and the node with the highest adaptation score is selected from the remaining candidate storage nodes as the recommended storage location.

[0019] Optionally, based on the load status and network connectivity, and combining the matching degree between the service characteristics of the data slicing unit and the current load mode of the candidate storage node, a matching score is calculated for each candidate storage node corresponding to the data slicing unit, including:

[0020] Based on the load status and network connectivity status of the distributed storage cluster, a state vector is constructed for each candidate storage node, and a feature vector is constructed for each data slice unit.

[0021] Based on the state vector and the feature vector, the capacity matching value, service matching value and network matching value are calculated respectively. The capacity matching value is the ratio of the current remaining storage capacity of the candidate storage node to the data size of the data slice unit. The service matching value is the degree of matching between the service type identifier of the data slice unit and the current load mode of the candidate storage node. The network matching value is the reciprocal of the network latency between the candidate storage node and the data request initiation location.

[0022] Based on the access priority identifier of the data slice unit, weight coefficients are assigned to the capacity matching value, service matching value and network matching value respectively. The matching value is multiplied by its corresponding weight coefficient and then summed to obtain the adaptation score.

[0023] Optionally, each data slice unit is stored in a corresponding storage node according to the storage location, and a distributed index directory is established for each data slice unit in each storage node. Simultaneously, the classification level and level of detail of the index directory are adjusted according to the frequency of data access in the data slice unit, including:

[0024] According to the storage location, each data slice unit is stored in the corresponding target storage node, and a local index is created for each storage node. The summary information of each local index is synchronized to the central service node to form a distributed index directory.

[0025] The number of times each data slice unit is accessed within a preset period is periodically counted. Based on the number of accesses, the data is divided into high-frequency access data and low-frequency access data by comparing it with preset active thresholds and inactive thresholds.

[0026] Detailed index entries and deep-level classification are used for the high-frequency access data, while simplified index entries and shallow-level classification are used for the low-frequency access data. A fast track is also created for the high-frequency access data.

[0027] Optionally, detailed index entries and deep-level classification are used for the high-frequency access data, while simplified index entries and shallow-level classification are used for the low-frequency access data. A fast track is also created for the high-frequency access data, including:

[0028] In the local index of the storage node corresponding to the high-frequency access data and the corresponding content of the index directory, fine-grained classification labels describing the content and attributes are added, and a multi-level index path pointing to the storage location is established.

[0029] In the local index of the storage node corresponding to the low-frequency access data and the corresponding content of the index directory, the levels to which the data classification tags belong are merged, and the amount of information describing the data content and attributes is reduced.

[0030] For the slice unit of the high-frequency access data, a copy of the index entry is created in memory, and a shortcut path is established from the distributed index directory directly to the storage location.

[0031] Optionally, the service type and priority information in the data stream are identified, data packets with service associations are aggregated into data slice units, and a feature identifier is attached to each data slice unit. The feature identifier contains service characteristics and expected processing paths, including:

[0032] Extract the header information field of each data packet from the data stream, and determine the service type and priority information corresponding to each data packet based on the values ​​of the protocol type, source port identifier and destination port identifier in the header information field;

[0033] Based on the service type and priority information, data packets are divided into different candidate service flow sets, wherein data packets belonging to the same candidate service flow set have the same service type;

[0034] Monitor the time interval between data packets arriving in each candidate service flow set. When the time interval exceeds a preset threshold, aggregate the data packets that arrived before the threshold with the data packets whose time intervals before and after the threshold have not exceeded the preset threshold to form a data slice unit.

[0035] Based on the statistics of header information fields and load information of all data packets in the data slicing unit, the service characteristics of the data slicing unit are determined, and the expected processing path of the data slicing unit is determined according to the service type, priority information, and preset network topology and service rules.

[0036] Generate a corresponding feature identifier for each data slice unit, which includes the business characteristics and the expected processing path.

[0037] Secondly, this application provides a distributed storage system for enterprise-scale operational data, comprising:

[0038] The receiving module is used to receive data streams generated during the enterprise's operations;

[0039] The aggregation module is used to identify the business type and priority information in the data stream, aggregate data packets with business association into data slice units, and attach a feature identifier to each data slice unit. The feature identifier contains business characteristics and expected processing path.

[0040] The evaluation module is used to obtain the load status and network connectivity status of the distributed storage cluster, and evaluate the storage location corresponding to each data slice unit by combining the expected processing path and business characteristics of the feature identifier.

[0041] The adjustment module is used to store each data slice unit to the corresponding storage node according to the storage location, and to establish a distributed index directory for each data slice unit in the storage node. At the same time, it adjusts the classification level and level of detail of the index directory according to the frequency of data access in the data slice unit.

[0042] Thirdly, this application provides an electronic device, comprising:

[0043] Memory, used to store computer programs;

[0044] A processor, used to execute the computer program to implement the steps of the network security slice data storage method in enterprise operations as described in the first aspect above.

[0045] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the network security slice data storage method for enterprise operations as described in the first aspect above.

[0046] The network security slice data storage method for enterprise operations provided in this application can directly receive the core data sources generated during enterprise operations by receiving data streams, providing a foundation for the orderly processing and storage of subsequent data and ensuring the integrity and timeliness of data collection. By identifying the business type and priority information in the data stream, business-related data packets are aggregated into data slice units and attached with feature identifiers containing business characteristics and expected processing paths. This enables data to be classified according to business logic, clarifying the direction and priority of data processing, avoiding disordered data accumulation, and laying the foundation for accurate storage and efficient querying. By obtaining the load status and network connectivity status of the distributed storage cluster and combining the feature identifiers to evaluate the storage location of each data slice unit, dynamic adaptation of storage resources can be achieved, avoiding overload of a single node, while ensuring network transmission efficiency during data storage and improving the overall stability of the system. By storing data slice units to corresponding nodes, establishing a distributed index directory, and adjusting the index classification level and level of detail according to the data access frequency, single-point bottlenecks in the index can be avoided. At the same time, the query path for high-frequency access data is made simpler, and low-frequency access data does not occupy unnecessary index resources, significantly improving data query efficiency.

[0047] Furthermore, upon receiving a data query request, the query intent is first parsed to determine the associated data slice range. Then, based on this range, the distributed index directory is queried to locate the storage node for the target data. Subsequently, specific data filtering and partial aggregation tasks from the query request are distributed to the corresponding storage nodes for execution. After each node completes its local data processing, the processing results from all nodes are aggregated to form the final query result. This step, by parsing the query intent first and using the index to locate storage nodes, avoids blindly scanning the entire dataset, significantly shortening the target data location time. Simultaneously, the distributed execution of data filtering and aggregation tasks allows each node to process only its local data, reducing cross-node data transfer volume, lowering the overall system computational pressure, and ultimately achieving rapid response to query requests. This further improves the query efficiency of enterprise operational data and the system's ability to handle concurrent queries. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a method for storing network security slices in enterprise operations, provided as an embodiment of this application;

[0050] Figure 2 A flowchart illustrating a specific implementation of a network security slice data storage method in enterprise operations, provided in this application embodiment;

[0051] Figure 3 A scenario diagram illustrating a method for storing network security slices in enterprise operations, provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a distributed storage system for enterprise-scale operational data provided in an embodiment of this application. Detailed Implementation

[0053] Enterprise operations require high-throughput, low-latency real-time data streams, placing clear demands on the throughput, latency, query efficiency, and stability of storage systems. However, existing centralized storage solutions combined with static sharding are insufficient to meet these requirements. This approach relies on a single metadata server, which is prone to bottlenecks caused by sudden increases in latency during high-throughput operations. Static sharding cannot adapt to dynamic loads, leading to overload on some nodes due to data skew. Furthermore, the lack of business relevance in sharding means that cross-business queries require traversing multiple shards, and the fixed index structure cannot optimize access efficiency. Under high concurrency, metadata node failures can also trigger system cascading failures. These issues collectively result in insufficient efficiency and stability in data storage and querying.

[0054] To address the aforementioned shortcomings, this application proposes a network security slice data storage method for enterprise operations. This method first receives data streams, identifies the business types and priorities within them, aggregates business-related data packets into slice units, and attaches identifiers containing business characteristics and expected processing paths. Then, considering the load and network status of the distributed storage cluster, it assesses suitable storage locations for each slice unit. Finally, it stores the slice units on the corresponding nodes, establishes a distributed index directory, and adjusts the index hierarchy based on data access frequency. By aggregating slices based on business relationships, the problem of low efficiency in cross-business queries is solved; dynamically allocating storage locations based on cluster status avoids node overload; and the distributed index and dynamic adjustment mechanism eliminate single-point bottlenecks in metadata and optimize query efficiency, ultimately meeting the multi-dimensional needs of enterprise real-time data streams for the storage system.

[0055] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] The core of this application is to provide a method for storing network security slices in enterprise operations, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0057] S101, Receive data streams generated during enterprise operations;

[0058] The data flow of enterprise operations mainly includes three categories: business transaction data, user behavior data, and device perception data. Each type of data carries clear business type and priority information.

[0059] Business transaction data streams are real-time records generated when enterprises carry out their main business. The business type directly indicates the specific business scenario corresponding to the data, and the priority information is divided according to the urgency of the business. This type of data is received through message queues such as Kafka. When the business system generates transaction data, it encapsulates the business type, priority, transaction identifier, amount and other information into JSON format and pushes it to the message queue in real time. The storage system subscribes to the corresponding queue topic and pulls the data in order.

[0060] User behavior data streams are trajectory data generated when users interact with enterprise products. The business type corresponds to the user's specific operation scenario, and the priority is set according to the impact of the behavior on business conversion. This type of data is received through front-end event tracking. Event tracking code is embedded in the product front-end. When a user triggers an interaction, the event tracking code automatically collects the behavior data and combines the business type, priority, user ID, interaction time, and other information, and sends it to the back-end collection server via an asynchronous HTTP request. The server then forwards the data to the storage system.

[0061] Device sensing data streams are sensor data reported by enterprise IoT devices or production equipment. The business type labels the corresponding device application scenario, and the priority is determined based on the urgency of the device status. This type of data is received via the MQTT protocol. The device reports the sensor data along with the business type and priority information to the edge gateway in real time. After simple cleaning, the gateway forwards the data to the storage system according to the business type to ensure accurate data transmission.

[0062] S102. Identify the business type and priority information in the data stream, aggregate data packets with business association into data slice units, and attach a feature identifier to each data slice unit. The feature identifier contains business characteristics and expected processing path.

[0063] Optionally, step S102 may specifically include the following steps:

[0064] S1021. Extract the header information field of each data packet from the data stream, and determine the service type and priority information corresponding to each data packet based on the values ​​of the protocol type, source port identifier and destination port identifier in the header information field.

[0065] S1022. Based on the service type and priority information, the data packets are divided into different candidate service flow sets, wherein data packets belonging to the same candidate service flow set have the same service type;

[0066] S1023. Monitor the time interval between data packets arriving in each candidate service flow set. When the time interval exceeds a preset threshold, aggregate the data packets that are before and after the data packets whose time interval does not exceed the preset threshold to form a data slice unit.

[0067] S1024. Based on the statistics of the header information field and load information of all data packets in the data slicing unit, determine the service characteristics of the data slicing unit, and determine the expected processing path of the data slicing unit according to the service type, priority information, and preset network topology and service rules.

[0068] S1025. Generate a corresponding feature identifier for each data slice unit, which includes the business features and the expected processing path.

[0069] In the above steps, the header information field is the area at the beginning of the data packet used to carry metadata, including key information such as protocol type, source port identifier, and destination port identifier; the protocol type refers to the communication rules followed by the data packet transmission, the source port identifier is the port number corresponding to the device or system sending the data packet, and the destination port identifier is the port number corresponding to the device or system receiving the data packet; the service type refers to the specific business scenario to which the data packet belongs, and the priority information is the level divided according to the urgency of the business; the candidate service flow set refers to the set of data packets with the same service type; the time interval refers to the time difference between the arrival of two consecutive data packets in the same candidate service flow set, and the preset threshold is the maximum time interval for determining whether to stop aggregating data packets; the load information statistics is a summary analysis of the specific business data carried in the data packet, and the business characteristics are the core business attributes of the slice unit extracted based on the header information and load information; the preset network topology refers to the connection structure and positional relationship of each node in the pre-configured distributed storage cluster, and the business rules are the pre-set criteria used to determine the data processing path; the expected processing path refers to the planned transmission and storage path of the data slice unit in the storage cluster; and the feature identifier is a unique identifier generated for each data slice unit, containing the business characteristics and expected processing path of the slice.

[0070] In this embodiment, firstly, the header information field of each data packet is extracted from the data stream in step S1021. Then, the service type and priority information are determined based on the values ​​of the protocol type, source port identifier, and destination port identifier in the header information field. For example, in the data stream of an e-commerce platform, the header information field of a data packet shows that the protocol type is TCP, the source port identifier is 8080 corresponding to the platform's order system's dedicated port, and the destination port identifier is 9090 corresponding to the storage system's receiving port. Combining this with the preset rule that TCP protocol + port 8080 corresponds to order service, the service type of this data packet can be determined as "order processing". At the same time, because order data needs to be processed in real time, the priority information is set to "high". Another data packet shows that the protocol type is HTTP and the source port identifier is 80 corresponding to the platform's front-end page port, so its service type is determined to be "product browsing", and the priority information is set to "low".

[0071] Secondly, in step S1022, based on the determined business type and priority information, the data packets are divided into different candidate business flow sets, and the data packets in the same candidate business flow set have the same business type. For example, all data packets with the business type "order processing" are uniformly divided into the "order candidate business flow set" regardless of priority, and all data packets with the business type "product browsing" are divided into the "browsing candidate business flow set," ensuring that data packets in the same set serve the same business scenario, which facilitates subsequent aggregation processing.

[0072] Next, step S1023 monitors the time interval between data packets arriving in each candidate business flow set. When the time interval exceeds a preset threshold, all data packets whose time intervals with the preceding and following data packets do not exceed the preset threshold are aggregated to form a data slice unit. For example, if the preset time threshold is 3 seconds, when monitoring the "order candidate business flow set", the time intervals of the first 5 data packets arriving are 1 second, 1.5 seconds, 2 seconds, and 2.5 seconds, respectively, all of which do not exceed the threshold. However, the time interval between the 5th and 6th data packets is 5 seconds, which exceeds the threshold. At this time, the first 5 data packets are aggregated to form an "order-type data slice unit". In the "browsing candidate business flow set", the time interval between data packets is mostly 2 seconds. When a certain interval reaches 4 seconds, the previously qualified data packets are also aggregated to form a "browsing-type data slice unit".

[0073] Then, in step S1024, the business characteristics are determined based on the header information fields and load information of all data packets within the data slice unit. The expected processing path is then determined according to the business type, priority information, and pre-defined network topology and business rules. For example, in the "order-related data slice unit," the header information is all for the "order processing" business type, and the load information statistics show data such as order number, payment amount, and user ID. Therefore, the business characteristic is determined to be "payment-related order data." Combined with the pre-defined business rules that prioritize the transmission of payment-related data to the core node of the storage cluster, the network topology that core node A is located in a low-load area of ​​the cluster, and the fact that this slice has a "high" priority, the expected processing path is determined to be "receiving module → core node A." The business characteristic of the "browsing-related data slice unit" is "non-transactional browsing records," with a "low" priority; the expected processing path is then determined to be "receiving module → edge node B."

[0074] Finally, step S1025 generates a corresponding feature identifier for each data slice unit, containing business characteristics and expected processing paths. For example, the feature identifier "Business characteristic: payment-related order data; expected processing path: receiving module → core node A" is generated for the "order-type data slice unit", and the feature identifier "Business characteristic: non-transactional browsing records; expected processing path: receiving module → edge node B" is generated for the "browsing-type data slice unit", ensuring that the core information of each slice unit can be quickly identified by subsequent storage steps.

[0075] In practical applications, during the annual promotional period of e-commerce platform A, it is necessary to process operational data streams such as user orders and product browsing in real time. The platform first extracts the header information field of each data packet: if the header shows TCP protocol, source port 8080, and destination port 9090, the business type is determined to be "order processing" based on preset rules, and the priority is set to "high" because it requires real-time inventory locking and payment verification; if it is HTTP protocol and source port 80, it is determined to be "product browsing", and the priority is set to "low". Subsequently, the platform groups "order processing" data packets into one set and "product browsing" data packets into another set to ensure that data in the same set corresponds to the same business. Next, the arrival time of data packets in each set is monitored, with a preset interval threshold of 3 seconds: in the "order processing" set, the arrival interval of the first few data packets is within 3 seconds. When the interval between two data packets reaches 5 seconds, the previous data packets are aggregated into a data slice unit; the same applies to the "product browsing" set, where slices are aggregated when the interval exceeds the threshold. Next, the sliced ​​data is analyzed: the "Order Processing" slice contains order numbers, payment amounts, etc., and its business characteristic is extracted as "payment-related order data." Combined with the pre-defined cluster topology and rules, the processing path is determined to be "receiving module → core node A"; the "Product Browsing" slice's characteristic is "non-transaction behavior records," and the path is defined as "receiving module → edge node B." Finally, a unique identifier is generated for each slice, marking its business characteristics and processing path, facilitating subsequent storage scheduling.

[0076] In the overall scheme of step S102 above, the business type and priority are accurately identified by extracting header information to ensure the accuracy of data classification; the preliminary organization of similar data is achieved by dividing the candidate business flow set; the scale of data slices is controlled by the time interval threshold to ensure the rationality of the slice unit; the expected processing path is determined by analyzing business characteristics and combining preset rules, and feature identifiers are generated to achieve clear identification of slice information. Overall, the transformation of data from disorder to order and from scattered to aggregated is realized, laying the foundation for the efficient execution of subsequent distributed storage, while improving the targeting and coherence of data processing.

[0077] S103. Obtain the load status and network connectivity status of the distributed storage cluster, and evaluate the storage location corresponding to each data slice unit by combining the expected processing path and business characteristics of the feature identifier.

[0078] Optionally, such as Figure 2 As shown, step S103 may specifically include the following steps:

[0079] S1031. Periodically monitor the load status and network connectivity status of each storage node in the distributed storage cluster.

[0080] S1032. Based on the expected processing path of the feature identifier corresponding to each data slice unit, filter the candidate storage nodes located on the network nodes traversed by the expected processing path.

[0081] S1033. When it is identified that the current remaining storage capacity of the candidate storage node is lower than a preset security capacity threshold, or the network latency between the candidate storage node and the data request initiation location is continuously higher than a preset threshold, the candidate storage node is temporarily removed from the candidate list currently being evaluated.

[0082] S1034. Based on the load status and network connectivity status, and combining the matching degree between the service characteristics of the data slicing unit and the current load mode of the candidate storage node, calculate the adaptation score of each candidate storage node corresponding to the data slicing unit, and select the node with the highest adaptation score from the remaining candidate storage nodes as the recommended storage location.

[0083] Specifically, step S1034 may include: constructing a state vector for each candidate storage node and a feature vector for each data slicing unit based on the load status and network connectivity status of the distributed storage cluster; calculating a capacity matching value, a service matching value, and a network matching value based on the state vector and the feature vector, wherein the capacity matching value is the ratio of the current remaining storage capacity of the candidate storage node to the data size of the data slicing unit, the service matching value is the degree of matching between the service type identifier of the data slicing unit and the current load mode of the candidate storage node, and the network matching value is the reciprocal of the network latency between the candidate storage node and the data request initiation location; assigning weight coefficients to the capacity matching value, service matching value, and network matching value according to the access priority identifier of the data slicing unit, and multiplying each matching value by its corresponding weight coefficient and summing the results to obtain an adaptation score.

[0084] In the above steps, the load status of the distributed storage cluster refers to the resource utilization of each storage node within the cluster, specifically including CPU utilization, memory utilization, and remaining storage capacity; network connectivity status refers to the stability of network connections, data transmission rates, and network latency between storage nodes and between nodes and the location initiating the data request; candidate storage nodes are nodes with storage capabilities selected from the network nodes along the expected processing path; the safety capacity threshold is a pre-set minimum remaining storage capacity for a node, and new data should not be received when the remaining capacity of a node is below this value; the network latency threshold is a pre-set maximum network latency between a node and the location initiating the data request, and exceeding this value will affect data access efficiency; the load mode refers to the type of data service and resource consumption characteristics that the node is currently processing, such as some nodes frequently processing transaction data, while others frequently process browsing data. According to the data slice unit, the matching score is a comprehensive indicator that measures the degree of matching between the candidate node and the data slice unit; the state vector is a vector formed by integrating the key indicators of the candidate node's load status and network connectivity status, which can be represented as [CPU utilization, remaining storage capacity, network latency]; the feature vector is a vector formed by integrating the data volume, service type, and access priority of the data slice unit; the capacity matching value is the ratio of the remaining storage capacity of the candidate node to the data volume of the data slice unit, used to measure the degree of capacity matching; the service matching value is the degree of matching between the service type of the data slice unit and the load mode of the candidate node, and the higher the matching degree, the larger the value; the network matching value is the reciprocal of the network latency between the candidate node and the location where the data request is initiated, and the lower the network latency, the larger the value; the weight coefficient is the importance coefficient assigned to each matching value according to the access priority of the data slice unit, and the coefficient ratio is different for different priorities.

[0085] In this embodiment, firstly, a periodic monitoring mechanism is adopted in step S1031. Using the SNMP protocol or a cluster monitoring tool like Zabbix, the load status and network connectivity of each storage node in the distributed storage cluster are collected. The monitoring period can be flexibly set according to business needs to ensure timely understanding of the real-time dynamics of each node. For example, during a major promotional period, e-commerce platform A sets the monitoring period to once every 30 seconds. Each monitoring session collects load status data such as CPU utilization, memory usage, and remaining storage capacity of each node. Simultaneously, it collects network connectivity data such as network latency and packet loss rate between the node and the location where the user request originated. After collection, all data is uploaded in real-time to the cluster management module for unified storage and management.

[0086] Secondly, in step S1032, the expected processing path of the corresponding data slice unit is retrieved from the cluster management module. Then, all network nodes along the path are traversed, and nodes with data storage capabilities are selected. These selected nodes are the candidate storage nodes. For example, for a data slice unit of the "order processing" type, the expected processing path is from the receiving module to core node A and then to core node B. When traversing this path, it is found that core node A and core node B both have data storage capabilities. Therefore, these two nodes are identified as candidate storage nodes. Edge nodes C and D, which are outside the path, are excluded from the candidate range because they are not on the expected processing path. This effectively reduces path detours during data transmission and reduces transmission consumption.

[0087] Next, in step S1033, real-time monitoring data of candidate storage nodes is retrieved from the cluster management module. The remaining storage capacity of each candidate node is compared with a preset safety capacity threshold, and the network latency between each candidate node and the data request initiation location is compared with a preset network latency threshold. If the remaining storage capacity of a candidate node is lower than the safety capacity threshold, or its network latency is consistently higher than the network latency threshold, the node is temporarily removed from the current candidate list. For example, the preset safety capacity threshold of e-commerce platform A is 20% of the total storage capacity of the nodes, and the network latency threshold is 50 milliseconds. When checking candidate nodes A and B, it is found that the remaining storage capacity of node A is only 15% of its total capacity, which is lower than the safety capacity threshold, while the remaining storage capacity of node B is 25% of the total capacity and the network latency is 45 milliseconds, both of which meet the threshold requirements. Therefore, node A is removed from the candidate list, and only node B is retained as a candidate storage node.

[0088] Finally, in step S1034, a state vector is constructed for each remaining candidate storage node, with elements representing key indicators of the node's load status and network connectivity. Simultaneously, a feature vector is constructed for the current data slice unit, with elements representing the slice's data volume, service type, and access priority. Next, three key matching values ​​are calculated: the capacity matching value is the ratio of the candidate node's current remaining storage capacity to the data volume of the data slice unit; the service matching value is determined based on the degree of matching between the data slice unit's service type and the candidate node's load pattern, with higher matching values ​​resulting in larger service matching values; and the network matching value is the reciprocal of the network latency between the candidate node and the location where the data request originated. Then, based on the data slice unit's access priority, corresponding weight coefficients are assigned to the capacity matching value, service matching value, and network matching value, respectively. Finally, the Score is calculated according to the formula... The algorithm calculates the fit score for each candidate node and selects the node with the highest fit score as the recommended storage location for that data slice unit. Here, "Score" represents the fit score. Represents the capacity matching value. The weight coefficients representing the capacity matching values. Represents the business matching value. This represents the weighting coefficient corresponding to the business matching value. Represents the network matching value. This represents the weight coefficient corresponding to the network matching value. For example, a high-priority "order processing" slice has a data size of 10GB, candidate node B has a current remaining storage capacity of 100GB, a network latency of 45 milliseconds to the request initiation location, and the node... The load mode primarily processes order-related data, and the business matching value is set at 0.9. First, three matching values ​​are calculated: capacity matching value... for Network matching value for Then, based on the weighting rules for high-priority data, the capacity weighting coefficient... Set to 0.3, business weight coefficient Set to 0.3, network weight coefficient Set it to 0.4, and substitute it into the formula to calculate the fit score. Finally, node B was determined as the recommended storage location for the "order processing" slice.

[0089] In practical applications, when e-commerce platform A processes a data slice of the "order processing" type during a major promotional period, it first monitors the load and network connectivity of all storage nodes in the cluster every 30 seconds and uploads the monitoring data to the cluster management module. Then, based on the expected processing path of the slice, core node A and core node B are selected as candidate storage nodes. Next, by comparing thresholds, node A is found to have insufficient remaining storage capacity and is removed from the candidate list, retaining only node B. Afterwards, a state vector is constructed for node B, and a feature vector is constructed for the slice. Capacity matching value, business matching value, and network matching value are calculated, and then combined with high-priority weight coefficients to calculate an adaptation score. Finally, node B is determined as the storage location. After the slice is stored on node B, the platform immediately updates the load data of node B to ensure that subsequent data slice evaluations are based on the latest node status data, guaranteeing the accuracy of the evaluation results.

[0090] In the overall solution of step S103 above, a periodic monitoring mechanism ensures real-time monitoring of storage node status changes, candidate nodes are selected based on the expected processing path to effectively reduce resource consumption during data transmission, and substandard nodes are removed through threshold comparison to ensure basic data storage conditions. Multi-dimensional calculation of the adaptation score achieves precise matching between data slices and storage nodes. The overall process avoids performance degradation due to storage node overload and prevents data access experience from being affected by excessive network latency, ensuring that each data slice unit is stored on the most suitable node, effectively improving data access efficiency and the overall operational stability of the distributed storage cluster.

[0091] S104. Store each data slice unit to the corresponding storage node according to the storage location, and establish a distributed index directory for each data slice unit in the storage node. At the same time, adjust the classification level and level of detail of the index directory according to the frequency of data access in the data slice unit.

[0092] Optionally, step S104 may specifically include the following steps:

[0093] S1041. According to the storage location, each data slice unit is stored in the corresponding target storage node, and a local index is created for each storage node. The summary information of each local index is synchronized to the central service node to form a distributed index directory.

[0094] S1042. Periodically count the number of times each data slice unit is accessed within a preset period, and divide the data into high-frequency access data and low-frequency access data based on the number of accesses compared with preset active thresholds and inactive thresholds.

[0095] S1043. Detailed index entries and deep-level classification are used for the high-frequency access data, and simplified index entries and shallow-level classification are used for the low-frequency access data. A shortcut channel is created for the high-frequency access data.

[0096] Specifically, step S1043 may include the following processes: adding fine-grained classification labels describing the content and attributes to the local index of the storage node corresponding to the high-frequency access data and the corresponding content of the index directory, and establishing a multi-level index path pointing to the storage location; merging the levels to which the data classification labels belong to the local index of the storage node corresponding to the low-frequency access data and the corresponding content of the index directory, and reducing the amount of information describing the data content and attributes; creating a copy of the index entry for the slice unit of the high-frequency access data in memory, and establishing a shortcut path directly pointing from the distributed index directory to the storage location.

[0097] In the above steps, the local index is an index built by each storage node for its own stored data slice units, recording basic attributes such as the slice's storage path and business type; the central service node is the core node used to aggregate the local index summary information of each storage node, not storing complete data but only synchronizing key summaries; the distributed index directory is a global index formed by the central service node integrating all local index summaries, which can quickly locate the storage node of any slice; the preset period is a fixed time interval for statistical data access counts; the number of times a data slice unit is accessed is the total number of times a data slice unit is queried or called within the preset period; the active threshold is a pre-set minimum number of times data is considered frequently accessed; and the inactive threshold is a pre-set number of times data is considered infrequently accessed. The most frequently accessed data is categorized into three levels: high-frequency access data (data accessed more than the active threshold, requiring rapid response to access requests) and low-frequency access data (data accessed less than the inactive threshold, requiring lower access speed). Fine-grained classification tags are labels used to describe the content and attributes of high-frequency access data in detail, such as business type, time range, and data format. Multi-level index paths are multi-level index structures built for high-frequency data, which can gradually narrow down the positioning scope. Simplified index entries are index records after deleting non-critical information from low-frequency data. Shallow classification reduces the number of index levels for low-frequency data, reducing index complexity. Fast paths are access paths built for high-frequency data that directly point to the storage location, including in-memory index copies and direct paths.

[0098] In this embodiment of the application, firstly, in step S1041, each data slice unit is sent to the corresponding target storage node via TCP / IP distributed transmission protocol according to the evaluated storage location. After receiving the slice, each target storage node establishes a local index for all slices stored therein. The local index records basic attributes such as the specific storage path and business type of the slice. Subsequently, each storage node synchronizes the summary information of the local index, such as the unique identifier of the slice, core attributes, and storage node ID, to the central service node. The central service node integrates and associates the summary information synchronized by all nodes, and finally forms a distributed index directory covering the entire cluster. For example, in e-commerce platform A, the slice for order processing is stored on core node B. After the slice is transmitted to core node B via TCP / IP, node B creates a local index for it, and the record storage path is the 20240510 subdirectory in the order data directory under core node B, with the business type being order processing. The slice for product browsing is stored on edge node C, and node C also creates a local index for it. Then, nodes B and C synchronize the summary information of their local indexes to the central service node. After integration by the central service node, a distributed index directory is formed. Through this directory, one can directly query whether the slice for order processing is on core node B or the slice for product browsing is on edge node C.

[0099] Secondly, by setting a fixed preset period in step S1042, and using a sliding window-based access log statistics algorithm, the number of times each data slice unit is accessed within the preset period is periodically counted. During the statistics, all query request logs for that slice are captured and counted. After the statistics are completed, the number of times each slice is accessed is compared with preset active thresholds and inactive thresholds. Slices with access counts higher than the active threshold are classified as high-frequency access data, and slices with access counts lower than the inactive threshold are classified as low-frequency access data. For example, e-commerce platform A sets the preset period to 1 hour, the active threshold to 50 times, and the inactive threshold to 10 times. The system captures query logs within 1 hour using the sliding window algorithm and finds that the order processing type slice is queried 80 times, which is higher than the active threshold, and is classified as high-frequency access data; the product browsing type slice is queried only 5 times, which is lower than the inactive threshold, and is classified as low-frequency access data.

[0100] Finally, in step S1043, differentiated indexing methods are applied to high-frequency and low-frequency access data. For high-frequency access data, fine-grained classification tags are added to the corresponding content of the local index and distributed index directory of its storage node. For example, under the order processing type, pre-sale orders, the time range of May 2024, and JSON data format are added. At the same time, a multi-level index path is established, with the path structure being from the distributed index directory to the core node B, then to the order data, then to the 20240510 subdirectory, and finally to the pre-sale order category. Simultaneously, a copy of the index entry is created in the memory of the storage node for the slice unit of high-frequency access data, and a shortcut path is established from the distributed index directory directly to the storage location of the slice. For low-frequency access data, the hierarchy of data classification tags is merged in the corresponding content of the local index and distributed index directory of its storage node. For example, the three-level categories of clothing, tops, and shirts under the product browsing type are merged into a two-level category of clothing and shirts under the product browsing type. At the same time, the amount of information describing the data content and attributes is reduced, retaining only key information such as business type, storage node ID, and storage time, and removing redundant descriptions. For example, the order processing type slice of e-commerce platform A is high-frequency data. Fine-grained tags are added to the local index and distributed directory of core node B, and multi-level paths are established. At the same time, index copies and shortcut paths are created in memory. The product browsing type slice is low-frequency data. The local index and distributed directory of edge node C merge the category hierarchy, reduce the descriptive information, and retain only the core content.

[0101] In practical applications, when processing order processing and product browsing data slices during a major promotional period, e-commerce platform A first stores the order processing slice in the core node B and the product browsing slice in the edge node C, based on their storage locations. Each node creates a local index for its respective slice and synchronizes the index summary to the central service node, which then integrates them to form a distributed index directory. Next, at a preset 1-hour cycle, the access frequency of the two types of slices is counted using a sliding window algorithm. The order processing slice, accessed 80 times, is classified as high-frequency data, while the product browsing slice, accessed 5 times, is classified as low-frequency data. Finally, fine-grained classification labels are added to the high-frequency order processing slice, multi-level index paths are constructed, and index copies are created in memory. For the low-frequency product browsing slice, classification levels are merged and descriptive information is reduced. This ensures that order data can be quickly queried while avoiding excessive index resource consumption by browsing data.

[0102] In the overall solution of step S104 above, the precise storage of data slices according to storage location ensures the orderliness and accuracy of data storage; the construction of the distributed index directory avoids the performance bottleneck of a single index node and realizes rapid location of global data; high-frequency and low-frequency data are distinguished by periodic statistics and threshold comparison, and then differentiated index processing is adopted, so that high-frequency data can be quickly accessed through detailed indexes and shortcuts, while low-frequency data can save resources by simplifying the index; the whole solution not only improves data access efficiency but also optimizes cluster resource allocation, meets the data processing requirements of different access needs in enterprise operations, and ensures the high efficiency and stability of the storage system.

[0103] The following is a complete embodiment for steps S101 to S104:

[0104] like Figure 3 As shown, e-commerce platform A needs to process massive amounts of operational data in its daily operations. It uses the above-mentioned distributed storage method to manage the data: First, it receives various data streams generated by the platform through the Kafka message queue, including order data generated by users placing orders, behavioral data generated by browsing products, temperature and humidity data reported by logistics equipment, etc., to ensure that all operational data is completely connected to the storage system.

[0105] Next, the system identifies the business type and priority of various data streams—order data is classified as "transaction" with "high" priority, browsing data is classified as "behavioral" with "medium" priority, and device data is classified as "sensor" with "low" priority. Subsequently, data packets of the same business type are aggregated, such as aggregating order data packets from the same time period into "transaction data slice units," and a feature identifier is attached to this slice, in which the business feature is marked "order payment related," and the expected processing path is set to "pointing to the cluster core storage area." Other types of slices are also attached with corresponding identifiers according to this logic.

[0106] Next, the system used the Zabbix monitoring tool to obtain the load and network connectivity status of the distributed storage cluster. It found that core node C had sufficient remaining storage capacity and low network latency, while edge node D had a relatively light load. Based on the slice characteristic identifiers—"transaction" slices were assigned to core node C according to the expected path, "behavioral" slices to edge node D, and "sensor" slices to another edge node E—the system stored each slice on its corresponding node and simultaneously created a distributed index directory for all nodes. After analyzing access frequency, it was found that "transaction" slices had a high average daily access frequency, so its index classification hierarchy was refined to "date-order status-user level," with added attribute descriptions. "Sensor" slices had a low access frequency, so the index hierarchy was simplified to "device type-date," reducing redundant information and achieving reasonable allocation of index resources.

[0107] The network security slicing data storage method for enterprise operations provided in this application avoids disordered data accumulation and makes data management more targeted by aggregating data according to business type and attaching feature identifiers. It combines cluster load and network status to assess storage location, effectively avoiding single-node overload, balancing cluster resource usage, and improving storage stability. By establishing a distributed index and adjusting the hierarchy according to access frequency, it simplifies the query path for high-frequency data, improving access efficiency, while preventing low-frequency data from consuming excessive index resources, thus achieving resource optimization. Overall, it solves the problems of data chaos, uneven node load, and low query efficiency in traditional storage solutions, stably supporting the storage and retrieval of enterprise-scale operational data, and meeting the enterprise's needs for efficient and stable data management.

[0108] Optionally, the method further includes: when a data query request is received, parsing the intent of the query request and determining the associated data slice range, querying the index directory according to the data slice range to locate the storage node of the target data; distributing the specific data filtering and local summarization tasks in the query request to each storage node for execution, and after each storage node completes data processing locally, summarizing the results obtained by all storage nodes to form the query result.

[0109] In the above steps, a data query request is a user's or business system's instruction to obtain specific enterprise operational data, including information such as query objectives and conditions; query intent is the core requirement extracted from the query request, such as obtaining order amounts within a certain time period or browsing history of a specific user; data slice range is the collection of all data slice units associated with the query intent, clearly defining the slice boundaries to be covered by the query; specific data filtering is filtering records that meet the requirements from the data according to query conditions, such as filtering orders in a certain status; partial aggregation task is performing statistical calculations on the filtered data within a single storage node, such as calculating the total amount of data that meets the conditions within the node; local data processing is the process of filtering and summarizing within the storage node's own storage scope, without transferring the full data to other nodes; result aggregation is the operation of integrating the partial processing results of all storage nodes into a complete query result.

[0110] Specifically, when a data query request is received, the system first extracts key information from the request using a semantic parsing algorithm to clarify the query intent. Then, based on the intent, it associates the stored data slice unit attributes to determine the corresponding data slice range. For example, if e-commerce platform A receives a request to "obtain pre-sale order information for a specific time period," semantic parsing clarifies that the query intent is to obtain pre-sale orders for a specific time period. Combining the business type and time attributes of the data slice unit, it determines all order-related data slices for the associated specific time period, forming the data slice range. Next, based on the determined data slice range, it queries the distributed index directory. Through the correspondence between slices and storage nodes recorded in the directory, it locates the target storage nodes storing these slices. For example, the distributed index directory shows that the aforementioned order slices are all stored on core node B. Afterward, the specific data filtering conditions and partial aggregation tasks in the query request are sent to all target storage nodes through a task distribution mechanism. After receiving the task, each storage node performs filtering and aggregation processing operations on its own stored slice data locally, without transmitting data to other nodes. For example, core node B locally filters out pre-sale orders that meet the conditions and counts the quantity. Finally, the local processing results of all target storage nodes are collected, and the results of each node are merged into a complete query result through a result integration algorithm, which is then fed back to the user or system that initiated the request.

[0111] In practical application, the operations staff of e-commerce platform A initiated a query request: "Retrieve the total order amount for apparel products from VIP users within a certain time period." The platform first analyzes the intent of the request, determining that the core requirement is to calculate the total order amount for a specific time period, a specific user level, and a specific product category. It then associates the data slice attributes, determining that the data slice range is all data slices marked as apparel orders and VIP user orders within a certain time period. Next, it queries the distributed index directory and finds that these slices are stored on core nodes C and D respectively. Then, the filtering task for orders with VIP user level and apparel product category, and the summarization task for calculating the total order amount after filtering, are distributed to core nodes C and D. Each node filters the order data that meets the criteria locally. Core node C calculates the local order amount that meets the criteria as X, and core node D calculates the local amount as Y. X and Y are then uploaded to the result summarization module. The summarization module adds X and Y to obtain the target total order amount for that time period and finally feeds the result back to the operations staff.

[0112] In the overall solution described above, the precise data slice range is determined by first parsing the query intent, avoiding blind scanning of the entire dataset and significantly narrowing the query scope. The distributed index directory quickly locates storage nodes, reducing node lookup time. Filtering and summarizing tasks are distributed to each storage node for local execution, avoiding large-scale data transfer across nodes and reducing network load and data processing latency. The results are summarized to obtain complete data, ensuring both the accuracy of query results and efficient handling of multi-node concurrent processing scenarios. This overall process improves data query response speed, enhances system stability under high query volume scenarios, and reduces resource consumption, meeting enterprises' needs for rapid acquisition and analysis of operational data.

[0113] Figure 4 This application provides a schematic diagram of the structure of a distributed storage system for enterprise-scale operational data, as shown in the embodiments of the present application. Figure 4 The system may include:

[0114] Receiver module 41 is used to receive data streams generated during enterprise operations;

[0115] The aggregation module 42 is used to identify the business type and priority information in the data stream, aggregate data packets with business association into data slice units, and attach a feature identifier to each data slice unit. The feature identifier contains business characteristics and expected processing path.

[0116] Evaluation module 43 is used to obtain the load status and network connectivity status of the distributed storage cluster, and evaluate the storage location corresponding to each data slice unit by combining the expected processing path and business characteristics of the feature identifier.

[0117] The adjustment module 44 is used to store each data slice unit to the corresponding storage node according to the storage location, and to establish a distributed index directory for each data slice unit in the storage node. At the same time, it adjusts the classification level and level of detail of the index directory according to the frequency of data access in the data slice unit.

[0118] The distributed storage system for enterprise-scale operational data in this application is used to implement the aforementioned method for storing network security slices in enterprise operations. Therefore, the specific implementation of the distributed storage system for enterprise-scale operational data can be found in the embodiment section of the method for storing network security slices in enterprise operations above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0119] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the network security slice data storage method for enterprise operations described above.

[0120] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for storing network security slices in enterprise operations.

[0121] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0122] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the network security slice data storage method in enterprise operations.

[0123] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0124] The foregoing has provided a detailed description of a network security slice data storage method, system, electronic device, and storage medium for enterprise operations provided by this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for storing network security slices in enterprise operations, characterized in that, include: Receive data streams generated during enterprise operations; Identify the business type and priority information in the data stream, aggregate data packets with business association into data slice units, and attach a feature identifier to each data slice unit, the feature identifier containing business characteristics and expected processing path; Obtain the load status and network connectivity status of the distributed storage cluster, and evaluate the storage location corresponding to each data slice unit by combining the expected processing path and business characteristics of the feature identifier. Each data slice unit is stored in its corresponding storage node according to the storage location, and a distributed index directory is established for each data slice unit in the storage node. At the same time, the classification level and level of detail of the index directory are adjusted according to the frequency of data access in the data slice unit.

2. The method according to claim 1, characterized in that, Also includes: When a data query request is received, the intent of the query request is parsed and the associated data slice range is determined. The index directory is then queried based on the data slice range to locate the storage node of the target data. The task of filtering and summarizing specific data in the query request is distributed to each storage node for execution. After each storage node completes the data processing locally, the results obtained by all storage nodes are summarized to form the query result.

3. The method according to claim 1, characterized in that, Obtain the load status and network connectivity of the distributed storage cluster, and, in conjunction with the expected processing path and business characteristics of the aforementioned feature identifiers, evaluate the storage location corresponding to each data slice unit, including: Periodically monitor the load status and network connectivity of each storage node in the distributed storage cluster; Based on the expected processing path of the feature identifier corresponding to each data slice unit, candidate storage nodes located on the network nodes traversed by the expected processing path are selected. When the current remaining storage capacity of the candidate storage node is found to be lower than a preset security capacity threshold, or the network latency between the candidate storage node and the data request initiation location is continuously higher than a preset threshold, the candidate storage node is temporarily removed from the candidate list currently being evaluated. Based on the load status and network connectivity, and combining the matching degree between the service characteristics of the data slicing unit and the current load mode of the candidate storage nodes, the adaptation score of each candidate storage node corresponding to the data slicing unit is calculated, and the node with the highest adaptation score is selected from the remaining candidate storage nodes as the recommended storage location.

4. The method according to claim 3, characterized in that, Based on the load conditions and network connectivity status, and combining the matching degree between the service characteristics of the data slicing unit and the current load mode of the candidate storage nodes, a suitability score is calculated for each candidate storage node corresponding to the data slicing unit, including: Based on the load status and network connectivity status of the distributed storage cluster, a state vector is constructed for each candidate storage node, and a feature vector is constructed for each data slice unit. Based on the state vector and the feature vector, the capacity matching value, service matching value and network matching value are calculated respectively. The capacity matching value is the ratio of the current remaining storage capacity of the candidate storage node to the data size of the data slice unit. The service matching value is the degree of matching between the service type identifier of the data slice unit and the current load mode of the candidate storage node. The network matching value is the reciprocal of the network latency between the candidate storage node and the data request initiation location. Based on the access priority identifier of the data slice unit, weight coefficients are assigned to the capacity matching value, service matching value and network matching value respectively. The matching value is multiplied by its corresponding weight coefficient and then summed to obtain the adaptation score.

5. The method according to claim 1, characterized in that, Each data slice unit is stored in its corresponding storage node according to its storage location, and a distributed index directory is established for each data slice unit in each storage node. Simultaneously, the classification hierarchy and level of detail of the index directory are adjusted according to the frequency of data access within each data slice unit, including: According to the storage location, each data slice unit is stored in the corresponding target storage node, and a local index is created for each storage node. The summary information of each local index is synchronized to the central service node to form a distributed index directory. The number of times each data slice unit is accessed within a preset period is periodically counted. Based on the number of accesses, the data is divided into high-frequency access data and low-frequency access data by comparing it with preset active thresholds and inactive thresholds. Detailed index entries and deep-level classification are used for the high-frequency access data, while simplified index entries and shallow-level classification are used for the low-frequency access data. A fast track is also created for the high-frequency access data.

6. The method according to claim 5, characterized in that, Detailed index entries and deep-level classification are used for the high-frequency access data, while simplified index entries and shallow-level classification are used for the low-frequency access data. A fast track is created for the high-frequency access data, including: In the local index of the storage node corresponding to the high-frequency access data and the corresponding content of the index directory, fine-grained classification labels describing the content and attributes are added, and a multi-level index path pointing to the storage location is established. In the local index of the storage node corresponding to the low-frequency access data and the corresponding content of the index directory, the levels to which the data classification tags belong are merged, and the amount of information describing the data content and attributes is reduced. For the slice unit of the high-frequency access data, a copy of the index entry is created in memory, and a shortcut path is established from the distributed index directory directly to the storage location.

7. The method according to claim 1, characterized in that, Identify the service type and priority information in the data stream, aggregate service-related data packets into data slice units, and attach a feature identifier to each data slice unit. The feature identifier contains service characteristics and the expected processing path, including: Extract the header information field of each data packet from the data stream, and determine the service type and priority information corresponding to each data packet based on the values ​​of the protocol type, source port identifier and destination port identifier in the header information field; Based on the service type and priority information, data packets are divided into different candidate service flow sets, wherein data packets belonging to the same candidate service flow set have the same service type; Monitor the time interval between data packets arriving in each candidate service flow set. When the time interval exceeds a preset threshold, aggregate the data packets that arrived before the threshold with the data packets whose time intervals before and after the threshold have not exceeded the preset threshold to form a data slice unit. Based on the statistics of header information fields and load information of all data packets in the data slicing unit, the service characteristics of the data slicing unit are determined, and the expected processing path of the data slicing unit is determined according to the service type, priority information, and preset network topology and service rules. Generate a corresponding feature identifier for each data slice unit, which includes the business characteristics and the expected processing path.

8. A distributed storage system for enterprise-scale operational data, characterized in that, include: The receiving module is used to receive data streams generated during the enterprise's operations; The aggregation module is used to identify the business type and priority information in the data stream, aggregate data packets with business association into data slice units, and attach a feature identifier to each data slice unit. The feature identifier contains business characteristics and expected processing path. The evaluation module is used to obtain the load status and network connectivity status of the distributed storage cluster, and evaluate the storage location corresponding to each data slice unit by combining the expected processing path and business characteristics of the feature identifier. The adjustment module is used to store each data slice unit to the corresponding storage node according to the storage location, and to establish a distributed index directory for each data slice unit in the storage node. At the same time, it adjusts the classification level and level of detail of the index directory according to the frequency of data access in the data slice unit.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the network security slice data storage method for enterprise operations as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the network security slice data storage method for enterprise operations as described in any one of claims 1 to 7.

Images