Trusted data operation service management system based on micro-service architecture

Abstract

The application discloses a trusted data operation service management system based on a micro-service architecture, relates to the technical field of data operation service management, and comprises the following steps: a request analysis module receives an operation service request, analyzes a data label and a call chain context; a feature generation module generates global data features according to the data label and the call chain context; a strategy engine module receives the global data features, performs hierarchical feature mapping, and obtains multi-level strategy matching results; an access arbitration module compares the multi-level strategy matching results, allocates temporary access rights and resource quotas; an index collection module collects node call indexes in real time and feeds back the node call indexes to a micro-service scheduler; and a scheduler module adjusts data routing weights of subsequent operation service requests according to the node call indexes. Through the construction of global data features and a hierarchical strategy mapping mechanism, combined with routing adjustment driven by runtime indexes, fine-grained access control and dynamic load balancing are realized.

Description

Technical Field

[0001] This invention relates to the field of data operation service management technology, specifically a trusted data operation service management system based on a microservice architecture. Background Technology

[0002] In a microservice-based data operations environment, data service requests initiated by different business units often involve data flow across multiple microservice nodes. The data tags and call chain context within these requests are scattered across different locations in the request header and body. Existing access control schemes typically perform static matching on single-dimensional tags in the request, treating data tags and call context as independent fields and processing them separately, without integrating them into a unified feature identifier that represents the global attributes of the request. This approach prevents policy matching from comprehensively considering the combined relationships between data owner, data sensitivity level, usage purpose, upstream service identifier, call timestamp, and call depth. Policy determination is coarse-grained, making it difficult to achieve fine-grained authorization and resource management for complex operational scenarios. Furthermore, existing scheduling mechanisms are mostly based on fixed polling or random weight allocation, failing to incorporate the real-time processing status of each microservice node during data service calls into routing decisions. Due to the lack of dynamic metrics such as request processing time and request queue length, when some nodes experience increased processing latency or queue accumulation, the service gateway continues to distribute requests to these nodes, causing node overload and worsening response latency, reducing the overall availability and processing efficiency of the data operations service. This invention aims to solve how to generate unified features based on multidimensional tags and call context of requests to achieve fine-grained, hierarchical strategy matching, and how to dynamically adjust data routing weights using real-time node call metrics to balance the load. Summary of the Invention

[0003] This invention provides a trusted data operation service management system based on a microservice architecture. The purpose is to integrate data tags and call chain context into global data features in complex microservice data operation scenarios, achieve precise allocation of permissions and resources through hierarchical feature mapping and multi-level policy matching, and dynamically adjust routing weights according to the running status indicators of each node, thereby optimizing the granularity of access control and the real-time response capability of service scheduling.

[0004] To achieve the above objectives, this invention provides the following technical solution: This invention provides a trusted data operation service management system based on a microservice architecture. The system includes a request parsing module, a feature generation module, a strategy engine module, an access arbitration module, a metric collection module, and a scheduler module. The request parsing module receives operation service requests and parses data tags and call chain context from the requests. The feature generation module generates global data features based on the parsed data tags and call chain context, thereby unifying and expressing the access subject, data attributes, and call path context information of the request, improving the accuracy of subsequent strategy matching. The strategy engine module receives the global data features, performs hierarchical feature mapping, and obtains multi-level strategy matching results, achieving layer-by-layer refined control over complex operation scenarios. The access arbitration module compares the multi-level strategy matching results and allocates temporary access permissions and resource quotas for the data service call based on the comparison results, avoiding permission abuse and achieving reasonable control over resource usage. The metric collection module collects node call metrics on microservice nodes in real time during the data service call process and feeds the node call metrics back to the microservice scheduler. The scheduler module adjusts the data routing weights of subsequent operational service requests based on node call metrics, enabling traffic to dynamically avoid high-load or high-latency nodes and ensuring the quality of service calls.

[0005] Preferably, when generating global data features, the feature generation module first extracts data tags from the operational service requests. These data tags include the data owner identifier, data sensitivity level, and data usage purpose code. It then extracts the upstream service identifier, call timestamp, and call chain depth value from the call chain context. Next, it concatenates each field from the data tags with each field from the call chain context at the field level to generate a combined feature vector. Finally, it performs a hash operation on the combined feature vector to obtain the global data features. As a technical solution of this invention, a consistent hashing algorithm is used to map the combined feature vector onto a fixed hash ring space, generating fixed-length global data features. This method ensures that identical or similar requests can be stably mapped to the same feature space, guaranteeing the consistency and traceability of policy judgments, while avoiding the bloat problem caused by the increase in the number of fields in the feature length.

[0006] After receiving global data features, the strategy engine module performs a hierarchical feature mapping process that further includes: the strategy execution engine calls the first-level mapping table to match the global data features to the data domain classification identifier; based on the data domain classification identifier, it calls the second-level mapping table to match the data domain classification identifier to the operational scenario group identifier; based on the operational scenario group identifier, it calls the third-level mapping table to match the operational scenario group identifier to the strategy combination template identifier; and based on the strategy combination template identifier, it extracts multiple candidate operational strategies from the strategy library to form a multi-level strategy matching result. This coarse-to-fine layer-by-layer mapping decomposes the originally complex multi-dimensional matching problem into multiple stages of precise table lookup operations, significantly reducing the computational overhead of rule matching. Furthermore, each layer of the mapping table can be maintained and updated independently, making it more suitable for microservice environments where data operation scenarios change frequently.

[0007] When allocating temporary access permissions and resource quotas, the access arbitration module iterates through each candidate operation policy in the multi-level policy matching results, extracting the permission level field and resource limit field from each candidate operation policy. The permission level field is then sorted in descending order of its value, and the permission level field with the highest value is selected as the temporary access permission. The resource limit field values ​​from all candidate operation policies are summed to obtain the total resource limit, which is then used as the resource quota. Finally, the temporary access permission and resource quota are bound to the initiator identifier of the operation service request. When the access arbitrator iterates through the multi-level policy matching results, if it detects that two or more candidate operation policies have the same permission level field, it selects the candidate operation policy with the most recent effective time as the temporary access permission. This mechanism ensures that in scenarios with multiple overlapping policies, access control conforms to the principle of least privilege and highest applicability, while also considering timeliness and the latest policy requirements. The allocation of resource quotas comprehensively considers the constraints of all matching policies, avoiding excessive resource usage due to insufficient restrictions from a single policy.

[0008] When collecting node call metrics in real time, the metric collection module deploys probes on each microservice node along the data service call path. Each probe records the request processing time and request queue length for each microservice node, and encapsulates these times into a node call metric. This metric also carries the microservice node's node identifier and timestamp. The probes push the node call metrics to the metric aggregator via an asynchronous message channel. The metric aggregator groups the node call metrics according to the node identifier and forwards the grouped metrics to the microservice scheduler. As a further preferred implementation, when encapsulating the node call metrics, the probes specifically read the node identifier pre-set in the microservice node's configuration file, read the current time from the microservice node's local clock as the timestamp, calculate the time difference between the request arriving at and leaving the microservice node as the request processing time, and count the number of requests waiting to be processed in the microservice node's request queue as the request queue length. Then, the node identifier, timestamp, request processing time, and request queue length are assembled into a node call metric in a key-value pair format. A message confirmation mechanism is introduced during the push process. If the probe does not receive a confirmation response from the indicator aggregator, the node will retransmit the indicator, thereby ensuring the reliable transmission of indicator data and avoiding scheduling decision deviations caused by message loss.

[0009] When the scheduler module adjusts data routing weights based on node call metrics, it extracts the request processing time and request queue length for each microservice node from the node call metrics; calculates the deviation between the request processing time of each microservice node and the average request processing time of all microservice nodes; determines the health score of each microservice node based on the deviation value, and applies a decay correction to the health score based on the request queue length; normalizes the corrected health score into data routing weights, and distributes the data routing weights to the service gateway. When determining the health score, the microservice scheduler compares the absolute value of the deviation value with a preset deviation threshold. If the absolute value of the deviation value is less than the preset deviation threshold, a base score is set as the first score; if the absolute value of the deviation value is greater than or equal to the preset deviation threshold, a base score is set as the second score, with the first score being greater than the second score. Then, it reads the request queue length and compares it with a preset queue length threshold. If the request queue length exceeds the preset queue length threshold, the base score is multiplied by a decay factor to obtain the corrected health score; if the request queue length does not exceed the preset queue length threshold, the health score remains unchanged. By combining the deviation of processing time with queuing depth to conduct a two-dimensional evaluation of nodes, the real-time load status of nodes can be more accurately reflected, thereby enabling subsequent requests to be evenly distributed to nodes with better processing capabilities, reducing tail latency, and improving the resource utilization efficiency of the overall service cluster.

[0010] Before receiving operational service requests, the service registry in the microservice architecture receives registration requests from each data service node. These requests carry the data service node's service identifier, network address, and the identifier of the data domain it serves. The service registry associates and stores the service identifier, network address, and data domain identifier in the service registry and periodically sends heartbeat probes to each data service node, updating the health status of the data service nodes in the service registry based on the heartbeat results. This allows policy matching and scheduling to be based on a real-time available service topology, avoiding routing requests to failed nodes.

[0011] When receiving and parsing an operational service request, the service gateway intercepts the request, reads the data tag from the HTTP header (a predefined set of key-value pairs), and extracts the call chain context from the request body. This call chain context includes the upstream service identifier, request initiation time, and global tracking identifier. The service gateway verifies the signature information in the data tag; if the signature verification passes, the data tag and call chain context are passed to the parser. The parser performs field splitting on the data tag and call chain context according to a predefined data dictionary, outputting structured data tags and call chain context. This pre-processing of signature verification and structured deconstruction ensures the authenticity and non-repudiation of the data source, and subsequent modules can directly consume standardized fields, reducing coupling between modules.

[0012] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0013] In terms of feature generation and policy matching, the data tags extracted from operational service requests include data owner identifiers, data sensitivity levels, and data usage purpose codes. The upstream service identifier, call timestamp, and call chain depth value extracted from the call chain context are used to generate a combined feature vector through field-level concatenation. A consistent hashing algorithm is then used to map this combined feature vector to a fixed hash ring space, generating a fixed-length global data feature. This global data feature merges previously scattered request attributes into a single, compact identifier, enabling subsequent policy matching to be based on the complete context of the request. After receiving the global data feature, the policy engine calls the first-level mapping table to match the global data feature to the data domain classification identifier, the second-level mapping table to match the data domain classification identifier to the operational scenario group identifier, and the third-level mapping table to match the operational scenario group identifier to the policy combination template identifier, extracting multiple candidate operational policies to form a multi-level policy matching result. Through this three-layer progressive mapping, policy matching gradually converges from coarse-grained data domains to specific operational scenarios and policy templates, ensuring that each request accurately matches the set of candidate policies corresponding to its business characteristics. This avoids excessively broad permission determination ranges or omissions of applicable policies due to single-dimensional matching. The arbitrator iterates through the multi-level policy matching results, extracts the permission level field and resource limit field from each candidate operational policy, sorts the permission level field in descending order, and selects the highest level as the temporary access permission. Simultaneously, the resource limit field values ​​are summed to obtain the total resource limit as the resource quota. The temporary access permission and resource quota are then bound to the request initiator's identifier. This arbitration method prioritizes the highest permission level and aggregates resource quotas in scenarios where multiple policies overlap, maintaining the principle of least privilege without reducing authorization strength due to policy conflicts. Regarding dynamic scheduling, probes deployed on each microservice node along the data service call path record request processing time and request queue length, encapsulating them into node call metrics carrying node identifiers and timestamps. These metrics are pushed to the metric aggregator via an asynchronous message channel, and then grouped by node identifier before being forwarded to the microservice scheduler. The microservice scheduler extracts the request processing time and request queue length for each node from the node call metrics. It calculates the deviation between the request processing time and the average request processing time of all nodes. Based on the comparison between the deviation and a preset deviation threshold, a base score is set. A higher first score is assigned if the deviation is less than the threshold; otherwise, a lower second score is assigned. The scheduler also reads the comparison between the request queue length and a preset queue length threshold. If the queue length exceeds the threshold, the base score is adjusted using a decay factor; otherwise, the base score is maintained. The adjusted health score is normalized into data routing weights and distributed to the service gateway.A health scoring mechanism based on both processing time deviation and queue length is introduced, enabling routing weights to reflect the actual load capacity and current pressure status of nodes in real time. When a node slows down or requests back up, its traffic share is automatically reduced, and requests are tilted towards lightly loaded nodes, thereby suppressing tail latency propagation in the link and improving the throughput stability of the entire microservice data operation service system under fluctuating workloads. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0015] Figure 1 This is a schematic diagram of the architecture of a trusted data operation service management system based on a microservice architecture;

[0016] Figure 2 This is a flowchart of the feature generation module.

[0017] Figure 3 This is a flowchart of the hierarchical feature mapping process of the strategy execution engine;

[0018] Figure 4 This is the access arbitration processing flowchart;

[0019] Figure 5 This is a flowchart of the node's indicator collection and transmission process;

[0020] Figure 6 This is a flowchart of microservice scheduling and data routing weight generation;

[0021] Figure 7 It is a global data feature frequency distribution map;

[0022] Figure 8 This is a schematic diagram illustrating the hierarchical mapping between global data features and data domain classification identifiers. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See Figure 1This invention provides a trusted data operation service management system based on a microservice architecture, including a request parsing module, a feature generation module, a policy engine module, an access arbitration module, a metric collection module, and a scheduler module. The request parsing module receives operation service requests and parses the data tags and call chain context within the requests. The feature generation module generates global data features based on the data tags and call chain context of the operation service requests. The policy engine module receives the global data features and performs hierarchical feature mapping to obtain multi-level policy matching results. The access arbitration module compares the multi-level policy matching results and allocates temporary access permissions and resource quotas for data services. The metric collection module collects node call metrics during the data service call process in real time and feeds these metrics back to the microservice scheduler. The scheduler module adjusts the data routing weights of subsequent operation service requests based on the node call metrics.

[0025] Example 1:

[0026] In specific implementation, please refer to Figure 2 The feature generation module retrieves data tags from the structured data output by the request parsing module. These data tags include a data owner identifier field, a data sensitivity level field, and a data usage purpose code field. The data owner identifier field is a string type, recording a unique identifier for the data's owner. The data sensitivity level field is an enumerated numerical value, ranging from four levels: public, internal, sensitive, and confidential. The data usage purpose code field is a predefined numeric code, identifying the business purpose for which the data is used, such as marketing analysis, risk control, or product optimization.

[0027] Meanwhile, the feature generation module extracts the upstream service identifier field, the call timestamp field, and the call chain depth value field from the call chain context output by the request parsing module. The upstream service identifier field is the service name or service identifier of the previous microservice that directly initiated this request in the call chain; the call timestamp field is the Unix millisecond timestamp recorded when the upstream service initiated the call; and the call chain depth value field is an integer value representing the number of service hops from the initial entry service to the current service.

[0028] The feature generation module concatenates the extracted data owner identifier, data sensitivity level, data usage purpose code, upstream service identifier, call timestamp, and call chain depth fields at the field level to generate a combined feature vector. The concatenation process is performed in a predetermined field order: data owner identifier, data sensitivity level, data usage purpose code, upstream service identifier, call timestamp, and call chain depth. Each field value is connected using a vertical bar "|" to form a complete string as the combined feature vector. For example, if the data owner identifier is "owner_01", the data sensitivity level is "2", the data usage purpose code is "1001", the upstream service identifier is "service_A", the call timestamp is "1715000000000", and the call chain depth is "3", then the concatenated combined feature vector string would be "owner_01|2|1001|service_A|1715000000000|3".

[0029] After obtaining the combined feature vector, the feature generation module performs a hash operation on the combined feature vector to generate global data features. The hash operation uses a consistent hashing algorithm, mapping the combined feature vector to a fixed hash ring space. The size of the hash ring space is preset, typically set to a value of [value missing]. The corresponding ring from 0 to The integer range. The consistent hashing algorithm can use either MD5 or SHA-1 as the hash function. It calculates a digest from the combined feature vector string, takes the first 4 bytes of the digest, converts it to a 32-bit unsigned integer, and then takes the modulo of the hash ring size to ensure the result falls within the ring space. The specific mapping process is implemented by the following formula:

[0030]

[0031] in, This represents the generated global data features, which are within a range of Integer values ​​within; This represents the combined feature vector string obtained after concatenating fields. This indicates the selected consistent hash function, which takes a string as input and outputs a 32-bit unsigned integer. This represents the size of the hash ring space. The specific calculation method is as follows: first use the MD5 algorithm to... Generate a 128-bit message digest, then take the highest 32 bits of the digest and interpret it as an unsigned integer to obtain the hash value.

[0032] Through the hash operation described above, regardless of the length of the combined feature vector string, it is ultimately mapped to a global data feature with a fixed value range and fixed length. This global data feature will serve as the unified input for subsequent hierarchical feature matching in the strategy engine module.

[0033] See Figure 7 This figure illustrates the frequency distribution of global data features. The horizontal axis represents the numerical values ​​of the global data features, ranging from approximately 0 to 4.5 × 10^9, while the vertical axis represents the frequency of the corresponding values, ranging from 0 to approximately 18. The figure uses a histogram to display the frequency distribution characteristics, with gray bars, and the legend reads "Frequency Distribution of Global Data Features".

[0034] As shown in the figure, the frequency of global data features exhibits a fluctuating distribution across the overall interval, without a clear single-peak concentration trend. Instead, multiple frequency peaks alternate. Intervals with relatively high frequencies are concentrated around 0.4×10^9, 0.9×10^9, 1.5×10^9, and 3.6×10^9, with the highest frequency peak reaching approximately 18 times. The frequency fluctuations across these intervals are quite significant, indicating that the global data features are distributed relatively discretely in the hash ring space without significant skewness.

[0035] This frequency distribution map reflects the distribution of global data features obtained by the feature generation module in Example 1 through field-level concatenation and consistent hashing within the predetermined hash ring space. The frequency fluctuations indicate that after hash mapping, the generated global data features from different combination feature vectors are evenly distributed on the hash ring, avoiding hotspot concentration. This is beneficial for load balancing and efficient querying during multi-level policy matching by the subsequent policy engine module. The overall uniformity of the frequency distribution also ensures that the system can reasonably allocate resources and permissions during policy matching and access arbitration, improving the stability and reliability of data operation services.

[0036] Example 2:

[0037] In specific implementation, please refer to Figure 3 The strategy engine module integrates a strategy execution engine. After receiving global data features output by the feature generation module, the strategy execution engine initiates a hierarchical feature mapping process. The strategy execution engine maintains three layers of mapping tables: a first-layer mapping table, a second-layer mapping table, and a third-layer mapping table. These three layers of mapping tables are loaded into the strategy execution engine's memory by the configuration management service during system initialization and can be dynamically updated through the management interface.

[0038] The first-level mapping table stores the mapping relationship between global data features and data domain classification identifiers. The structure of the first-level mapping table is a set of key-value pairs, where the key is the hash value range of the global data feature, and the value is the corresponding data domain classification identifier. The data domain classification identifier is a predefined string constant used to identify the business domain to which the data belongs, such as "financial domain," "medical domain," or "government domain." The process of the policy execution engine calling the first-level mapping table for matching is as follows: the policy execution engine compares the received global data feature value with the range range of each record in the first-level mapping table one by one to determine the range range in which the global data feature value falls, and obtains the data domain classification identifier corresponding to the range range. If the global data feature value does not fall into the range range of any record, a default data domain classification identifier is returned. The default data domain classification identifier is set to "public domain" in the system configuration.

[0039] After the first-level mapping table matching is completed, the strategy execution engine obtains a definite data domain category identifier. The strategy execution engine uses this data domain category identifier as input to call the second-level mapping table. The second-level mapping table stores the mapping relationship between data domain category identifiers and operational scenario group identifiers. The structure of the second-level mapping table is also a set of key-value pairs, where the key is the data domain category identifier string and the value is a list of operational scenario group identifiers. The operational scenario group identifier is used to identify a group of operational scenarios under a specific data domain, such as "Risk Assessment Scenario Group" and "Customer Marketing Scenario Group" under the "Financial Domain". The strategy execution engine uses the data domain category identifier as the key to perform an exact lookup in the second-level mapping table to obtain the corresponding list of operational scenario group identifiers. If no key in the second-level mapping table completely matches the data domain category identifier, a default list of operational scenario group identifiers is returned, which contains an element identified as "General Operational Scenario Group".

[0040] After obtaining the list of operational scenario group identifiers, the strategy execution engine calls the third-level mapping table for each operational scenario group identifier in the list. The third-level mapping table stores the mapping relationship between operational scenario group identifiers and strategy combination template identifiers. The third-level mapping table is a key-value pair structure, where the key is the operational scenario group identifier string, and the value is the corresponding strategy combination template identifier. The strategy combination template identifier is a predefined template number within the system, and each strategy combination template identifier points to a set of associated candidate operational strategies in the strategy library. The strategy execution engine performs an exact lookup in the third-level mapping table using the operational scenario group identifier as the key to obtain a strategy combination template identifier. If no key in the third-level mapping table completely matches the operational scenario group identifier, the operational scenario group identifier is skipped, and the next operational scenario group identifier is processed.

[0041] The strategy execution engine collects all matched strategy combination template identifiers into a set, and then extracts candidate operational strategies from the strategy repository based on this set. The strategy repository is a strategy storage table in a relational database, with a structure that includes at least a strategy identifier, a strategy combination template identifier, a strategy content field, a strategy effective time field, and a strategy priority field. The strategy execution engine constructs a database query statement, with the query conditions being that the value of the strategy combination template identifier field belongs to the aforementioned set of strategy combination template identifiers, and that the strategy effective time field is less than or equal to the current system time. The query results return one or more strategy records, each containing a complete operational strategy definition, which includes permission level and resource limit fields. The strategy execution engine assembles all the operational strategy records returned by the query into a multi-level strategy matching result. The multi-level strategy matching result is a list of strategy objects, with each element in the list corresponding to a candidate operational strategy. The multi-level strategy matching result is then passed to the access arbitration module for further processing.

[0042] See Figure 8 In the figure, the horizontal axis represents the global data feature values ​​generated by the strategy engine module, ranging from approximately 0 to 4.5 × 10^9. The vertical axis represents the data domain classification labels, including four categories: "Financial Domain," "Medical Domain," "Government Domain," and "Public Domain." Different symbols are used in the legend to identify the four data domain categories: circles represent the financial domain, triangles represent the medical domain, squares represent the government domain, and diamonds represent the public domain.

[0043] As shown in the figure, the distribution of global data feature values ​​across different data domains exhibits clear hierarchical and interval-based characteristics: the global data feature values ​​for the financial domain are concentrated in the range of approximately 0 to 1.1 × 10^9, forming a relatively compact set of values; the global data feature values ​​for the medical domain are mainly distributed in the range of approximately 1.1 × 10^9 to 2.2 × 10^9; the global data feature values ​​for the government domain are distributed in the range of approximately 2.2 × 10^9 to 3.2 × 10^9; and the values ​​for the public domain are concentrated in the range of approximately 3.2 × 10^9 to 4.3 × 10^9. The value intervals between each data domain category are independent and non-overlapping, as evidenced by the discrete distribution points of the category labels on the vertical axis forming a horizontal distribution band structure along the horizontal axis.

[0044] This diagram illustrates the mapping relationship between the range of global data feature hash values ​​and data domain classification identifiers in the first-level mapping table within the strategy execution engine. Specifically, the first-level mapping table uses predefined hash value ranges to hierarchically map global data features and output corresponding data domain classification identifiers. The four consecutive and non-overlapping hash ranges shown in the diagram correspond to the classification identifiers of the four data domains: "Financial Domain," "Medical Domain," "Government Domain," and "Public Domain," respectively, demonstrating a clear and stable mapping relationship.

[0045] Therefore, the strategy engine module can accurately locate the business domain to which the data belongs based on the global data feature values, complete the matching process of the first-level mapping table, and provide the basic input for the subsequent second-level operational scenario grouping identifier mapping. The global data feature distribution and classification identifier correspondence shown in the figure verify the rationality and effectiveness of the hierarchical feature mapping process of the strategy execution engine in Example 2.

[0046] Example 3:

[0047] In specific implementation, please refer to Figure 4 The access arbitration module internally sets up an access arbitrator, which receives the multi-level policy matching results output by the policy engine module. The multi-level policy matching results are a list of policy objects, each corresponding to a candidate operational policy. Each policy object contains a permission level field and a resource limit field. The permission level field is an integer, with a predefined value range of 1 to 5; a larger value indicates a higher access level. The resource limit field is also an integer, representing the maximum number of computing resources allowed to be accessed by the candidate operational policy, expressed in computing resource units.

[0048] The access arbitrator iterates through the list of policy objects in the multi-level policy matching results. During this iteration, the access arbitrator extracts the values ​​of the permission level field and the resource limit field from the currently encountered policy object, storing the extracted permission level values ​​in a permission level array and the extracted resource limit values ​​in a resource limit array. The iteration operation covers all policy objects in the policy object list, ensuring that no candidate operational policy is missed.

[0049] After the traversal operation is complete, the access arbitrator sorts all permission level values ​​in the permission level array in descending order. The access arbitrator selects the element at the beginning of the sorted array, which is the permission level value with the highest value, as the temporary access permission for this request. The temporary access permission is recorded as an integer value.

[0050] During the descending sorting of the permission level array, if the access arbitrator detects that two or more policy objects have the same value in their permission level field, it further reads the policy effective time field from each policy object. The policy effective time field is a timestamp, recording the moment the policy was created or updated. The access arbitrator sorts the policy objects with the same permission level value in descending order by their policy effective time field and extracts the permission level value from the policy object with the latest effective time, using this value as the temporary access permission. If a policy object does not contain a policy effective time field, the access arbitrator maintains the original permission level value selection result.

[0051] The access arbitrator performs an accumulation operation on all resource limit values ​​in the resource limit array. The accumulation operation is implemented by the following formula:

[0052]

[0053] in, This represents the calculated total resource limit, which is also the final allocated resource quota; This represents the total number of candidate operational strategies in the multi-level strategy matching results, and this value is equal to the length of the resource limit array. Indicates the first The value of the resource cap field in the candidate operational strategy. The arbitrator will sum the values. The value represents the resource quota for this request.

[0054] After obtaining temporary access permissions and resource quotas, the access arbitrator reads the initiator identifier from the context of the operational service request. The initiator identifier is a string type and records the identity information of the entity initiating this operational service request; it can be a user account identifier or an application identifier. The access arbitrator binds the temporary access permission value and resource quota value to the initiator identifier in key-value pairs, forming an access control record. The access control record contains four fields: initiator identifier, temporary access permissions, resource quota, and binding timestamp. The access arbitrator writes the access control record to the access control cache. The access control cache uses a key-value storage structure, with the initiator identifier as the key and a structure containing the temporary access permissions and resource quota as the value, and sets an expiration time. The expiration time is the duration of the temporary access session configured by the system, which defaults to 3600 seconds. After the expiration time, the access control record in the access control cache automatically expires and is reclaimed. After completing the binding operation, the access arbitrator returns the temporary access permissions and resource quotas to the service gateway along with the response message. The service gateway then performs subsequent service call routing and resource rate limiting operations based on the temporary access permissions and resource quotas.

[0055] Example 4:

[0056] In specific implementation, please refer to Figure 5 The metrics collection module is responsible for collecting runtime data from each microservice node in real time during data service calls. The module is deployed by installing a probe within each microservice node along the data service call path. This probe is a lightweight component running as a software agent, deployed in the same runtime environment as the microservice node's main process, and obtains call information by intercepting request entry and exit points.

[0057] The probe records the request processing time and request queue length for each microservice node. The request processing time is calculated by subtracting the timestamp of a request leaving the microservice node from its arrival time. The specific calculation method is as follows:

[0058]

[0059] in, This indicates the request processing time, measured in milliseconds. This indicates the timestamp of the request leaving the microservice node, which is read from the microservice node's local clock by the probe when the microservice node completes request processing and returns a response to the caller; This represents the timestamp when the request arrives at the microservice node, which is read from the microservice node's local clock by the probe when the request is received.

[0060] The request queue length is the number of requests waiting to be processed in the request queue of a microservice node, as counted by the probe. The probe obtains the request queue length by reading the current depth of the request queue of the microservice node. The request queue depth is returned by an interface provided by the microservice framework and is a non-negative integer value.

[0061] The probe encapsulates request processing time and request queue length into node call metrics. The encapsulation process includes the following steps: The probe reads the node identifier pre-defined in the microservice node's configuration file. The node identifier is a unique string name, written to the configuration file by operations personnel during microservice node deployment. The probe reads the current time from the microservice node's local clock as a timestamp, using the Unix millisecond timestamp format. The probe calculates the request processing time using the formula mentioned above and records it as an integer value. The probe calculates the request queue length and records it as an integer value. The probe assembles the node identifier, timestamp, request processing time, and request queue length into a node call metric using key-value pairs. The key-value pair format uses a JSON object structure, where the key "node_id" corresponds to the node identifier string, the key "timestamp" corresponds to the timestamp value, the key "proc_time" corresponds to the request processing time value, and the key "queue_len" corresponds to the request queue length value.

[0062] After the probe generates node call metrics, it pushes these metrics to the metric aggregator via an asynchronous message channel. The asynchronous message channel is implemented using a message queue, with middleware such as Kafka or RabbitMQ as possible. The probe, acting as a message producer, serializes the node call metrics into a message body and sends it to the specified topic in the message queue. The metric aggregator, acting as a message consumer, subscribes to the corresponding topic and pulls messages.

[0063] When the probe pushes node call metrics via an asynchronous message channel, it employs a message confirmation mechanism. After sending each message, the probe waits for an acknowledgment from the metric aggregator. If the probe receives an acknowledgment from the metric aggregator within a preset timeout period, it marks the node call metric push as successful. If no acknowledgment is received after the timeout period expires, the probe determines the push has failed and initiates a retransmission mechanism, resending the same node call metric to the asynchronous message channel. The maximum number of retransmissions is 3, with a 1-second interval between retransmissions. If no acknowledgment is received after 3 retransmissions, the probe writes the node call metric to a local persistent log file and triggers an alarm.

[0064] After receiving node call metrics from various probes, the metric aggregator performs a grouping operation on all node call metrics. The aggregator reads the node identifier field from each node call metric, groups them according to the node identifier string, and categorizes node call metrics with the same node identifier into the same group. The aggregator then sorts the node call metrics within each group in ascending order of timestamp, encapsulates them into batch metric messages, and forwards these batch metric messages to the microservice scheduler via an internal RPC channel. Upon receiving the batch metric messages, the microservice scheduler completes the node call metric collection process.

[0065] Example 5:

[0066] In specific implementation, please refer to Figure 6 The core component of the scheduler module is the microservice scheduler, which receives grouped node call metrics forwarded by the metric aggregator in the metric collection module. The microservice scheduler processes each group of node call metrics, starting with extracting the request processing time and request queue length. The microservice scheduler parses each group of node call metric data packets, which contain multiple node call metric records arranged in ascending order of timestamps. Each record includes a node identifier field, a timestamp field, a request processing time field, and a request queue length field. The microservice scheduler iterates through all records in the data packet, extracting the value of the request processing time field into a request processing time sequence and the value of the request queue length field into a request queue length sequence.

[0067] After obtaining the request processing time series for all microservice nodes, the microservice scheduler calculates the average request processing time for all microservice nodes. The scheduler then aggregates the request processing time series from all microservice nodes, merging all request processing time values ​​into a global request processing time set. Finally, it calculates the arithmetic mean of all values ​​in the global request processing time set to obtain the average request processing time, denoted as . The microservice scheduler then calculates the deviation between the request processing time and the average request processing time for each microservice node. The deviation is calculated as follows: for each microservice node, the scheduler takes the most recently collected request processing time value from the node's request processing time sequence as the current request processing time. ,calculate and The difference is calculated, and the absolute value of the difference is taken to obtain the deviation value. The formula for calculating the deviation value is expressed as:

[0068]

[0069] in, The deviation value representing a microservice node is a non-negative real number, measured in milliseconds. This indicates the most recent request processing time collected by the microservice node, extracted from the last record of the request processing time sequence of that microservice node; This represents the average request processing time across all microservice nodes. This indicates the operation of taking the absolute value.

[0070] The microservice scheduler is based on the deviation value Determine the health score of each microservice node. The microservice scheduler internally maintains a preset deviation threshold, which is a configuration parameter set by operations personnel based on system experience. The set value is 50 milliseconds, based on the assumption that when the request processing time of a microservice node deviates from the average by more than 50 milliseconds, it indicates significant performance fluctuations or resource bottlenecks for that microservice node. The microservice scheduler then calculates the deviation value for each microservice node. Compare with a preset deviation threshold. If the deviation value... If the deviation is less than the preset deviation threshold, the base score for this microservice node is set to the first score, which is 100; if the deviation value is less than the preset deviation threshold, the base score for this microservice node is set to the first score. If the score is greater than or equal to the preset deviation threshold, the basic score for that microservice node is set to the second score, which is 60. A first score of 100 is greater than a second score of 60, indicating that a microservice node with stable processing time receives a higher basic score.

[0071] After determining the base score, the microservice scheduler applies a decay correction to the health score based on the request queue length. The microservice scheduler extracts the most recently collected request queue length value from the request queue length sequence of that microservice node. The microservice scheduler internally maintains a preset queue length threshold, which is a configuration parameter set to 10. This threshold is based on the principle that when the request queue length of a microservice node exceeds 10, it indicates severe backlog on that microservice node, and it should not handle too many new requests even if request processing time is normal. The microservice scheduler manages the request queue length... Compare with a preset queue length threshold. If a queue length is requested... If the queue length exceeds a preset threshold, the microservice scheduler multiplies the base score by a decay factor to obtain a corrected health score. The decay factor is a configurable parameter set to 0.7. This value is set so that, even when the queue backlog exceeds the threshold, the corrected score is maintained at 70% of the base score, reflecting that the node still has processing capacity while significantly reducing its probability of being scheduled. (The last sentence appears to be incomplete and possibly refers to a different context.) If the preset queue length threshold is not exceeded, the health score remains unchanged as the base score and no decay correction is performed.

[0072] After obtaining the corrected health score for each microservice node, the microservice scheduler normalizes all health scores to obtain the data routing weight for each microservice node. The normalization is calculated by dividing each microservice node's health score by the sum of all microservice node health scores; the quotient is the data routing weight for that microservice node. The data routing weight is a floating-point number between 0 and 1, and the sum of the data routing weights for all microservice nodes equals 1. The microservice scheduler encapsulates the node identifier and corresponding data routing weight of each microservice node into a routing weight configuration message and sends it to the service gateway via an internal RPC channel. Upon receiving the routing weight configuration message, the service gateway updates its in-memory routing weight table. Subsequent operational service requests are then distributed to each microservice node using a weighted round-robin approach according to the routing weight table.

[0073] During system startup, before receiving operational service requests, the system needs to complete a service registration process. The service registry in the microservice architecture receives registration requests from each data service node. Each data service node sends a registration request to the service registry upon startup. The message body of the registration request carries the data service node's service identifier, network address, and the identifier of the data domain it serves. The service identifier is a string and is the unique name of the data service node in the microservice architecture; the network address is a combination of IP address and port number, in the format "IP:PORT"; the data domain identifier is a string indicating the business domain to which the data managed by the data service node belongs. Upon receiving the registration request, the service registry associates the service identifier, network address, and data domain identifier using the service identifier as the primary key and the network address and data domain identifier as attribute fields, constructing a service registration record and storing it in the service registry. The service registry is a hash table structure in the service registry's memory, supporting fast lookups.

[0074] After storing the service registration records, the service registry initiates a heartbeat mechanism. The service registry periodically sends heartbeat requests to all active data service nodes in the service registry every 30 seconds. The heartbeat request is a lightweight HTTP health check request with the request path "health". Upon receiving the heartbeat request, if the data service node is functioning normally, it returns a 200 status code response; if the node fails to respond or returns a non-200 status code, the service registry marks the heartbeat attempt as failed. The service registry updates the health status of the data service nodes in the service registry based on the heartbeat results. If three consecutive heartbeat attempts fail, the service registry marks the data service node's health status as "unhealthy" and removes it from the list of active nodes in the service registry until the data service node resubmits a registration request or the heartbeat is restored. If the heartbeat attempt is successful, the service registry maintains or updates the data service node's health status to "healthy".

[0075] During the phase of receiving operational service requests, the service gateway acts as the traffic entry point, intercepting all external operational service requests. The service gateway reads data tags from the HTTP header of the operational service request. Data tags exist as a predefined set of key-value pairs, where the key is the tag name and the value is the tag content, with multiple key-value pairs separated by semicolons. Predefined keys include "data-owner" (data owner identifier), "sensitivity-level" (data sensitivity level), and "purpose-code" (purpose encoding for data use). The service gateway parses the values ​​of these key-value pairs from the HTTP header of the operational service request, field by field, to construct a data tag object.

[0076] The service gateway extracts the call chain context from the request body of the service request. The call chain context is embedded in the "trace_context" field of the request body in JSON format. The service gateway deserializes the value of the "trace_context" field to obtain the call chain context object, which contains an upstream service identifier field, a request initiation time field, and a global trace identifier field. The upstream service identifier field is the name of the upstream service that initiated the call; the request initiation time field is the Unix millisecond timestamp of the request initiated by the upstream service; and the global trace identifier field is a unique distributed trace ID that runs through the entire call chain.

[0077] After extracting the data tag and call chain context, the service gateway verifies the signature information in the data tag. The data tag includes a signature field, "signature," whose value is generated by the data owner using a pre-shared key to perform an HMAC-SHA256 operation on the remaining key-value pairs in the data tag. The service gateway reads the signature field from the data tag and recalculates the HMAC-SHA256 values ​​for all key-value pairs except the signature field using the same pre-shared key. It then compares the recalculated values ​​with the signature field value. If they match, the signature verification passes, and the service gateway passes the data tag and call chain context to the parser in the request parsing module. If they do not match, the signature verification fails, the service gateway rejects the service request, and returns an authentication failure response.

[0078] After receiving the data tags and call chain context from the service gateway, the parser splits the data tags and call chain context into fields according to a predefined data dictionary. The predefined data dictionary is loaded into the parser's memory as a configuration file, defining the name, data type, and whether each field in the data tags and call chain context is required. The parser iterates through each key-value pair in the data tags, looks up the corresponding field definition in the data dictionary, converts the value in the key-value pair to the data type specified by the data dictionary, and fills it into the corresponding attribute of the structured data tag output object. Similarly, the parser iterates through each field in the call chain context object, performs type validation and conversion according to the data dictionary, and outputs a structured call chain context object. The parser outputs both the structured data tags and the structured call chain context for use by the subsequent feature generation module.

[0079] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A trusted data operation service management system based on a microservice architecture, characterized in that, include: The request parsing module is used to receive operational service requests and parse the data tags and call chain context in the requests. The feature generation module is used to generate global data features based on the data tags and call chain context of the operational service requests; The strategy engine module receives global data features and performs hierarchical feature mapping to obtain multi-level strategy matching results, including: After receiving global data features, the strategy execution engine calls the first-level mapping table to match the global data features to the data domain classification identifier; The second-level mapping table is called based on the data domain classification identifier to match the data domain classification identifier to the operational scenario group identifier; The third-level mapping table is invoked based on the operational scenario group identifier to match the operational scenario group identifier to the strategy combination template identifier; Multiple candidate operational strategies are extracted from the strategy library based on the strategy combination template identifier to form a multi-level strategy matching result; The access arbitration module is used to compare the matching results of multi-level policies and allocate temporary access permissions and resource quotas for data services, including: Access the arbitrator to traverse each candidate operation strategy in the multi-level strategy matching results, and extract the permission level field and resource limit field from each candidate operation strategy; Sort the permission level fields in descending order of their level values, and select the permission level field with the highest level value as the temporary access permission. The total resource limit is obtained by summing the values ​​of the resource limit field in all candidate operation strategies, and the total resource limit is used as the resource quota. Bind temporary access permissions and resource quotas to the identifier of the initiator of the operational service request; The metrics collection module is used to collect node call metrics during the data service call process in real time and feed the node call metrics back to the microservice scheduler. The scheduler module is used to adjust the data routing weights of subsequent operational service requests based on node call metrics.

2. The trusted data operation service management system based on microservice architecture according to claim 1, characterized in that, The generation of global data features based on the data tags and call chain context of the operation service request includes: Extract data tags from operational service requests. Data tags include data owner identifier, data sensitivity level, and data usage purpose code. Extract the upstream service identifier, call timestamp, and call chain depth value from the call chain context; The fields in the data label are concatenated with the fields in the call chain context at the field level to generate a combined feature vector. Perform a hash operation on the combined feature vectors to obtain the global data features.

3. The trusted data operation service management system based on microservice architecture according to claim 2, characterized in that, The step of performing a hash operation on the combined feature vector to obtain global data features includes using a consistent hashing algorithm to map the combined feature vector onto a fixed hash ring space to generate fixed-length global data features.

4. The trusted data operation service management system based on microservice architecture according to claim 3, characterized in that, When the access arbitrator traverses each candidate operation strategy in the multi-level strategy matching results, if it detects that two or more candidate operation strategies have the same permission level field, it selects the candidate operation strategy with the latest policy effective time as the temporary access permission.

5. The trusted data operation service management system based on microservice architecture according to claim 1, characterized in that, The real-time data collection process includes node call metrics, which are then fed back to the microservice scheduler. Deploy probes on each microservice node along the data service call path. The probes record the request processing time and request queue length of each microservice node. The probe encapsulates the request processing time and request queue length into a node call metric, which also carries the node identifier and timestamp of the microservice node. The probe pushes the node's invoked metrics to the metric aggregator via an asynchronous message channel; The metric aggregator groups the metric calls from nodes according to their node identifiers and then forwards the grouped metric calls to the microservice scheduler.

6. The trusted data operation service management system based on microservice architecture according to claim 5, characterized in that, When the probe pushes the node call metrics to the metric aggregator through an asynchronous message channel, it adopts a message confirmation mechanism. If no confirmation response is received from the metric aggregator, the node call metrics are retransmitted.

7. The trusted data operation service management system based on microservice architecture according to claim 5, characterized in that, The probe encapsulates the request processing time and request queue length into a node call metric. The node call metric also carries the node identifier and timestamp of the microservice node, including: The probe reads the node identifier pre-defined in the configuration file of the microservice node; the probe reads the current time from the local clock of the microservice node as a timestamp; The probe calculates the time difference between when a request arrives at a microservice node and when the request leaves the microservice node, and uses this time difference as the request processing time. The probe counts the number of requests waiting to be processed in the request queue of the microservice node, and uses the number of requests as the request queue length; The probe assembles the node identifier, timestamp, request processing time, and request queue length into a node call metric in a key-value pair format.

8. The trusted data operation service management system based on microservice architecture according to claim 1, characterized in that, The adjustment of data routing weights for subsequent operational service requests based on node call metrics includes: The microservice scheduler extracts the request processing time and request queue length for each microservice node from the node call metrics. The microservice scheduler calculates the deviation between the request processing time of each microservice node and the average request processing time of all microservice nodes. The microservice scheduler determines the health score of each microservice node based on the deviation value and performs decay correction on the health score based on the request queue length. The microservice scheduler normalizes the corrected health score into a data routing weight and distributes the data routing weight to the service gateway.

Images