Trusted data space dynamic expansion method and system of micro-service architecture

Abstract

The invention provides a trusted data space dynamic expansion method and system of a micro-service architecture, and relates to the technical field of data processing. Task prediction is performed according to a task database by establishing a hybrid expansion strategy of a computing resource layer, and a task prediction result is established; performing adaptation evaluation on a hybrid expansion strategy in a computing resource layer by using a task prediction result, pre-configuring the hybrid expansion strategy, performing hotspot calling prediction of data, synchronously establishing a distributed data storage layer by using a hotspot data prediction result, and establishing a distributed data storage layer and hybrid expansion strategy mapping; and dynamic expansion of the trusted data space is completed. The technical problem that in the prior art, a trusted data space of a micro-service architecture cannot flexibly cope with high concurrency and task dynamic changes is solved. The technical effects that the trusted data space is dynamically expanded, the flexibility of resource scheduling is improved, and meanwhile data safety and high availability are ensured are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a method and system for dynamically expanding a trusted data space in a microservices architecture. Background Art

[0002] With the wide application of cloud computing, microservices architecture, and big data technology, modern business systems are facing complex challenges such as high concurrency of data, dynamic changes in tasks, and real-time scheduling of resources. Most traditional resource expansion methods adopt static configuration and cannot flexibly adjust computing resources according to the real-time fluctuations of task loads, often resulting in insufficient resources during peak periods and wasted resources during off-peak periods in the system. At the same time, traditional systems lack the ability to predict and manage task behaviors, and respond slowly when dealing with a large number of requests or abnormal tasks, which easily causes system congestion or service interruption.

[0003] The prior art has the technical problem that the trusted data space in the microservices architecture cannot flexibly cope with high concurrency and dynamic changes in tasks. Summary of the Invention

[0004] This application provides a method and system for dynamically expanding a trusted data space in a microservices architecture, which is used to solve the technical problem that the trusted data space in the microservices architecture of the prior art cannot flexibly cope with high concurrency and dynamic changes in tasks.

[0005] In view of the above problems, this application provides a method and system for dynamically expanding a trusted data space in a microservices architecture.

[0006] In the first aspect of this application, a method for dynamically expanding a trusted data space in a microservices architecture is provided. The method includes: establishing a hybrid expansion strategy for the computing resource layer, where the hybrid expansion strategy includes a Kubernetes, WASM, FaaS hybrid strategy; configuring a task database, making task predictions according to the task database, and establishing a task prediction result, where the task prediction result carries a task feature identifier and a trust identifier; using the task prediction result to perform an adaptation evaluation of the hybrid expansion strategy within the computing resource layer, and pre-configuring the hybrid expansion strategy according to the adaptation evaluation result; performing a hot call prediction of data based on the hybrid expansion strategy and the task prediction result, and establishing a hot data prediction result; using the hot data prediction result to synchronously establish a distributed data storage layer, and after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, completing the dynamic expansion of the trusted data space.

[0007] In the second aspect of the present application, a dynamic expansion system for a trusted data space in a microservices architecture is provided. The system includes: a policy establishment module for establishing a hybrid expansion policy for the computing resource layer, where the hybrid expansion policy includes a Kubernetes, WASM, FaaS hybrid policy; a task prediction module for configuring a task database, performing task prediction based on the task database, and establishing a task prediction result with a task feature identifier and a trust identifier; an adaptation evaluation module for using the task prediction result to perform an adaptation evaluation of the hybrid expansion policy within the computing resource layer and pre-configuring the hybrid expansion policy according to the adaptation evaluation result; a data prediction module for performing a hot call prediction of data based on the hybrid expansion policy and the task prediction result and establishing a hot data prediction result; and a dynamic expansion module for using the hot data prediction result to synchronously establish a distributed data storage layer and completing the dynamic expansion of the trusted data space after establishing the mapping between the distributed data storage layer and the hybrid expansion policy.

[0008] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0009] The method provided in the embodiment of the present application establishes a hybrid expansion policy for the computing resource layer, where the hybrid expansion policy includes a Kubernetes, WASM, FaaS hybrid policy; configures a task database, performs task prediction based on the task database, and establishes a task prediction result with a task feature identifier and a trust identifier; uses the task prediction result to perform an adaptation evaluation of the hybrid expansion policy within the computing resource layer and pre-configures the hybrid expansion policy according to the adaptation evaluation result; performs a hot call prediction of data based on the hybrid expansion policy and the task prediction result and establishes a hot data prediction result; uses the hot data prediction result to synchronously establish a distributed data storage layer and completes the dynamic expansion of the trusted data space after establishing the mapping between the distributed data storage layer and the hybrid expansion policy. It achieves the technical effects of dynamically expanding the trusted data space, improving the flexibility of resource scheduling, and ensuring data security and high availability. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a schematic diagram of the process of the method for dynamically expanding the trusted data space in the microservices architecture provided by the present application;

[0012] Figure 2 Schematic structural diagram of the trusted data space dynamic expansion system for the microservice architecture provided by this application.

[0013] Description of the reference numerals: Policy establishment module 11, Task prediction module 12, Adaptation evaluation module 13, Data prediction module 14, Dynamic expansion module 15. Detailed implementation manners

[0014] This application provides a method and system for dynamically expanding the trusted data space of the microservice architecture, which is used to solve the technical problem that the trusted data space of the microservice architecture in the prior art cannot flexibly cope with high concurrency and dynamic changes in tasks. It achieves the technical effects of dynamically expanding the trusted data space, improving the flexibility of resource scheduling, and ensuring data security and high availability at the same time.

[0015] Next, the technical solutions in the present invention will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited by the example embodiments described here. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention. Additionally, it should be noted that for the sake of description, only the parts related to the present invention are shown in the accompanying drawings rather than all of them.

[0016] Embodiment 1, as Figure 1 shown, this application provides a method for dynamically expanding the trusted data space of the microservice architecture, and the method includes:

[0017] Establish a hybrid expansion policy for the computing resource layer, and the hybrid expansion policy includes a Kubernetes, WASM, and FaaS hybrid policy.

[0018] Specifically, before constructing the hybrid scaling strategy, first clarify the goal of the hybrid scaling strategy, which is to achieve the dynamic scaling of the trusted data space in the microservices architecture, including elastic scaling, performance optimization, resource optimization, etc. Elastic scaling means that the resource layer automatically adjusts the number of resources according to the load changes. Performance optimization means ensuring that the microservices architecture can efficiently process different types of computing tasks, reducing latency and resource waste. Resource optimization means reducing resources during low load and increasing resources during high load to avoid unnecessary resource consumption. Then, establish the hybrid scaling strategy for the computing resource layer according to the goal. The computing resource layer is where the system computing resources for executing various tasks are located, including containers, virtual machines, service instances, etc., and is responsible for functions such as processing user requests, data computing, and service response. The hybrid scaling strategy can select the most suitable scaling method according to the characteristics of the tasks, system load, and resource requirements, including the Kubernetes, WASM, and FaaS hybrid strategy. Among them, Kubernetes is a commonly used container orchestration tool in the microservices architecture, responsible for managing containers in the cluster. In the hybrid scaling strategy, it is responsible for the scaling and management of computing resources. WebAssembly (WASM) is a lightweight binary format suitable for high-performance computing tasks. Especially when a large amount of computing needs to be performed in microservices, WASM can provide fast computing capabilities. FaaS can decompose computing tasks into independent functions and dynamically execute these functions as needed. In the hybrid scaling strategy, FaaS can be used to handle short-term, high-concurrency, or irregular tasks. By configuring the Kubernetes cluster and HPA, automatically scale or shrink service instances according to the load of the service, such as CPU or memory occupancy, integrate the WASM module, compile the service module with high computing requirements into the WASM module, and deploy it into the microservices to ensure that these compute-intensive tasks can run efficiently in the WASM environment. At the same time, deploy the FaaS environment, use functions to handle short-term and sudden requests, automatically start computing instances according to the task trigger conditions, establish the hybrid scaling strategy for the resource layer, and use a service mesh, such as Istio, to manage the communication between microservices, monitor the resource usage, and perform resource scaling or reduction when needed. By combining the hybrid scaling strategy of Kubernetes, WASM, and FaaS, the microservices architecture can dynamically adjust computing resources according to different task types and load requirements, utilize the automatic scaling of Kubernetes containerization, the efficient computing of WASM, and the elastic scaling of FaaS, ensuring the high performance and high availability of the microservices architecture under high load and large-scale concurrency, while reducing resource waste and providing flexible resource management capabilities.

[0019] Configure the task database, perform task prediction according to the task database, and establish a task prediction result, where the task prediction result carries a task feature identifier and a trust identifier.

[0020] Specifically, configure a task database for storing task information and task status. The task database records relevant data of all tasks in the microservice architecture, such as task type, task parameters, task status, task creation time, expected execution time, execution result, etc. The task database provides historical data and patterns for task prediction. By analyzing this data, the characteristics of future tasks can be predicted. Based on the historical data and reserved tasks in the task database, task prediction is performed through data analysis to predict the requirements and load conditions of future tasks, and a task prediction result is established. The task prediction result is the output data generated based on the task prediction process, with a task characteristic identifier. The task characteristic identifier is an identifier assigned to each task according to elements such as the type, priority, and resource requirements of the task, used to identify and process the task. Each task will be classified into different types or priorities according to its characteristic identifier, thus affecting subsequent resource allocation and task scheduling strategies. By configuring the task database, performing task prediction, and generating task prediction results, it helps the microservice architecture to predict future task requirements in advance, perform automated scaling and resource scheduling, thereby optimizing resource allocation and scheduling, and improving resource utilization and response speed.

[0021] Further, the task prediction based on the task database and the establishment of the task prediction result include: obtaining the reserved tasks in the task database, and constructing the first prediction result under the trust identifier according to the reserved tasks and the reserved task subject; parsing the task database, performing time-point task backtracking according to the parsing result, and establishing a time-point task backtracking result; extracting the time-point task backtracking results that meet the preset probability, and establishing the second prediction result under the trust identifier according to the extraction result; establishing the task prediction result according to the first prediction result and the second prediction result.

[0022] Specifically, the reservation tasks are retrieved from the task database. A reservation task refers to a task that is set in advance and scheduled to be executed at a certain time point or within a certain time period. The relevant information of the reservation tasks, such as task type, execution time, priority, resource requirements, etc., will be recorded in the task database. Then, the first prediction result under the trust identifier is constructed based on the reservation task and the reservation task entity. The reservation task entity refers to the entity or dependent object that executes the task, such as the user who initiates the task, a system module, or a certain service, etc. By identifying the task entity, the background, importance, and execution dependencies of the task can be understood, which further helps with task prediction and scheduling. The trust identifier is based on the evaluation results of task historical data and the task entity, and is used to identify the reliability and importance of the task. During the task prediction process, the trust identifier helps to prioritize tasks and determine which tasks require more resources or need to be executed faster. For example, the trust identifier of a task is constructed based on the task execution history, task priority, and the reputation of the task entity. Furthermore, based on the task entity and the trust identifier of the task, the first prediction result for the reservation task is generated, such as the estimated execution time and resource requirements.

[0023] After obtaining the reservation task, the historical task data in the task database is parsed. First, the historical task data is preprocessed to remove invalid or incorrect data, such as missing values or records with incorrect formats. Then, features are extracted based on fields such as task type, execution time, resource consumption, etc., and the parsing result is obtained, including task execution time, resource consumption, task status, and task category, etc. According to the parsing result, time window or specific time point, task backtracking is performed to extract the task execution characteristics before and after a specific time point from the historical tasks, analyze and identify periodic or time-related task load patterns, such as resource consumption and time consumption during task execution, etc., and establish the time point task backtracking result. The time point task backtracking result refers to based on a specific time point, backtracking to the historical task data within a period of time before this time point, analyzing and summarizing the execution of past tasks, which can provide a prediction basis for tasks in future similar time periods. Time point task backtracking helps to identify periodic or time-related task load patterns. For example, in the data of an e-commerce platform in the past month, it is found that the order processing tasks between 9 am and 11 am every day are often delayed due to excessive concurrent requests. These historical tasks can be analyzed through time point task backtracking to predict the task load in future similar time periods and make corresponding resource scheduling adjustments. Furthermore, after completing the time point task backtracking, task data that meets the preset conditions is extracted from the backtracked historical data. The preset conditions are set according to the regularity of the historical data, such as a preset probability. It is set that when the execution time of the task exceeds a certain percentage of the standard, such as when the execution time of the task exceeds 80% of the average value, or when the resource requirement exceeds a certain threshold, the task is considered a high-load task, and the data result that meets the preset probability condition is extracted. According to the task data extracted by backtracking and combined with the trust identifier of the task, a second prediction result is generated. The second prediction result contains more refined task load characteristics, such as high-load tasks. Finally, the first prediction result obtained based on the reservation task and trust identifier and the second prediction result obtained based on the analysis of time point task backtracking and task identifier are combined to generate the final task prediction result. By combining reservation task analysis, task backtracking, and trust identifier, the execution situation of the task can be accurately predicted, load peaks and troughs can be pre-identified, intelligent expansion and contraction can be performed, and the efficient operation of the microservices architecture under different loads can be ensured, improving the overall service quality.

[0024] Use the task prediction result to perform the adaptation evaluation of the hybrid expansion strategy within the computing resource layer, and pre-configure the hybrid expansion strategy according to the adaptation evaluation result.

[0025] Further, the adaptation evaluation of the hybrid expansion strategy within the computing resource layer is performed using the task prediction result, and the hybrid expansion strategy is preconfigured according to the adaptation evaluation result, including: obtaining the Kubernetes configuration information of the current running state; invoking the task feature identifier in the task prediction result, and performing execution adaptation analysis of the hybrid expansion strategy respectively according to the task feature identifier to establish an adaptation evaluation result; and configuring the hybrid expansion strategy using the adaptation evaluation result and the Kubernetes configuration information.

[0026] Specifically, the Kubernetes configuration information of the current running state is obtained through the Kubernetes API, including Pod configuration, HPA configuration, and resource usage, etc. The Pod configuration refers to the number of deployed container instances and resource allocation, such as CPU and memory usage, etc. HPA (Horizontal Pod Autoscaler) is used to automatically expand the number of Pods according to metrics such as CPU usage rate. The resource usage includes the usage of resources such as CPU, memory, and storage, so as to evaluate whether expansion or adjustment is needed. The task feature identifier in the task prediction result is extracted and invoked to identify the key features of each task, and the adaptation degree between the task feature identifier and the current hybrid expansion strategy is evaluated. This belongs to the process of evaluating the matching degree between the task prediction result and the current resource configuration, and is used to judge whether the existing resource configuration and expansion strategy can meet the task requirements, including resource requirement evaluation and expansion strategy adaptation. For example, evaluate whether the resource requirements of the task, CPU and memory, match the current resource configuration. If not, whether the number of Pods needs to be expanded. According to the priority and resource requirements of the task, judge whether the existing expansion strategy is sufficient, whether HPA can meet the task load requirements, or whether FaaS needs to be used to handle burst tasks. The judgment process can use Kubernetes HPA combined with the task prediction result to automatically evaluate the adaptability of the existing resource configuration and generate an adaptation evaluation result. Then, on the basis of the adaptation evaluation, the hybrid expansion strategy is further configured using the adaptation evaluation result and the Kubernetes configuration information. By analyzing the characteristics and resource requirements of the task, the adaptability of the existing resource configuration is evaluated, and an appropriate expansion strategy is preconfigured according to the evaluation result to ensure that the trusted space of the microservice architecture can dynamically and flexibly adjust resources according to the task requirements, meet the task requirements, and ensure the efficient operation of the system.

[0027] Further, configuring the hybrid expansion strategy by using the adaptation evaluation result and the Kubernetes configuration information includes: invoking the trust identifier in the task prediction result, establishing a first evaluation index according to the trust identifier; performing time sensitivity analysis on the task prediction result, establishing a second evaluation index according to the time sensitivity analysis result; performing task importance analysis on the task prediction result, establishing a third evaluation index according to the task importance analysis result; performing multi-dimensional index screening by using the first evaluation index, the second evaluation index, and the third evaluation index, mapping and screening the adaptation evaluation result according to the multi-dimensional index screening result, and based on the primary hybrid layer, standby hybrid layer, and emergency handling layer of the Kubernetes configuration, using the primary hybrid layer, standby hybrid layer, and emergency handling layer as the hybrid expansion strategy.

[0028] Specifically, first, extract the trust tags used to represent the credibility of the task from the call task prediction results, and establish the first evaluation index based on the trust identifier of the task. The first evaluation index is used to quantify the task credibility, and tasks with high credibility have high first evaluation index values. According to the time sensitivity analysis of the task prediction results, evaluate the time dependence of the task. Time sensitivity analysis refers to analyzing the timeliness requirements of the task. Based on this, it can be judged whether the task has strict requirements for latency. For example, some tasks may need to be completed as soon as possible, such as payment verification and transaction settlement, while other tasks can tolerate longer latencies, such as report generation and data synchronization. Generate the second evaluation index according to the time sensitivity analysis results. The second evaluation index is based on the time sensitivity analysis results of the task and is used to quantify the urgency and timeliness requirements of the task within a specific time period. For tasks with higher time sensitivity, the value of the second evaluation index is higher, indicating that the task needs to be processed first. Then, analyze the importance of the task in the task prediction results to generate the third evaluation index, which is used to quantify the overall impact of the task on the system. The importance of the task is usually determined based on business requirements. Tasks with high importance will have a higher third evaluation index, thus obtaining more resources and higher execution priorities. After obtaining multiple evaluation indexes such as the first evaluation index, the second evaluation index, and the third evaluation index, comprehensively evaluate the characteristics and requirements of the task according to the multiple evaluation indexes, conduct screening and sorting, determine which tasks need to be executed first, which tasks can be gradually executed when resources are sufficient, and which tasks should be used as emergency processing tasks when the system load is high, to obtain the multi-dimensional index screening results. Then, perform mapping screening on the adaptation evaluation results according to the multi-dimensional index screening results, which can accurately identify and classify the priorities of tasks, divide the tasks into different levels, provide a basis for subsequent resource scheduling and expansion strategy configuration, ensure that critical tasks can obtain timely resource allocation, and at the same time ensure the efficient operation and stability of the system under different load conditions. Through mapping screening, map the tasks with the highest scores to the primary hybrid layer. These tasks are usually the ones to be executed first; map the tasks with medium scores to the standby hybrid layer. These tasks will be started when resources permit; while the tasks with the lowest scores are mapped to the emergency processing layer and will only be executed when resources are scarce or abnormal. After mapping screening, configure the primary hybrid layer, the standby hybrid layer, and the emergency processing layer based on the Kubernetes according to the task requirements, and use the primary hybrid layer, the standby hybrid layer, and the emergency processing layer as the hybrid expansion strategy. Among them, the primary hybrid layer contains the tasks with the highest scores after multi-dimensional index screening. These tasks are usually the most prioritized tasks in the system. The standby hybrid layer contains those tasks that may be executed. They will be executed when resources are sufficient and are usually in the gray-scale startup state, waiting to be triggered when the system load changes. The emergency processing layer contains the tasks that need to be processed emergently in the system. They are usually tasks executed in case of system resource shortage or abnormality. Their priorities are relatively low, but they need to be processed in case of emergency.Through multi-dimensional index screening and priority division, dynamically configure the hybrid expansion strategy, configure different execution levels and resource strategies for each type of task, reduce latency, and improve task execution efficiency.

[0029] Based on the hybrid expansion strategy and the task prediction result, perform hot call prediction on data, and establish a hot data prediction result.

[0030] Specifically, according to the hybrid expansion strategy and the task prediction result, predict the hot call of data. The hot call prediction is based on historical tasks and data access patterns to predict the parts of data that may be frequently accessed in the future. The hot call prediction can be achieved by training a prediction model. For example, first obtain relevant data such as historical task data, hybrid expansion strategy, and task prediction result as training data, select a suitable machine learning model, such as a decision tree model, and train it with the training data. During the training process, the model will learn how to predict the hotness of data access based on input features. The output result of the model is to predict whether a certain data or service will become a hot spot. After the model training is completed and evaluated as qualified, it can be used for hot call prediction. Hot data usually refers to data that is frequently accessed or requires high-priority resources within a specific time. Predicting these hot data helps to prepare resources in advance and avoid access latency. According to the hot call prediction, convert the prediction result into a hot data prediction result, specifically indicating which data will be frequently accessed or require higher-priority resources in the future. By predicting these data, sufficient computing resources can be allocated to them in advance, optimizing the data access speed and response ability. For example, Kubernetes can dynamically increase cache instances according to the predicted hot data request volume, and FaaS can provide higher priority for hot data processing tasks to ensure that tasks under high load are promptly responded to. Through hot data prediction, more resources can be allocated to high-access-frequency data in advance, avoiding access latency, and improving the response speed of data access and data processing ability. Especially in high-concurrency and critical-task scenarios, by predicting and loading hot data in advance, it is ensured that tasks can be promptly responded to, reducing user experience problems caused by latency.

[0031] Use the hot data prediction result to synchronously establish a distributed data storage layer, and after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, complete the dynamic expansion of the trusted data space.

[0032] Specifically, using the hot data prediction results, analyze which data will become hot spots in the future. These data may be due to the increase in user access frequency, or the task prediction results show that certain data will be frequently accessed in a specific time period. By analyzing the hot data prediction results, these data can be loaded into the distributed data storage system in advance to ensure that the data can be efficiently accessed during high load periods. After identifying and loading the hot data, use tools such as Apache Cassandra, Amazon DynamoDB, HDFS, etc. to establish a distributed data storage layer. The distributed data storage layer refers to an architecture that stores data distributed on multiple computing nodes or servers. In the distributed data storage layer, data is stored in a decentralized manner and managed through a distributed database or file system to improve the availability, performance, and fault tolerance of the data. Through distributed storage, large-scale concurrent access can be efficiently handled, and data can still be accessed normally when a node fails. After the distributed data storage layer is established, the resource requirements of the distributed data storage layer are mapped to the hybrid expansion strategy to achieve computing-storage collaborative expansion, that is, computing resources and storage resources are linked to expand, ensuring that hot tasks not only have sufficient computing resources, but also can quickly obtain data, thereby greatly improving the overall response efficiency. For example, based on the hot data prediction results, identify the data set that will be frequently accessed in the future, and simultaneously establish a distributed data storage layer. Subsequently, according to the hybrid expansion strategy, it is evaluated on which computing resources the tasks or services associated with these hot data should be deployed. For example, some high-frequency data request services are suitable for running in WASM containers to improve response speed, or using FaaS to achieve rapid expansion and contraction. At this time, a mapping relationship is established between the storage nodes of the distributed data storage layer and the hybrid expansion strategy, that is, which computing node will be given priority to process a certain type of data, how to route and load balance, etc., so as to achieve close coupling of data flow and resource allocation. After establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, the dynamic expansion of the trusted data space is completed. The trusted data space refers to ensuring the security, reliability and compliance of data during data storage, processing and access, while ensuring the trust transmission and transparent management of data between different systems. Through the dynamic expansion of the trusted data space, it is possible to automatically adjust the number or capacity of computing resources according to load and task requirements, automatically expand or shrink resources, and ensure the high availability and high performance of the trusted space of the microservice architecture. By automatically building a distributed data architecture driven by hot data predictions and coordinating deployment with hybrid expansion strategies, we build a full range of elastic expansion capabilities from service instances to data layers, supporting the dynamic scaling and safe and reliable operation of trusted data spaces in large-scale, high-concurrency scenarios.

[0033] Further, after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, the following steps are included: introducing an asynchronous message queue in the access layer, writing task requests into the asynchronous message queue, and then pulling and processing them by the background; performing real-time monitoring of the task data in the asynchronous message queue to establish a real-time monitoring data set; and reporting a task queue warning based on the real-time monitoring data set.

[0034] Specifically, after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, an asynchronous message queue is introduced in the access layer. At the access layer, all task requests will be written into the asynchronous message queue. The asynchronous message queue is a decoupling component that allows the generation of task requests and the task processing process to be asynchronous, thereby reducing the latency when task requests are generated and improving the system throughput. After writing the task requests into the message queue, the background will pull and process these tasks from the message queue. By performing real-time monitoring on the task data in the asynchronous message queue, the status and processing situation of task requests can be dynamically grasped, and a real-time monitoring data set is formed based on the monitoring data. The real-time monitoring data set continuously records various metrics of tasks, such as the enqueue time, processing time, and the number of successes and failures of tasks. Through these monitoring data, the execution efficiency of tasks and the resource consumption situation can be understood, and corresponding adjustments can be made based on this information. Furthermore, a task queue warning is issued based on the data information in the real-time monitoring data set. The task queue warning function can timely detect potential task processing bottlenecks or resource shortages in the real-time monitoring data set. When it is detected that the processing speed of some tasks decreases, the queue is severely backlogged, or the task failure rate increases, the warning mechanism is triggered to remind the operation and maintenance personnel or automatically trigger resource expansion operations. Through real-time warning, measures can be taken before task processing anomalies occur to avoid the impact of service interruption or task backlog and ensure the high availability of the service.

[0035] Further, after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, the following steps are also included: configuring a backpressure response mechanism in the primary hybrid layer; when a primary backpressure signal appears in the primary hybrid layer, feeding back the primary backpressure signal to the asynchronous message queue; and the asynchronous message queue adjusts task allocation according to the activation status of the standby hybrid layer.

[0036] Specifically, after establishing the distributed data storage layer and mapping it with the hybrid expansion strategy, a backpressure response mechanism is further introduced to automatically adjust the processing traffic of tasks under high load conditions. The primary hybrid layer is responsible for processing tasks with the highest multi-dimensional metrics, usually including those tasks that are crucial to the business and have high resource requirements. To ensure the stability and performance of the primary hybrid layer, a backpressure response mechanism is configured in the primary hybrid layer. The backpressure response mechanism refers to the technology of actively controlling the traffic when the system load is too high, preventing the system from crashing due to overloading by restricting the processing rate of tasks. The backpressure mechanism usually notifies the task queue or the system through a feedback signal, requesting to temporarily reduce the request volume or delay the processing. When it is detected that the resource consumption of the primary hybrid layer exceeds the expectation, a primary backpressure signal is generated and fed back to the asynchronous message queue. The primary backpressure signal is a feedback signal from the primary hybrid layer, indicating that the current processing capacity of the primary hybrid layer has reached or is close to the upper limit, and it is necessary to reduce the task inflow or wait until the system resources are idle before processing. After receiving the primary backpressure signal, the asynchronous message queue adjusts the task allocation according to the activation status of the standby hybrid layer. If the standby hybrid layer is in an active state, that is, the resources are sufficient and it can process tasks, the queue transfers the tasks to the standby hybrid layer for processing. If the standby hybrid layer is not activated, the task inflow continues to be reduced until the primary hybrid layer has sufficient resources to resume processing. Through the backpressure response mechanism, the load can be actively managed. When the tasks are overloaded, the tasks can be smoothly transferred from the primary hybrid layer to the standby hybrid layer, ensuring stability during high load periods, avoiding crashes or performance degradation caused by task backlogs, and at the same time, through real-time monitoring and warning mechanisms, ensuring the ability to quickly respond to resource shortages and maintaining the high availability of the trusted data space.

[0037] Furthermore, the asynchronous message queue adjusts the task allocation according to the activation status of the standby hybrid layer, including: configuring a backpressure response mechanism in the standby hybrid layer; if the activation status of the standby hybrid layer is a busy activation state, generating a standby backpressure signal; and feeding back the standby backpressure signal to the asynchronous message queue to activate the emergency processing layer for emergency response management.

[0038] Specifically, a backpressure response mechanism is also configured in the standby hybrid layer, and the load condition of the standby hybrid layer is monitored in real time, and it is judged whether it is in a busy activation state according to the monitoring results. The busy activation state means that the standby hybrid layer has reached or is close to the maximum of its processing capacity, and at this time, this layer cannot continue to effectively process new tasks. If it is detected that the standby hybrid layer is in a busy activation state, a standby backpressure signal is triggered to notify the system that the current standby layer cannot continue to carry more tasks and the task flow needs to be regulated or transferred. The standby backpressure signal can be fed back to the asynchronous message queue in real time, thereby triggering task flow control and scheduling decisions. When the standby hybrid layer enters the busy activation state and generates a standby backpressure signal, the signal is passed to the asynchronous message queue. After receiving the standby backpressure signal, the asynchronous message queue adjusts the task allocation according to the current load condition. When neither the standby hybrid layer nor the primary hybrid layer can process new tasks, the emergency processing layer will be activated for emergency response management to ensure that important tasks can still be given priority when the system load is too high, and the emergency processing layer will process those tasks that can be delayed, thereby avoiding system crashes or delays. The asynchronous message queue adjusts the task allocation according to the activation state of the standby hybrid layer, ensuring a smooth transition of tasks when the load is too high and a reasonable allocation and processing of tasks through the standby and emergency processing layers when resources are insufficient, ensuring that tasks can be completed in a timely manner and maintaining the high availability and efficient operation of the system.

[0039] The step of synchronously establishing a distributed data storage layer by using the hot data prediction result further includes: configuring a zero-trust security architecture in the computing resource layer and the distributed data storage layer; and managing the data interaction between the computing resource layer and the distributed data storage layer through the zero-trust security architecture.

[0040] Specifically, on the basis of synchronously establishing a distributed data storage layer by using the prediction results of hotspot data, in order to ensure the security and controllability of data during large-scale distribution and access, a zero-trust security architecture is configured in the distributed data storage layer and the computing resource layer, and on this basis, data interaction management between the two layers is realized. The zero-trust security architecture is a modern network security model that emphasizes default distrust and continuous verification. It does not trust regardless of the user, device, application or network location. Every access request must undergo strict authentication and authorization to ensure the security of data access. That is, identity management is carried out for both the computing nodes and storage nodes participating in data interaction. To implement this mechanism, fine-grained security policies are embedded in each data access request, such as using identity-based access control or attribute-based access control models, and access decisions are made in combination with dimensions such as user identity, request purpose, access time, and data sensitivity. Through the zero-trust security architecture, the data interaction between the computing resource layer and the distributed data storage layer is managed. Each data request will go through links such as authentication, access control, and data encryption to ensure that only authorized computing resources can access specific data. Moreover, in the data interaction between the computing resource layer and the distributed data storage layer, according to the priority of tasks and resource requirements, computing resources are dynamically scheduled to ensure that high-priority tasks can be processed in a timely manner, while low-priority tasks are scheduled according to the system load and resource conditions. By configuring the zero-trust security architecture, the data access between the computing resource layer and the data storage layer can be effectively managed, preventing unauthorized access and data leakage, and ensuring the confidentiality, security, and availability of data.

[0041] Furthermore, after the dynamic expansion of the trusted data space is completed, it includes: establishing an operation data record, and identifying abnormal behaviors of the data record through a deep learning channel; activating a self-healing mechanism according to the abnormal behavior identification result to perform self-optimizing management of task execution.

[0042] Specifically, after completing the dynamic expansion of the trusted data space, data recording is performed during runtime, continuously collecting various key metrics and task execution data, including system performance data, task execution data, and service status data, etc. Among them, system performance data such as CPU usage, memory usage, disk I / O, network traffic, etc.; task execution data: such as task execution time, resource consumption, completion status, failure rate, etc.; service status data: such as service health status, task queue length, request response time, etc. Based on the collected runtime data, anomaly behavior recognition is performed through the deep learning channel. The deep learning model can analyze a large amount of historical runtime data, identify the differences between normal runtime patterns and anomaly behaviors, and can predict possible future anomalies based on these patterns. Deep learning models such as time series analysis models, supervised learning, etc. Once an anomaly behavior is identified, the self-healing mechanism is activated according to the identified anomaly results. The goal of the self-healing mechanism is to automatically repair problems in the system, restore the normal operation of the system, and perform self-optimizing management on the task execution process. The self-healing mechanism is managed in the following ways: According to the anomaly recognition results, automatically adjust the allocation of computing resources. For tasks that have failures or anomalies, automatically reschedule the tasks to idle computing resource nodes for execution, avoiding task failures due to insufficient resources or node failures. According to the priority and anomaly degree of the tasks, adjust the execution order of the tasks to ensure that critical tasks are given priority. For example, when the system detects that the execution time of a certain task is abnormally extended under high load, such as exceeding 1 hour, the self-healing mechanism will trigger an automatic expansion operation, adding more instances through Kubernetes to improve the task processing ability, thereby avoiding task delays caused by resource bottlenecks. On the basis of the self-healing mechanism, further improve the task execution efficiency through intelligent scheduling and self-optimizing management, including load balancing and dynamically adjusting the execution strategy. Load balancing automatically distributes tasks to different computing nodes to ensure that tasks can be evenly distributed among multiple nodes, avoiding single-point overload, and dynamically adjusting the task execution strategy according to the actual execution situation of the tasks. Through real-time monitoring and anomaly behavior recognition, and activating the self-healing mechanism, the need for manual intervention is reduced, and the autonomy and operation efficiency of the system are improved. Through self-optimizing management, the task scheduling and resource configuration can be adjusted according to real-time task data to ensure that tasks can be processed in a timely manner, improving the stability and response speed of the system.

[0043] Embodiment 2, based on the same inventive concept as the method for dynamically expanding the trusted data space in the microservice architecture in the foregoing embodiment, as Figure 2 shown, this application provides a system for dynamically expanding the trusted data space in the microservice architecture, where the system includes:

[0044] A policy establishment module 11 for establishing a hybrid extension policy for the computing resource layer, where the hybrid extension policy includes a Kubernetes, WASM, and FaaS hybrid policy; a task prediction module 12 for configuring a task database, performing task prediction based on the task database, and establishing a task prediction result, where the task prediction result carries a task feature identifier and a trust identifier; an adaptation evaluation module 13 for using the task prediction result to perform an adaptation evaluation of the hybrid extension policy within the computing resource layer, and pre-configuring the hybrid extension policy according to the adaptation evaluation result; a data prediction module 14 for performing a hot call prediction of data based on the hybrid extension policy and the task prediction result, and establishing a hot data prediction result; a dynamic expansion module 15 for using the hot data prediction result to synchronously establish a distributed data storage layer, and after establishing the mapping between the distributed data storage layer and the hybrid extension policy, completing the dynamic expansion of the trusted data space.

[0045] Further, the task prediction module 12 is further configured to perform the following steps: obtain the reserved tasks in the task database, and construct a first prediction result under the trust identifier according to the reserved tasks and the reserved task subjects; parse the task database, perform a time-point task backtracking according to the parsing result, and establish a time-point task backtracking result; extract the time-point task backtracking results that meet the preset probability, and establish a second prediction result under the trust identifier according to the extraction results; establish a task prediction result according to the first prediction result and the second prediction result.

[0046] Further, the adaptation evaluation module 13 is further configured to perform the following steps: obtain the Kubernetes configuration information of the current running state; call the task feature identifier in the task prediction result, and perform an execution adaptation analysis of the hybrid extension policy according to the task feature identifier respectively, and establish an adaptation evaluation result; configure the hybrid extension policy by using the adaptation evaluation result and the Kubernetes configuration information.

[0047] Further, the adaptation evaluation module 13 is further configured to perform the following steps: call the trust identifier in the task prediction result, and establish a first evaluation index according to the trust identifier; perform a time sensitivity analysis of the task prediction result, and establish a second evaluation index according to the time sensitivity analysis result; perform a task importance analysis of the task prediction result, and establish a third evaluation index according to the task importance analysis result; perform a multi-dimensional index screening by using the first evaluation index, the second evaluation index, and the third evaluation index, map and screen the adaptation evaluation result according to the multi-dimensional index screening result, and based on the Kubernetes configuration, a primary hybrid layer, a standby hybrid layer, and an emergency handling layer, use the primary hybrid layer, the standby hybrid layer, and the emergency handling layer as the hybrid extension policy.

[0048] Furthermore, the dynamic expansion module 15 is also used to perform the following steps: introduce an asynchronous message queue at the access layer, write the task request into the asynchronous message queue, and then pull and process it by the background; perform real-time monitoring of the task data in the asynchronous message queue, and establish a real-time monitoring data set; report a task queue warning according to the real-time monitoring data set.

[0049] Furthermore, the dynamic expansion module 15 is also used to perform the following steps: configure a backpressure response mechanism in the primary hybrid layer; when a primary backpressure signal appears in the primary hybrid layer, feedback the primary backpressure signal to the asynchronous message queue; the asynchronous message queue adjusts the task allocation according to the activation state of the standby hybrid layer.

[0050] Furthermore, the dynamic expansion module 15 is also used to perform the following steps: configure a backpressure response mechanism in the standby hybrid layer; if the activation state of the standby hybrid layer is a busy activation state, generate a standby backpressure signal; feedback the standby backpressure signal to the asynchronous message queue to activate the emergency processing layer for emergency response management.

[0051] Furthermore, the dynamic expansion module 15 is also used to perform the following steps: configure a zero-trust security architecture in the computing resource layer and the distributed data storage layer; manage the data interaction between the computing resource layer and the distributed data storage layer through the zero-trust security architecture.

[0052] Furthermore, the dynamic expansion module 15 is also used to perform the following steps: establish an operation data record, and identify abnormal behaviors of the data record through a deep learning channel; activate a self-healing mechanism according to the abnormal behavior identification result to perform self-optimization management of task execution.

[0053] The above are only the preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0054] This specification and the drawings are only exemplary descriptions of the present application, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and deformations to the present application without departing from the scope of the present application. Thus, if these modifications and deformations of the present application fall within the scope of the present application and its equivalent technologies, the present application is intended to include these changes and deformations.

Claims

1. A method for dynamically expanding a trusted data space in a microservices architecture, characterized in that The method includes: Establish a hybrid expansion strategy for the computing resource layer, where the hybrid expansion strategy includes a Kubernetes, WASM, and FaaS hybrid strategy; Configure a task database, perform task prediction based on the task database, and establish a task prediction result, where the task prediction result carries a task feature identifier and a trust identifier; Use the task prediction result to perform an adaptation evaluation of the hybrid expansion strategy within the computing resource layer, and pre-configure the hybrid expansion strategy according to the adaptation evaluation result; Based on the hybrid expansion strategy and the task prediction result, perform hot call prediction of data, and establish a hot data prediction result; Use the hot data prediction result to synchronously establish a distributed data storage layer, and after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, complete the dynamic expansion of the trusted data space.

2. The method for dynamically expanding a trusted data space of a microservice architecture according to claim 1, wherein The performing task prediction based on the task database and establishing a task prediction result includes: Obtain the reserved tasks in the task database, and construct a first prediction result under the trust identifier based on the reserved tasks and the reserved task subjects; Parse the task database, perform time-point task backtracking according to the parsing result, and establish a time-point task backtracking result; Extract the time-point task backtracking results that meet the preset probability, and establish a second prediction result under the trust identifier according to the extraction results; Establish a task prediction result based on the first prediction result and the second prediction result.

3. The method for dynamically expanding the trusted data space of the microservice architecture according to claim 1, characterized in that, The using the task prediction result to perform an adaptation evaluation of the hybrid expansion strategy within the computing resource layer and pre-configuring the hybrid expansion strategy according to the adaptation evaluation result includes: Obtain the Kubernetes configuration information of the current running state; Call the task feature identifier in the task prediction result, and perform execution adaptation analysis of the hybrid expansion strategy respectively according to the task feature identifier, and establish an adaptation evaluation result; Use the adaptation evaluation result and the Kubernetes configuration information to configure the hybrid expansion strategy.

4. The method for dynamically expanding the trusted data space of the microservice architecture according to claim 3, wherein, The using the adaptation evaluation result and the Kubernetes configuration information to configure the hybrid expansion strategy includes: Call the trust identifier in the task prediction result, and establish a first evaluation index according to the trust identifier; Perform time sensitivity analysis of the task prediction result, and establish a second evaluation index according to the time sensitivity analysis result; Perform task importance analysis of the task prediction result, and establish a third evaluation index according to the task importance analysis result; Use the first evaluation index, the second evaluation index, and the third evaluation index for multi-dimensional index screening. After mapping and screening the adaptation evaluation result according to the multi-dimensional index screening result, based on the Kubernetes configuration, the primary hybrid layer, the standby hybrid layer, and the emergency handling layer, use the primary hybrid layer, the standby hybrid layer, and the emergency handling layer as the hybrid expansion strategy.

5. The method for dynamically expanding a trusted data space of a microservice architecture according to claim 4, characterized in that, After establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, it includes: Introduce an asynchronous message queue at the access layer. After writing the task request into the asynchronous message queue, it is pulled and processed by the background; Perform real-time monitoring of the task data in the asynchronous message queue, and establish a real-time monitoring data set; Report a task queue warning based on the real-time monitoring data set.

6. The method for dynamically expanding a trusted data space of a microservice architecture according to claim 5, wherein After establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, it further includes: Configure a backpressure response mechanism in the primary hybrid layer; When a primary backpressure signal appears in the primary hybrid layer, feedback the primary backpressure signal to the asynchronous message queue; The asynchronous message queue adjusts task allocation according to the activation status of the standby hybrid layer.

7. The method for dynamically expanding the trusted data space of the microservice architecture according to claim 6, wherein, The asynchronous message queue adjusts task allocation according to the activation status of the standby hybrid layer, including: Configure a backpressure response mechanism in the standby hybrid layer; If the activation status of the standby hybrid layer is a busy activation status, generate a standby backpressure signal; Feedback the standby backpressure signal to the asynchronous message queue to activate the emergency processing layer for emergency response management.

8. The method for dynamically expanding the trusted data space of the microservice architecture according to claim 1, wherein, When using the hot data prediction result to synchronously establish the distributed data storage layer, it further includes: Configure a zero-trust security architecture in the computing resource layer and the distributed data storage layer; Manage data interaction between the computing resource layer and the distributed data storage layer through the zero-trust security architecture.

9. The method for dynamically expanding the trusted data space of the microservice architecture according to claim 1, wherein After completing the dynamic expansion of the trusted data space, it includes: Establish an operation data record and identify abnormal behaviors of the data record through the deep learning channel; Activate the self-healing mechanism according to the abnormal behavior identification result for self-optimization management of task execution.

10. A trusted data space dynamic expansion system for a microservices architecture, characterized in that, Steps for implementing the method for dynamically expanding the trusted data space of the microservice architecture according to any one of claims 1 to 9 include: A policy establishment module for establishing a hybrid expansion policy for the computing resource layer, where the hybrid expansion policy includes a Kubernetes, WASM, FaaS hybrid policy; A task prediction module for configuring a task database, performing task prediction according to the task database, and establishing a task prediction result, where the task prediction result carries a task feature identifier and a trust identifier; An adaptation evaluation module for using the task prediction result to perform adaptation evaluation of the hybrid expansion policy within the computing resource layer and pre-configuring the hybrid expansion policy according to the adaptation evaluation result; A data prediction module for performing hot call prediction of data based on the hybrid expansion policy and the task prediction result and establishing a hot data prediction result; A dynamic expansion module for using the hot data prediction result to synchronously establish the distributed data storage layer and, after establishing the mapping between the distributed data storage layer and the hybrid expansion policy, completing the dynamic expansion of the trusted data space.

Images