Dynamic extension method and system of trusted data space of micro-service architecture

Abstract

The application provides a trusted data space dynamic expansion method and system of a micro-service architecture, relates to the technical field of data processing, and establishes a hybrid expansion strategy of a computing resource layer, performs task prediction according to a task database, establishes a task prediction result, performs adaptive evaluation of the hybrid expansion strategy in the computing resource layer by using the task prediction result, preconfigures the hybrid expansion strategy, performs hot spot calling prediction of data, synchronously establishes a distributed data storage layer by using a hot spot data prediction result, and after mapping the distributed data storage layer and the hybrid expansion strategy, the dynamic expansion of the trusted data space is completed. The technical problem that the trusted data space of the micro-service architecture cannot flexibly cope with high concurrency and dynamic changes in tasks in the prior art is solved. The technical effects that the trusted data space is dynamically expanded, the flexibility of resource scheduling is improved, and data security and high availability are ensured are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a method and system for dynamically expanding a trusted data space of a micro-service architecture. BACKGROUND

[0002] With the wide application of cloud computing, micro-service architecture and big data technology, modern business systems are facing complex challenges of high concurrency of data, dynamic changes of tasks and real-time scheduling of resources. Most of the traditional resource expansion methods use static configuration, which cannot flexibly adjust the computing resources according to the real-time fluctuations of task load, often leading to problems of insufficient resources during peak periods and waste of resources during low periods. At the same time, the traditional system lacks the ability to predict and manage task behavior, and when processing massive requests or abnormal tasks, the response is slow, which easily causes system congestion or service interruption.

[0003] The prior art has the technical problem that the trusted data space of the micro-service architecture cannot flexibly cope with high concurrency and dynamic changes of tasks. SUMMARY

[0004] The present application provides a method and system for dynamically expanding a trusted data space of a micro-service architecture, to solve the technical problem that the prior art has the technical problem that the trusted data space of the micro-service architecture cannot flexibly cope with high concurrency and dynamic changes of tasks.

[0005] In view of the above problems, the present application provides a method and system for dynamically expanding a trusted data space of a micro-service architecture.

[0006] In a first aspect of the present application, a method for dynamically expanding a trusted data space of a micro-service architecture is provided, the method comprising: establishing a hybrid expansion strategy of a computing resource layer, the hybrid expansion strategy comprising a Kubernetes, WASM, FaaS hybrid strategy; configuring a task database, predicting tasks according to the task database, establishing a task prediction result, the task prediction result having a task feature identifier and a trust identifier; using the task prediction result to evaluate the adaptation of the hybrid expansion strategy in the computing resource layer, and preconfiguring the hybrid expansion strategy according to the evaluation result; predicting hot data calling based on the hybrid expansion strategy and the task prediction result, and establishing a hot data prediction result; using the hot data prediction result to synchronously establish a distributed data storage layer, and after mapping the distributed data storage layer and the hybrid expansion strategy, completing the dynamic expansion of the trusted data space.

[0007] In a second aspect of the present application, a trusted data space dynamic expansion system of a microservice architecture is provided, and the system comprises: a strategy establishing module configured to establish a hybrid expansion strategy of a computing resource layer, wherein the hybrid expansion strategy comprises a Kubernetes, WASM and FaaS hybrid strategy; a task prediction module configured to configure a task database, perform task prediction based on the task database, establish a task prediction result, and the task prediction result is provided with a task feature identifier and a trust identifier; an adaptation evaluation module configured to perform adaptation evaluation of the hybrid expansion strategy in the computing resource layer by using the task prediction result, and pre-configure the hybrid expansion strategy according to the adaptation evaluation result; a data prediction module configured to perform hot data call prediction based on the hybrid expansion strategy and the task prediction result, and establish a hot data prediction result; and a dynamic expansion module configured to establish a distributed data storage layer by using the hot data prediction result, and complete dynamic expansion of the trusted data space after mapping the distributed data storage layer and the hybrid expansion strategy.

[0008] The one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0009] The method provided by the embodiments of the present application establishes a hybrid expansion strategy of a computing resource layer, wherein the hybrid expansion strategy comprises a Kubernetes, WASM and FaaS hybrid strategy; configures a task database, performs task prediction based on the task database, establishes a task prediction result, and the task prediction result is provided with a task feature identifier and a trust identifier; performs adaptation evaluation of the hybrid expansion strategy in the computing resource layer by using the task prediction result, and pre-configure the hybrid expansion strategy according to the adaptation evaluation result; performs hot data call prediction based on the hybrid expansion strategy and the task prediction result, and establishes a hot data prediction result; and establishes a distributed data storage layer by using the hot data prediction result, and completes dynamic expansion of the trusted data space after mapping the distributed data storage layer and the hybrid expansion strategy. The technical effect of dynamic expansion of the trusted data space is achieved, the flexibility of resource scheduling is improved, and data security and high availability are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0011] Figure 1 A flowchart of a trusted data space dynamic expansion method of a microservice architecture is provided in the present application.

[0012] Figure 2 A structural schematic diagram of a trusted data space dynamic expansion system of a micro-service architecture provided in the present application.

[0013] Reference signs: policy establishment module 11, task prediction module 12, adaptation evaluation module 13, data prediction module 14, dynamic expansion module 15. DETAILED DESCRIPTION

[0014] The present application provides a trusted data space dynamic expansion method and system of a micro-service architecture, which is used to solve the technical problem that the trusted data space of the micro-service architecture cannot flexibly cope with high concurrency and dynamic task changes in the prior art. The technical effects of dynamic expansion of the trusted data space, improvement of the flexibility of resource scheduling, and guarantee of data security and high availability are achieved.

[0015] Hereinafter, the technical solutions in the present application will be described clearly and completely with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application. In addition, it should be noted that, for convenience of description, only parts related to the present application are shown in the drawings, rather than all.

[0016] In one embodiment, as shown in the accompanying drawings, the present application provides a trusted data space dynamic expansion method of a micro-service architecture, which comprises the following steps: Figure 1

[0017] establishing a hybrid expansion strategy of the computing resource layer, wherein the hybrid expansion strategy comprises a Kubernetes, WASM, and FaaS hybrid strategy.

[0018] ​Specifically, before building the hybrid expansion strategy, first of all, the target of the hybrid expansion strategy is clear, that is, in order to realize the dynamic expansion of trusted data space of micro service architecture, including elastic expansion, performance optimization, resource optimization, etc. Elastic expansion refers to the automatic adjustment of the number of resources by the resource layer according to the load change, performance optimization refers to ensuring that the micro service architecture can efficiently handle different types of computing tasks, reduce delay and resource waste. Resource optimization refers to reducing resources at low load and increasing resources at high load to avoid unnecessary resource consumption. Then, according to the target, the hybrid expansion strategy of the computing resource layer is established. The computing resource layer is the system computing resource where various tasks are executed, including containers, virtual machines, service instances, etc., responsible for processing user requests, data computing and service response functions. The hybrid expansion strategy can select the most suitable expansion mode according to the characteristics of the task, system load and resource demand, including Kubernetes, WASM and FaaS hybrid strategy. Among them, Kubernetes is a commonly used container orchestration tool in micro service architecture, responsible for managing containers in the cluster, and in the hybrid expansion strategy, it is responsible for the expansion and management of computing resources. WebAssembly (WASM) is a lightweight binary format suitable for high-performance computing tasks, especially when a large number of calculations need to be performed in micro services, WASM can provide fast computing power. FaaS can decompose computing tasks into independent functions and dynamically execute these functions as needed. In the hybrid expansion strategy, FaaS can be used to handle short-term, high-concurrency or irregular tasks. By configuring the Kubernetes cluster and HPA, automatically expanding or shrinking service instances according to the load of the service, such as CPU or memory usage, integrating WASM modules, compiling high-computing-demand service modules into WASM modules and deploying them to micro services, ensuring that these computing-intensive tasks can run efficiently in the WASM environment. At the same time, deploy the FaaS environment, use functions to handle short-term, burst requests, automatically start computing instances according to task trigger conditions, establish a hybrid expansion strategy of the resource layer, and use service mesh such as Istio to manage communication between micro services, monitor resource usage, and expand or reduce resources when needed. By combining the hybrid expansion strategy of Kubernetes, WASM and FaaS, micro service architecture can dynamically adjust computing resources according to different task types and load requirements, utilize the automatic expansion of Kubernetes containers, the efficient computing of WASM and the elastic scaling of FaaS to ensure the high performance and high availability of micro service architecture under high load and large-scale concurrency, while reducing resource waste and providing flexible resource management capabilities.

[0019] Configure a task database, perform task prediction based on the task database, establish a task prediction result, and the task prediction result has a task feature identifier and a trust identifier.

[0020] Specifically, a task database for storing task information and task status is configured, which records the relevant data of all tasks in the microservice architecture, such as task type, task parameter, task status, task creation time, expected execution time, execution result, etc. The task database provides historical data and patterns for task prediction. By analyzing these data, the characteristics of future tasks can be predicted. Based on the historical data and scheduled tasks in the task database, task prediction is performed through data analysis to predict the demand and load of future tasks, and a task prediction result is established. The task prediction result is the output data generated based on the task prediction process, with a task feature identifier. The task feature identifier is an identifier assigned to each task according to its type, priority, resource demand, etc., used to identify and process tasks. Each task will be classified into different types or priorities according to its feature identifier, thereby affecting subsequent resource allocation and task scheduling strategies. By configuring the task database, performing task prediction and generating the task prediction result, the microservice architecture can predict future task demand in advance, automatically expand and schedule resources, thereby optimizing resource allocation and scheduling, improving resource utilization and response speed.

[0021] Further, the task prediction based on the task database and the establishment of the task prediction result include: obtaining scheduled tasks in the task database, constructing a first prediction result under a trust identifier according to the scheduled tasks and the subject of the scheduled tasks; analyzing the task database, performing time point task backtracking according to the analysis result, and establishing a time point task backtracking result; extracting time point task backtracking results that meet a preset probability, and establishing a second prediction result under the trust identifier according to the extracted results; and establishing the task prediction result according to the first prediction result and the second prediction result.

[0022] Specifically, a scheduled task is proposed from the task database, which refers to a task set in advance and scheduled to be executed at a certain time point or time period. The task database records relevant information of the scheduled task, such as task type, execution time, priority, resource demand, etc. Then, the first prediction result under the trust identification is constructed according to the scheduled task and the scheduled task subject, which refers to the subject or dependent object of the task execution, such as the user, system module or certain service initiating the task, etc. By identifying the task subject, the background, importance and execution dependency of the task can be understood, further helping the task prediction and scheduling. The trust identification is based on the evaluation result of the task history data and the task subject, and is used to identify the reliability and importance of the task. In the task prediction process, the trust identification helps to prioritize the tasks and decide which tasks need more resources or faster execution. For example, the trust identification of the task is constructed by the task execution history, task priority and reputation of the task subject. Further, according to the trust identification of the task subject and the task, the first prediction result of the scheduled task is generated, such as the predicted execution time and resource demand.

[0023] After obtaining the reservation task, the historical task data in the task database is parsed. First, the historical task data is preprocessed to remove invalid or erroneous data, such as missing values or format error records. Then, according to the task type, execution time, resource consumption and other fields, the features are extracted to obtain the parsing results, including task execution time, resource consumption, task status and task category, etc. According to the parsing results, time window or specific time point, the task is traced back to extract the task execution features before and after a specific time point from the historical task data, analyze and identify the periodic or time-dependent task load pattern, such as resource consumption, time consumption, etc. during task execution, and establish the time point task trace back result. The time point task trace back result refers to tracing back to the historical task data within a period of time before a specific time point based on the specific time point, analyzing and summarizing the execution of past tasks, which can provide a basis for predicting future tasks in a similar time period. Time point task trace back helps to identify periodic or time-dependent task load patterns. For example, an e-commerce platform found in the past month's data that the order processing task between 9:00 and 11:00 am every day often causes delays due to too many concurrent requests. These historical tasks can be analyzed through time point task trace back to predict the task load in a similar time period in the future and make corresponding resource scheduling adjustments. Further, after completing the time point task trace back, the historical data obtained by the trace back is extracted to meet the preset conditions, which are set according to the regularity of historical data, such as a preset probability. When the execution time of the task exceeds the preset percentage standard, such as when the execution time of the task exceeds 80% of the average value, or when the resource demand exceeds a certain threshold, the task is considered to be a high-load task, and the data result meeting the preset probability condition is extracted. According to the task data extracted by the trace back combined with the trust identifier, a second prediction result is generated, which contains more detailed task load features, such as high-load tasks. Finally, the first prediction result obtained based on the reservation task and the trust identifier is combined with the second prediction result obtained based on the analysis of the time point task trace back and the task identifier to generate the final task prediction result. By combining reservation task analysis, task trace back and trust identifier, the execution of the task can be accurately predicted, the load peak and valley can be identified in advance, intelligent expansion and contraction can be performed, the efficient operation of the microservice architecture under different loads can be ensured, and the overall service quality can be improved.

[0024] The task prediction result is used to evaluate the adaptation of the mixed expansion strategy in the computing resource layer, and the mixed expansion strategy is preconfigured according to the adaptation evaluation result.

[0025] Further, the adaptation evaluation of the computing resource layer mixed expansion strategy using the task prediction result includes: obtaining the Kubernetes configuration information of the current running state; calling the task feature identifier in the task prediction result, performing execution adaptation analysis of the mixed expansion strategy according to the task feature identifier, and establishing an adaptation evaluation result; and configuring the mixed expansion strategy using the adaptation evaluation result and the Kubernetes configuration information.

[0026] Specifically, the Kubernetes configuration information of the current running state is obtained through the Kubernetes API, including Pod configuration, HPA configuration, and resource usage, etc. The Pod configuration refers to the number of deployed container instances, resource allocation, such as CPU and memory usage, etc. HPA (HorizontalPodAutoscaler) is used to automatically expand the number of Pods according to CPU usage and other indicators. The resource usage includes the usage of CPU, memory, storage, etc. to evaluate whether expansion or adjustment is needed. The task feature identifier in the task prediction result is extracted to identify the key characteristics of each task, and the adaptation degree of the task feature identifier and the current mixed expansion strategy is evaluated, which is a process of evaluating the matching degree of the task prediction result and the current resource configuration, used to judge whether the existing resource configuration and expansion strategy can meet the task requirements, including resource demand evaluation and expansion strategy adaptation, for example, evaluating whether the task resource demand CPU and memory match the current resource configuration, and whether the number of Pods needs to be expanded if they do not match. According to the priority and resource demand of the task, it is judged whether the existing expansion strategy is sufficient, whether HPA can meet the demand of task load, or whether FaaS needs to be used to handle burst tasks. The judgment process can use Kubernetes HPA combined with task prediction results to automatically evaluate the adaptability of the existing resource configuration and generate an adaptation evaluation result. Then, based on the adaptation evaluation, the mixed expansion strategy is further configured using the adaptation evaluation result and the Kubernetes configuration information. By analyzing the characteristics and resource demand of the task, the adaptability of the existing resource configuration is evaluated, and appropriate expansion strategy is pre-configured according to the evaluation result to ensure that the trusted space of the microservice architecture can dynamically and flexibly adjust resources according to the demand of the task, meet the task demand, and at the same time guarantee the efficient operation of the system.

[0027] Further, the mixed expansion strategy configured by using the adaptation evaluation result and the Kubernetes configuration information comprises: calling a trust identifier in a task prediction result, establishing a first evaluation index according to the trust identifier; performing time sensitivity analysis on the task prediction result, establishing a second evaluation index according to a time sensitivity analysis result; performing task importance analysis on the task prediction result, establishing a third evaluation index according to a task importance analysis result; performing multi-dimensional index screening by using the first evaluation index, the second evaluation index and the third evaluation index; and after mapping and screening the adaptation evaluation result according to a multi-dimensional index screening result, taking a primary mixed layer, a backup mixed layer and an emergency processing layer as the mixed expansion strategy based on the Kubernetes configuration.

[0028] Specifically, first, the trust label representing the credibility of the task is extracted from the task prediction result, and a first evaluation index is established based on the trust label of the task. The first evaluation index is used to quantify the credibility of the task, and the first evaluation index value of the task with high credibility is also high. According to the time sensitivity analysis of the task prediction result, the dependence of the task on time is evaluated. The time sensitivity analysis refers to analyzing the timeliness requirement of the task, based on which it can be judged whether the task has strict requirements for delay. For example, some tasks may need to be completed as soon as possible, such as payment verification and transaction settlement, while other tasks can tolerate longer delays, such as report generation and data synchronization. According to the time sensitivity analysis result, a second evaluation index is generated, which is based on the time sensitivity analysis result of the task and is used to quantify the urgency and timeliness requirement of the task within a certain time period. For tasks with high time sensitivity, the value of the second evaluation index is higher, indicating that the task needs to be processed in priority. Then, the importance of the task in the task prediction result is analyzed, and a third evaluation index is generated to quantify the overall impact of the task on the system. The importance of the task is usually determined based on business requirements, and tasks with high importance will have higher third evaluation indexes, thus obtaining more resources and higher execution priority. After obtaining multiple evaluation indexes such as the first evaluation index, the second evaluation index and the third evaluation index, the characteristics and requirements of the tasks are comprehensively evaluated according to the multiple evaluation indexes, and screening and sorting are performed to determine which tasks need to be executed in priority, which tasks can be gradually executed when resources are sufficient, and which tasks should be treated as emergency processing tasks when the system load is high, obtaining a multi-dimensional index screening result. Then, the multi-dimensional index screening result is mapped to the adaptive evaluation result for screening, which can accurately identify and classify the priority of the tasks, and divide the tasks into different levels to provide a basis for subsequent resource scheduling and expansion strategy configuration, ensuring that critical tasks can obtain timely resource allocation, while ensuring the efficient operation and stability of the system under different load conditions. Through mapping and screening, the tasks with the highest scores are mapped to the primary hybrid layer, which are usually the most priority tasks; the tasks with medium scores are mapped to the standby hybrid layer, which will be started when resources allow; and the tasks with the lowest scores are mapped to the emergency processing layer, which will only be executed in resource shortage or abnormal conditions. After mapping and screening, the Kubernetes is configured based on the task requirements to configure the primary hybrid layer, the standby hybrid layer and the emergency processing layer, and the primary hybrid layer, the standby hybrid layer and the emergency processing layer are used as the hybrid expansion strategy. The primary hybrid layer contains tasks with the highest scores after multi-dimensional index screening, which are usually the most priority tasks in the system. The standby hybrid layer contains tasks that may be executed, which will be executed when resources are sufficient, usually in a gray start state, waiting for system load changes to trigger. The emergency processing layer contains tasks that need to be processed in emergency, which are usually tasks executed in system resource shortage or abnormal conditions, with lower priority but need to be processed in emergency.Through multi-dimensional index screening and priority division, a hybrid expansion strategy is dynamically configured, different execution levels and resource strategies are configured for each type of task, delay is reduced, and task execution efficiency is improved.

[0029] Based on the hybrid expansion strategy and the task prediction result, a hot data prediction result is established.

[0030] Specifically, according to the hybrid expansion strategy and the task prediction result, the hot calling of data is predicted. The hot calling prediction is based on historical tasks and data access patterns to predict the data parts that may be frequently accessed in the future. The hot calling prediction can be realized by training a prediction model. For example, first, obtain historical task data, hybrid expansion strategy, task prediction result and other related data as training data, select a suitable machine learning model such as a decision tree model, and train the model through the training data. During the training process, the model learns how to predict the hotness of data access according to the input features. The model output result is to predict whether a certain data or service becomes hot. After the model is trained and evaluated, it can be used for hot calling prediction. Hot data usually refers to data that is frequently accessed or requires high priority resources within a certain time. Predicting these hot data helps to prepare resources in advance and avoid access delay. According to the hot calling prediction, the prediction result is converted into a hot data prediction result, which specifically indicates which data will be frequently accessed or require higher priority resources in the future. Through the prediction of these data, sufficient computing resources can be allocated to them in advance to optimize data access speed and response capability. For example, Kubernetes can dynamically increase cache instances according to the predicted hot data request volume, and FaaS can provide higher priority for hot data processing tasks to ensure timely response of tasks under high load. Through hot data prediction, more resources can be allocated to high-access-frequency data in advance to avoid access delay and improve data access response speed and data processing capability. Especially in high-concurrency and critical task scenarios, by predicting and loading hot data in advance, tasks can be ensured to respond in time, reducing user experience problems caused by delay.

[0031] The hot data prediction result is used to synchronize the establishment of a distributed data storage layer, and after the mapping of the distributed data storage layer and the hybrid expansion strategy is established, the dynamic expansion of the trusted data space is completed.

[0032] In particular, using the hotspot data prediction results, analyze which data will become hotspots in the future. These data can be because of the increase in user access frequency, or the task prediction results show that certain data will be frequently accessed at a certain time period. By analyzing the hotspot data prediction results, these data can be loaded into the distributed data storage system in advance to ensure that data can be accessed efficiently during high load. After identifying and loading the hotspot data, use tools such as Apache Cassandra, Amazon DynamoDB, HDFS, etc. to build a distributed data storage layer. The distributed data storage layer refers to an architecture that stores data distributedly on multiple computing nodes or servers. In the distributed data storage layer, data is stored and managed by a distributed database or file system to improve data availability, performance, and fault tolerance. Through distributed storage, large-scale concurrent access can be efficiently handled, and data can still be accessed normally in the event of node failure. After building the distributed data storage layer, map the resource requirements of the distributed data storage layer to the hybrid expansion strategy to achieve computing-storage collaborative expansion, i.e., computing resources and storage resources are expanded together to ensure that hotspot tasks not only have sufficient computing resources but also can quickly access data, thereby greatly improving overall response efficiency. For example, according to the hotspot data prediction results, identify the data set that will be accessed frequently in the future and synchronously build a distributed data storage layer. Then, according to the hybrid expansion strategy, evaluate which computing resources the tasks or services associated with these hotspot data should be deployed on, such as running some high-frequency data request services in WASM containers to improve response speed, or using FaaS to achieve rapid expansion and contraction. At this time, a mapping relationship is established between the storage nodes of the distributed data storage layer and the hybrid expansion strategy, i.e., which computing node should prioritize processing certain type of data, how to route and load balance, etc., thereby realizing the close coupling of data flow and resource allocation. Further, after establishing the mapping between the distributed data storage layer and the hybrid expansion strategy, dynamic expansion of the trusted data space is completed. The trusted data space refers to ensuring the security, reliability, and compliance of data during data storage, processing, and access, while ensuring the trust transfer and transparent management of data between different systems. Through dynamic expansion of the trusted data space, the number or capacity of computing resources can be automatically adjusted according to the load and task requirements, and resources can be automatically expanded or contracted to ensure high availability and high performance of the trusted space of the microservice architecture. By automatically building a distributed data architecture driven by hotspot data prediction and coordinating deployment with a hybrid expansion strategy, a full-range elastic expansion capability from service instances to data layers is built to support dynamic scaling and safe and reliable operation of the trusted data space in large-scale, high-concurrency scenarios.

[0033] Further, after the mapping of the distributed data storage layer and the hybrid expansion strategy is established, an asynchronous message queue is introduced in the access layer, and after a task request is written into the asynchronous message queue, the task request is pulled and processed by a background; real-time monitoring of task data of the asynchronous message queue is performed to establish a real-time monitoring data set; and a task queue early warning is reported according to the real-time monitoring data set.

[0034] Specifically, after the mapping of the distributed data storage layer and the hybrid expansion strategy is established, an asynchronous message queue is introduced in the access layer, and all task requests are written into the asynchronous message queue in the access layer. The asynchronous message queue is a decoupling component that allows the generation of task requests and the asynchronous processing of tasks, thereby reducing the delay when the task request is generated and improving the throughput of the system. After the task request is written into the message queue, the background pulls and processes the tasks from the message queue. By monitoring the task data in the asynchronous message queue in real time, the state and processing of the task request can be dynamically mastered, and a real-time monitoring data set is formed according to the monitoring data. The real-time monitoring data set continuously records various indicators of the task, such as the task queue time, processing time, number of successes and failures, etc. Through these monitoring data, the execution efficiency of the task and the consumption of resources can be understood, and corresponding adjustments can be made based on this information. Further, based on the data information in the real-time monitoring data set, a task queue early warning is performed, and the task queue early warning function can timely discover potential task processing bottlenecks or resource shortages in the real-time monitoring data set. When the processing speed of some tasks decreases, the queue is seriously backed up, or the task failure rate increases, the early warning mechanism is triggered, reminding the operation and maintenance personnel or automatically triggering resource expansion operations. Through real-time early warning, measures can be taken before the task processing is abnormal, avoiding the impact of service interruption or task backlog, and ensuring the high availability of the service.

[0035] Further, after the mapping of the distributed data storage layer and the hybrid expansion strategy is established, an asynchronous message queue is introduced in the access layer, and after a task request is written into the asynchronous message queue, the task request is pulled and processed by a background; real-time monitoring of task data of the asynchronous message queue is performed to establish a real-time monitoring data set; and a task queue early warning is reported according to the real-time monitoring data set.

[0036] Specifically, after the distributed data storage layer is established and mapped with the hybrid expansion strategy, a back pressure response mechanism is further introduced to automatically adjust the processing flow of tasks under high load conditions. The primary hybrid layer is responsible for processing tasks with the highest multi-dimensional indicators, usually including those tasks that are critical to business and have high resource requirements. To ensure the stability and performance of the primary hybrid layer, a back pressure response mechanism is configured in the primary hybrid layer. The back pressure response mechanism refers to a technique that actively controls the flow by limiting the processing rate of tasks to prevent the system from collapsing due to overload. The back pressure mechanism usually sends feedback signals to the task queue or system, requiring temporary reduction of request volume or delayed processing. When it is detected that the resource consumption of the primary hybrid layer exceeds the expectation, a primary back pressure signal is generated and fed back to the asynchronous message queue. The primary back pressure signal is a feedback signal from the primary hybrid layer indicating that the current processing capacity of the primary hybrid layer has reached or is close to the upper limit, and it is necessary to reduce the task inflow or wait for system resources to be free before processing. After receiving the primary back pressure signal, the asynchronous message queue adjusts the allocation of tasks according to the activation state of the standby hybrid layer. If the standby hybrid layer is in an active state, i.e., the resources are sufficient and can process tasks, the queue transfers the tasks to the standby hybrid layer for processing. If the standby hybrid layer is not activated, the task inflow is reduced until the primary hybrid layer has sufficient resources to resume processing. Through the back pressure response mechanism, the load can be actively managed, and when the task is overloaded, the task can be smoothly transferred from the primary hybrid layer to the standby hybrid layer, ensuring stability during high load and avoiding collapse or performance degradation caused by task backlog. At the same time, through real-time monitoring and early warning mechanisms, it can quickly respond to resource shortages and maintain high availability of trusted data space.

[0037] Further, the asynchronous message queue adjusts the task allocation according to the activation state of the standby hybrid layer, including: configuring a back pressure response mechanism in the standby hybrid layer; if the activation state of the standby hybrid layer is a busy activation state, a standby back pressure signal is generated; the standby back pressure signal is fed back to the asynchronous message queue to activate the emergency processing layer for emergency response management.

[0038] Specifically, a back pressure response mechanism is also configured in the backup hybrid layer, and the load condition of the backup hybrid layer is monitored in real time, and whether it is in a busy activation state is judged according to the monitoring result. The busy activation state refers to that the backup hybrid layer has reached or approached the maximum value of its processing capacity, at which time the layer cannot continue to effectively process new tasks. If it is detected that the backup hybrid layer is in the busy activation state, a backup back pressure signal is triggered to inform the system that the current backup layer cannot continue to carry more tasks and needs to regulate or transfer the task flow. The backup back pressure signal can be fed back to the asynchronous message queue in real time, thereby triggering task flow control and scheduling decisions. When the backup hybrid layer enters the busy activation state and generates the backup back pressure signal, the signal is transmitted to the asynchronous message queue. After receiving the backup back pressure signal, the asynchronous message queue adjusts the task allocation according to the current load condition, and when the backup hybrid layer and the main hybrid layer cannot process new tasks, the emergency processing layer will be activated for emergency response management, ensuring that important tasks can still be prioritized when the system load is too high, while the emergency processing layer will process tasks that can be delayed, thereby avoiding system crashes or delays. The asynchronous message queue adjusts the task allocation according to the activation state of the backup hybrid layer, ensuring that the task can be smoothly transitioned when the load is too high, and that the task can be reasonably allocated and processed through the backup and emergency processing layers when the resources are insufficient, ensuring that the task can be completed in time, maintaining the high availability and efficient operation of the system.

[0039] The distributed data storage layer is established by synchronously using the hotspot data prediction result, further comprising: configuring a zero-trust security architecture in the computing resource layer and the distributed data storage layer; and performing data interaction management of the computing resource layer and the distributed data storage layer through the zero-trust security architecture.

[0040] Specifically, on the basis of the establishment of a distributed data storage layer using hotspot data prediction results, in order to ensure the security and controllability of data in the process of large-scale distribution and access, a zero-trust security architecture is configured in the distributed data storage layer and the computing resource layer, and data interaction management between the two layers is realized on this basis. The zero-trust security architecture is a modern network security model that emphasizes default distrust and continuous verification. Regardless of the user, device, application or network location, every access request must undergo strict identity verification and authorization to ensure the security of data access. That is, the identity management of the computing nodes and storage nodes participating in data interaction is carried out. To achieve this mechanism, fine-grained security policies are embedded in each data access request, such as using identity-based access control or attribute-based access control model, combining user identity, request purpose, access time, data sensitivity and other dimensions to make access decisions. Through the zero-trust security architecture, data interaction between the computing resource layer and the distributed data storage layer is managed. Each data request will go through identity verification, access control and data encryption, etc. to ensure that only authorized computing resources can access specific data. And in the data interaction between the computing resource layer and the distributed data storage layer, according to the priority of the task and the resource demand, the computing resources are dynamically scheduled to ensure that high-priority tasks can be processed in time, while low-priority tasks are scheduled according to the system load and resource situation. By configuring the zero-trust security architecture, data access between the computing resource layer and the data storage layer can be effectively managed, unauthorized access and data leakage can be prevented, and the confidentiality, security and availability of data can be ensured.

[0041] Further, after completing the dynamic expansion of the trusted data space, the method comprises: establishing a running data record and identifying abnormal behavior of the data record through a deep learning channel; activating a self-healing mechanism according to the abnormal behavior identification result to perform self-optimization management of the task execution.

[0042] Specifically, after completing the dynamic expansion of the trusted data space, data records are made at runtime, and various key indicators and task execution data are continuously collected, including system performance data, task execution data, and service status data, etc. The system performance data includes CPU usage, memory usage, disk IO, network traffic, etc. The task execution data includes execution time, resource consumption, completion status, failure rate, etc. The service status data includes service health status, task queue length, request response time, etc. Based on the collected running data, abnormal behavior recognition is performed through a deep learning channel. The deep learning model can analyze a large amount of historical running data, identify the difference between normal operation mode and abnormal behavior, and predict possible future abnormal situations based on these patterns. The deep learning model includes time series analysis model, supervised learning, etc. Once abnormal behavior is identified, the self-healing mechanism is activated according to the identified abnormal result. The goal of the self-healing mechanism is to automatically repair problems in the system, restore normal operation of the system, and perform self-optimization management on the task execution process. The self-healing mechanism manages in the following ways: according to the abnormal identification result, automatically adjust the allocation of computing resources. For tasks that have failed or are abnormal, automatically reschedule the tasks to idle computing resource nodes for execution, avoiding task failure due to insufficient resources or node failure. According to the priority and abnormal degree of the task, adjust the execution order of the task to ensure that critical tasks are processed first. For example, the system detects that a task is abnormally delayed in execution under high load, such as more than 1 hour, and the self-healing mechanism triggers automatic expansion operation to increase more instances through Kubernetes to improve task processing capacity, thereby avoiding task delay due to resource bottleneck. On the basis of the self-healing mechanism, intelligent scheduling and self-optimization management are further used to improve the efficiency of task execution, including load balancing and dynamic adjustment of execution strategy. Load balancing automatically distributes tasks to different computing nodes to ensure that tasks are evenly distributed among multiple nodes to avoid single-point overload. According to the actual execution of the task, the task execution strategy is dynamically adjusted. Through real-time monitoring and abnormal behavior recognition, and activation of the self-healing mechanism, the need for manual intervention is reduced, and the autonomy and operation efficiency of the system are improved. Through self-optimization management, task scheduling and resource configuration can be adjusted according to real-time task data to ensure that tasks can be processed in a timely manner, improving the stability and response speed of the system.

[0043] In the second embodiment, based on the same inventive concept as the method for dynamically expanding the trusted data space of the microservice architecture in the preceding embodiments, as shown in the following table, the present application provides a system for dynamically expanding the trusted data space of the microservice architecture, wherein the system comprises: Figure 2

[0044] ​The strategy establishing module 11 is configured to establish a hybrid expansion strategy of the computing resource layer, and the hybrid expansion strategy comprises a Kubernetes, WASM and FaaS hybrid strategy; the task prediction module 12 is configured to configure a task database, perform task prediction according to the task database, establish a task prediction result, and the task prediction result is provided with a task feature identifier and a trust identifier; the adaptive evaluation module 13 is configured to perform adaptive evaluation of the hybrid expansion strategy in the computing resource layer by using the task prediction result, and pre-configure the hybrid expansion strategy according to the adaptive evaluation result; the data prediction module 14 is configured to perform hot data call prediction based on the hybrid expansion strategy and the task prediction result, and establish a hot data prediction result; and the dynamic expansion module 15 is configured to synchronize the establishment of a distributed data storage layer by using the hot data prediction result, and complete the dynamic expansion of the trusted data space after mapping the distributed data storage layer and the hybrid expansion strategy.

[0045] Further, the task prediction module 12 is further configured to perform the following steps: obtaining a pre-booking task in the task database, constructing a first prediction result under the trust identifier according to the pre-booking task and a pre-booking task subject; parsing the task database, performing time point task backtracking according to the parsing result, and establishing a time point task backtracking result; extracting the time point task backtracking result satisfying a preset probability, and establishing a second prediction result under the trust identifier according to the extraction result; and establishing the task prediction result according to the first prediction result and the second prediction result.

[0046] Further, the adaptive evaluation module 13 is further configured to perform the following steps: obtaining Kubernetes configuration information of a current running state; calling a task feature identifier in the task prediction result, respectively performing execution adaptive analysis of the hybrid expansion strategy according to the task feature identifier, and establishing an adaptive evaluation result; and configuring the hybrid expansion strategy by using the adaptive evaluation result and the Kubernetes configuration information.

[0047] Further, the adaptive evaluation module 13 is further configured to perform the following steps: calling a trust identifier in the task prediction result, establishing a first evaluation index according to the trust identifier; performing time sensitivity analysis of the task prediction result, establishing a second evaluation index according to the time sensitivity analysis result; performing task importance analysis of the task prediction result, establishing a third evaluation index according to the task importance analysis result; performing multi-dimensional index screening by using the first evaluation index, the second evaluation index and the third evaluation index, mapping and screening the adaptive evaluation result according to the multi-dimensional index screening result, and taking the primary hybrid layer, the standby hybrid layer and the emergency processing layer as the hybrid expansion strategy based on the Kubernetes configuration.

[0048] Further, the dynamic expansion module 15 is further configured to perform the following steps: introducing an asynchronous message queue in the access layer, writing a task request into the asynchronous message queue, and then pulling the task request by the background; performing real-time monitoring of the task data of the asynchronous message queue, and establishing a real-time monitoring data set; and issuing a task queue warning according to the real-time monitoring data set.

[0049] Further, the dynamic expansion module 15 is further configured to perform the following steps: configuring a back pressure response mechanism in the primary hybrid layer; when a primary back pressure signal appears in the primary hybrid layer, feeding back the primary back pressure signal to the asynchronous message queue; and adjusting the task distribution according to the activation state of the standby hybrid layer.

[0050] Further, the dynamic expansion module 15 is further configured to perform the following steps: configuring a back pressure response mechanism in the standby hybrid layer; if the activation state of the standby hybrid layer is a busy activation state, generating a standby back pressure signal; and feeding back the standby back pressure signal to the asynchronous message queue to activate the emergency processing layer to perform emergency response management.

[0051] Further, the dynamic expansion module 15 is further configured to perform the following steps: configuring a zero-trust security architecture in the computing resource layer and the distributed data storage layer; and performing data interaction management of the computing resource layer and the distributed data storage layer through the zero-trust security architecture.

[0052] Further, the dynamic expansion module 15 is further configured to perform the following steps: establishing a running data record, and identifying abnormal behaviors of the data record through a deep learning channel; activating a self-healing mechanism according to the abnormal behavior identification result, and performing self-optimization management of task execution.

[0053] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0054] The specification and drawings are merely exemplary of the present application, and any and all modifications, variations, combinations or equivalents that are within the scope of the present application should be considered covered by the present application. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is intended to include these modifications and variations.

Claims

1. A method for dynamic scaling of a trusted data space of a microservices architecture, characterized in that, The method comprises: A hybrid expansion strategy of a computing resource layer is established, the hybrid expansion strategy comprising a Kuber netes, WASM, FaaS hybrid strategy; A task database is configured, task prediction is performed according to the task database, a task prediction result is established, the task prediction result is provided with a task feature identifier and a trust identifier; Adaptation evaluation of the hybrid expansion strategy in the computing resource layer is performed by using the task prediction result, and the hybrid expansion strategy is preconfigured according to the adaptation evaluation result; Hot spot calling prediction of data is performed based on the hybrid expansion strategy and the task prediction result, and a hot spot data prediction result is established; After the hot spot data prediction result is used to synchronously establish a distributed data storage layer, and the distributed data storage layer is mapped with the hybrid expansion strategy, dynamic expansion of a trusted data space is completed; The adaptation evaluation of the hybrid expansion strategy in the computing resource layer by using the task prediction result, and the preconfiguration of the hybrid expansion strategy according to the adaptation evaluation result, comprise: Kubernetes configuration information of a current running state is obtained; A task feature identifier in the task prediction result is called, execution adaptation analysis of the hybrid expansion strategy is respectively performed according to the task feature identifier, and an adaptation evaluation result is established; The hybrid expansion strategy is configured by using the adaptation evaluation result and the Kubernetes configuration information; The configuration of the hybrid expansion strategy by using the adaptation evaluation result and the Kubernetes configuration information comprises: A trust identifier in the task prediction result is called, and a first evaluation index is established according to the trust identifier; Time sensitivity analysis of the task prediction result is performed, and a second evaluation index is established according to a time sensitivity analysis result; Task importance analysis of the task prediction result is performed, and a third evaluation index is established according to a task importance analysis result; Multi-dimensional index screening is performed by using the first evaluation index, the second evaluation index and the third evaluation index, after the mapping and screening of the adaptation evaluation result according to the multi-dimensional index screening result, the primary hybrid layer, the standby hybrid layer and the emergency processing layer are used as the hybrid expansion strategy based on the Kubernetes configuration.

2. The method of claim 1, wherein the microservices architecture of the trusted data space dynamic extension is characterized by, The task prediction is performed according to the task database, and the task prediction result is established, which comprises: A scheduled task in the task database is obtained, and a first prediction result under a trust identifier is constructed according to the scheduled task and a scheduled task subject; The task database is parsed, time point task backtracking is performed according to a parsing result, and a time point task backtracking result is established; Time point task backtracking results meeting a preset probability are extracted, and a second prediction result under the trust identifier is established according to the extraction result; The task prediction result is established according to the first prediction result and the second prediction result.

3. The method of claim 1, wherein the method further comprises: After the mapping of the distributed data storage layer and the hybrid expansion strategy is established, it comprises: An asynchronous message queue is introduced in an access layer, and after a task request is written into the asynchronous message queue, the task request is pulled and processed by a background; Real-time monitoring of task data of the asynchronous message queue is performed, and a real-time monitoring data set is established; According to the real-time monitoring data set, a task queue warning is reported.

4. The method of claim 3, wherein the microservices architecture of the trusted data space dynamically extends, After the mapping of the distributed data storage layer and the hybrid expansion strategy is established, the method further includes: A back pressure response mechanism is configured for the primary hybrid layer. When a primary back pressure signal appears in the primary hybrid layer, the primary back pressure signal is fed back to the asynchronous message queue. The asynchronous message queue adjusts task allocation according to the activation state of the standby hybrid layer.

5. The method of claim 4, wherein, The asynchronous message queue adjusts task allocation according to the activation state of the standby hybrid layer, including: A back pressure response mechanism is configured for the standby hybrid layer. If the activation state of the standby hybrid layer is a busy activation state, a standby back pressure signal is generated. The standby back pressure signal is fed back to the asynchronous message queue to activate the emergency processing layer to perform emergency response management.

6. The method of claim 1, wherein, The synchronous establishment of the distributed data storage layer by using the hot data prediction result further includes: A zero-trust security architecture is configured for the computing resource layer and the distributed data storage layer. Data interaction management of the computing resource layer and the distributed data storage layer is performed through the zero-trust security architecture.

7. The method of claim 1, wherein, After the dynamic expansion of the trusted data space is completed, the method further includes: Running data records are established, and abnormal behavior identification of the data records is performed through a deep learning channel; According to the abnormal behavior identification result, a self-healing mechanism is activated to perform self-optimization management of task execution.

8. A trusted data space dynamic extension system for microservice architecture, characterized in that, Steps for implementing the method for dynamic expansion of the trusted data space of the microservice architecture according to any one of claims 1 to 7, including: A policy establishment module is configured to establish a hybrid expansion strategy for the computing resource layer, and the hybrid expansion strategy includes a Kubernetes, WASM, and FaaS hybrid strategy. A task prediction module is configured to configure a task database, perform task prediction according to the task database, establish a task prediction result, and the task prediction result has a task feature identifier and a trust identifier. An adaptation evaluation module is configured to perform adaptation evaluation of the hybrid expansion strategy in the computing resource layer by using the task prediction result, and preconfigure the hybrid expansion strategy according to the adaptation evaluation result. A data prediction module is configured to perform hot data call prediction based on the hybrid expansion strategy and the task prediction result, and establish a hot data prediction result. A dynamic expansion module is configured to synchronously establish a distributed data storage layer by using the hot data prediction result, and after mapping of the distributed data storage layer and the hybrid expansion strategy is established, the dynamic expansion of the trusted data space is completed.

Images