Service switching method and device based on stream processing framework and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a service switching method and device based on a stream processing framework and a storage medium. Comprising the following steps: when it is determined that service switching needs to be carried out, sending a switching instruction to an operation side; the switching instruction is used for indicating creation of a job management component and a deployment mode when the created job management component deploys a task corresponding to a to-be-switched service, creating a task execution component, and injecting an archived file corresponding to the to-be-switched service into a stream processing framework mirror image; obtaining a port address configured for each node of the operation side for the to-be-switched service, and synchronizing the port address to the operation side; and determining service information of the to-be-switched service, and synchronizing the service information to the configuration mapping of the operation side. According to the embodiment of the invention, the switching process can be shortened, and user expenditure is saved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a service switching method, device and storage medium based on a stream processing framework. Background Art

[0002] The development of science and technology has brought about a rapid growth in the amount of data. The emergence of stream processing frameworks (for example, Flink, Storm, etc.) provides a more suitable solution for processing these massive amounts of data. Stream processing frameworks can be classified into the operation and maintenance side maintained by managers and the operation side used by tenants. For small and medium-sized private cloud scenarios, the components of the stream processing framework can be deployed on the operation and maintenance side to meet business needs. However, for large private cloud scenarios, not only does there exist large amounts of data to be processed, but there may also be requirements for network security isolation between the operation and maintenance side and the operation side, and the network bandwidth of the operation and maintenance side is less than the network bandwidth of the operation side. This results in the components of the stream processing framework deployed on the operation and maintenance side not being able to meet the needs of the operation side business, but it is necessary to switch the stream processing framework on the operation and maintenance side and the corresponding operation side business to the operation side for operation.

[0003] At present, the way to switch the stream processing framework and the corresponding operation-side business from the operation and maintenance side to the operation side is usually to redeploy the components of the stream processing framework on the operation side, receive the original data from the operation side, and write it to the relevant middleware and storage on the operation side after processing. It can be seen that this switching method requires the redeployment of the components of the stream processing framework on the operation side, and the replacement of related hardware on the operation side, and the switching process is relatively complicated. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a service switching method, device and storage medium based on a stream processing framework, which can shorten the switching process and save user expenses.

[0005] In the first aspect, the present application provides a service switching method based on a stream processing framework, which is applied to a security operation center, including: when it is determined that service switching is required, sending a switching instruction to the operation side; the switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the service to be switched, creating a task execution component, and injecting the archive file corresponding to the service to be switched into the stream processing framework image; obtaining the port address configured for each node on the operation side for the service to be switched, and synchronizing the port address to the operation side; determining the service information of the service to be switched, and synchronizing the service information to the configuration mapping on the operation side.

[0006] On the second aspect, the present application provides a service switching method based on a stream processing framework, which is applied to the operation side management center, including: when receiving customized resources configured by a multi-cluster management system, creating a job management component and a task execution component, and injecting the archive file corresponding to the service to be switched into the stream processing framework image; receiving the port address of each node, and configuring the port address of each node; the node is a device called when the operation side runs the service; receiving the service information of the service to be switched, and storing the service information in the configuration mapping; according to the service information, deploying and executing the tasks corresponding to the service to be switched through the job management component in an independent operation mode; the execution of the tasks corresponding to the service to be switched relies on the job management component, the task execution component, the archive file, the node, and the service information.

[0007] On the third aspect, the present application provides a business switching device based on a stream processing framework, which is applied to a security operation center, including: a sending module, which is used to send a switching instruction to the operation side when it is determined that a business switching is required; the switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, the creation of a task execution component, and the injection of the archive file corresponding to the business to be switched into the stream processing framework image; an acquisition module, which is used to obtain the port address configured for each node on the operation side for the business to be switched, and synchronize the port address to the operation side; a determination module, which is used to determine the business information of the business to be switched, and synchronize the business information to the configuration mapping on the operation side.

[0008] In the fourth aspect, the present application provides a business switching device based on a stream processing framework, which is applied to the operation side management center, including: a creation module, which is used to create a job management component and a task execution component when receiving customized resources configured by a multi-cluster management system, and inject the archive file corresponding to the business to be switched into the stream processing framework image; a configuration module, which is used to receive the port address of each node and configure the port address of each node; the node is a device called when the operation side runs the business; a storage module, which is used to receive the business information of the business to be switched, and store the business information in the configuration map; a processing module, which is used to deploy and execute the tasks corresponding to the business to be switched through the job management component in an independent operation manner according to the business information; the execution of the tasks corresponding to the business to be switched relies on the job management component, the task execution component, the archive file, the node, and the business information.

[0009] In a fifth aspect, the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the service switching method based on the stream processing framework as in the first aspect or the second aspect is implemented.

[0010] In a sixth aspect, the present application provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the service switching method based on the stream processing framework as in the first aspect or the second aspect.

[0011] In a seventh aspect, the present application provides a computer program product, comprising: when the computer program product runs on a computer, the computer implements the service switching method based on the stream processing framework as in the first aspect or the second aspect.

[0012] The technical solution provided by the present application has the following advantages compared with the prior art: First, when the security operation center determines that a business switch is required, it sends a switch instruction to the operation side, the port address configured for each node on the operation side for the business to be switched, and the business information of the business to be switched. After that, when the operation side management center receives the customized resources, port address, and business information, it creates a job management component and a task execution component, injects the archive file corresponding to the business to be switched into the stream processing framework image, configures the port address for each node, stores the business information in the configuration map, and deploys and executes the tasks corresponding to the business to be switched through the job management component in an independent operation mode according to the business information. Among them, the node is the device called when the operation side runs the business; the execution of the task corresponding to the business to be switched depends on the job management component, the task execution component, the archive file, the node, and the business information. In this way, when the business of the operation and maintenance side is switched to the operation side, a pure software-level switch can be performed, and it is no longer necessary to redeploy all components of the stream processing framework on the operation side, and it is not necessary to replace the hardware on the operation side, which saves user expenses and reduces the switching process. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] Figure 1 A flow chart of a service switching method based on a stream processing framework provided in an embodiment of the present application;

[0016] Figure 2 One of the application scenario schematic diagrams of the service switching method based on the stream processing framework provided in an embodiment of the present application;

[0017] Figure 3 A second schematic diagram of an application scenario of the service switching method based on the stream processing framework provided in an embodiment of the present application;

[0018] Figure 4 One of the structural schematic diagrams of the service switching device based on the stream processing framework provided in an embodiment of the present application;

[0019] Figure 5 A second structural diagram of a service switching device based on a stream processing framework provided in an embodiment of the present application;

[0020] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0022] The stream processing framework provides a more suitable solution for the processing of massive data. The stream processing framework can be a Flink framework, a Storm framework, etc. The embodiment of the present application takes the Flink framework as an example for explanation. As the mainstream framework of the stream processing framework, the Flink framework depends on its input upstream and written downstream related networks, middleware, and storage architecture during use. The stream processing framework can be classified into the operation and maintenance side maintained by managers and the operation side used by tenants. The operation and maintenance of small and medium-sized private cloud scenarios is dominant. The upstream data network, middleware, and storage architecture that need to be processed run on the operation and maintenance side, so the Flink components themselves also need to be deployed on the operation and maintenance side. Due to the large-scale data volume of its tenant business, the security requirements of network isolation on the operation and maintenance side, and the network bandwidth on the operation and maintenance side being less than the network bandwidth requirements on the operation side, the Flink component architecture on the operation and maintenance side is difficult to meet the requirements. It is necessary to deploy the Flink components on the operation side, receive the original data on the operation side, and write the related middleware and storage on the operation side after processing. As a cloud provider, it is necessary to consider both usage scenarios at the same time. At the same time, as the business scale of users with small and medium-sized private clouds expands, they need to switch the Flink architecture from the operation and maintenance side to the operation side while ensuring the continuity of business and security data.

[0023] At present, the way to switch the stream processing framework on the operation and maintenance side and the corresponding operation-side business to the operation side is usually to redeploy the components of the stream processing framework on the operation side, receive the original data on the operation side, and write it to the relevant middleware and storage on the operation side after processing. It can be seen that this switching method requires the redeployment of the components of the stream processing framework on the operation side, and the replacement of related hardware on the operation side, and the switching process is relatively complicated. In addition, cloud-native Flink relies on the Flink component to communicate with the Kubernetes APIServer based on the Operator mechanism. When the Flink component switches to the operation side that is isolated from the Kubernetes network on the operation and maintenance side network, there are also problems with the communication between the Flink component and the Operator.

[0024] In response to the above problems, an embodiment of the present application provides a service switching method for a stream processing framework, including: when the security operation center determines that service switching is required, the security operation center sends a switching instruction to the operation side; the switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, the creation of a task execution component, and the injection of the archive file corresponding to the business to be switched into the stream processing framework image; obtaining the port address configured for each node on the operation side for the business to be switched, and synchronizing the port address to the operation side; determining the business information of the business to be switched, and synchronizing the business information to the configuration mapping on the operation side. When the operation management center receives the customized resources configured by the multi-cluster management system, it creates a job management component and a task execution component, and injects the archive file corresponding to the business to be switched into the stream processing framework image; receives the port address of each node, and configures the port address for each node; the node is the device called when the operation side runs the business; receives the business information of the business to be switched, and stores the business information in the configuration map; according to the business information, the task corresponding to the business to be switched is deployed and executed by the job management component in an independent operation mode; the task execution corresponding to the business to be switched depends on the job management component, the task execution component, the archive file, the node, and the business information. In this way, on the one hand, when switching the business of the operation and maintenance side to the operation side, a pure software-level switch can be performed, which saves user expenses and reduces the switching process. On the other hand, the port addresses configured for each node on the operation side for the business to be switched can also be configured in advance to ensure the normal communication of each node on the operation side.

[0025] The service switching method based on the stream processing framework provided in the embodiment of the present application can be executed by a service switching device based on the stream processing framework, and the service switching device based on the stream processing framework can be hardware or software. When the service switching device based on the stream processing framework is hardware, it can be various electronic devices with a service switching function based on the stream processing framework, including but not limited to mobile phones, computers, computers, tablet computers, etc. When the service switching device based on the stream processing framework is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0027] Figure 1 A flow chart of a service switching method based on a stream processing framework provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the service switching method based on the stream processing framework may include the following steps.

[0028] S11. When the security operation center determines that a service switching is required, the security operation center sends a switching instruction to the operation side.

[0029] Among them, the Security Operation Center (SOC) is a system that manages business switching, such as Figure 2 As shown in the figure, the security operation center is on the operation and maintenance side in terms of classification dimension, and can interact with the operation management center (Flink Operator) on the operation side through the Kubernetes multi-cluster management system (Karmada).

[0030] First, the security operation center determines whether service switching is required.

[0031] Specifically, the security operation center may determine whether a service switching is required by determining that a service switching is required when at least one of the following switching conditions is met. The switching conditions include: reading a service switching command; the number of tenants on the operation side is greater than a preset threshold; the timer reaches the timing duration. The preset threshold is preset in advance, such as a default value, or a value set by relevant personnel according to actual conditions.

[0032] In some embodiments, the way for a user to issue a service switching command may be to demarcate a storage variable in the basic unit corresponding to the availability zone (AZ) on the operation and maintenance side, and issue a service switching command by changing the content in the storage variable. For example, a Kubernetes element configuration map (Configmap) named soc-default-sysvpc is declared in the availability zone control Cell on the operation and maintenance side, and a service switching command is issued by writing switching data in the Configmap. Among them, the switching data includes at least information such as the availability zone for pulling up the stream processing framework and the network subnet required to allocate the tenant virtual private cloud (VPC) to the stream processing framework component. Among them, the stream processing framework can be a Flink framework.

[0033] After that, the security operation center sends a switching instruction to the operation side. The switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, the creation of a task execution component, and the injection of the archive file corresponding to the business to be switched into the stream processing framework image.

[0034] Specifically, Figure 2 As shown, the security operation center can interact with the operation side management center of the operation side through the multi-cluster management system. The security operation center can send a switching instruction to the operation side by configuring a custom resource (Custom Resource) for the operation side through the multi-cluster management system to instruct the operation side to create a job management component, the created job management component deploys the task corresponding to the business to be switched, creates a task execution component, and injects the archive file corresponding to the business to be switched into the stream processing framework image.

[0035] For example, Figure 3 As shown, taking the Flink framework as an example, when the security operation center (SOC) determines that a business switch is required, the process of sending a switching instruction to the operation side can be: the user writes the switching data in the Configmap to issue a business switching command, the SOC reads the switching data in the Configmap, determines that the user has issued a business switching command, and then sends a switching instruction (including the switching data) to the multi-cluster management system. When the multi-cluster management system receives the switching instruction, it configures customized resources for the operation side management center according to the switching data, and sends it to the operation side management center to instruct the operation side management center to create a job management component, the created job management component deploys the corresponding task of the business to be switched, creates a task execution component, and injects the archive file corresponding to the business to be switched into the stream processing framework image.

[0036] S12. When receiving the customized resources configured by the multi-cluster management system, the operation-side management center creates a job management component and a task execution component, and injects the archive file corresponding to the business to be switched into the stream processing framework image.

[0037] Custom resources are resources allocated by multi-cluster management for creating job management components and task execution components based on switching data; job management components are components responsible for business management, scheduling, and status monitoring; task execution components are components responsible for executing specific business tasks; and archive files are files required for the business to be switched. When the stream processing framework is the Flink framework, such as Figure 3 As shown in the figure, the job management component is the Flink job management component, the task execution component is the Flink task execution component, the archive file is the Jar package, and the stream processing framework image is the Flink image.

[0038] First, when the operation management center receives the customized resources configured by the multi-cluster management system, it creates the job management component and the task execution component according to the customized resources configured by the multi-cluster management system. Secondly, the operation management center injects the archive file corresponding to the business to be switched into the stream processing framework image.

[0039] Specifically, take the Flink framework as an example. Figure 3As shown in the figure, in the Flink framework on the operation and maintenance side, the CR of the stream processing task (FlinkSessionJob) sent by the SOC to the operation and maintenance side management center (Flink Operator) contains the Jar package required by the business corresponding task. This Jar package is downloaded by the operation and maintenance side management center from the Kubernetes Service provided by the SOC through the network, but the SOC component is not included on the operation side, so the CR of the stream processing task (FlinkSessionJob) cannot be sent to achieve this purpose. Therefore, it is necessary to inject the archive file (i.e., Jar package) required for the business to be switched into the pre-created Flink framework image through initContainer; the CR for creating the Flink job management component no longer uses the FlinkDeployment in Session mode, but the FlinkDeployment in Application mode. Among them, initContainer is a container type in Kubernetes, which is executed before the main application container is started and is used to initialize the task; Session mode is a job running mode in the Flink framework, which allows multiple jobs to share the same Flink job management component and cluster resources; Application mode is another job running mode in the Flink framework. In this mode, each job starts an independent Flink job management component and cluster resources, which is suitable for long-running applications.

[0040] In some embodiments, before injecting the archive file corresponding to the service to be switched into the stream processing framework image, the service switching method based on the stream processing framework further includes: establishing a stream processing framework image in the operation side management center.

[0041] S13. The security operation center obtains the port address configured for each node on the operation side for the service to be switched, and synchronizes the port address to the operation side.

[0042] Take the Flink framework as an example, Figure 3As shown, in the Flink framework on the operation and maintenance side, the Flink job management component and the Flink task execution component can directly interact with each other through the internal Domain Name System (DNS) resolution service, without the need to pre-set the corresponding Internet Protocol (IP) address and other network communication information. However, the network plane where the Flink framework on the operation side is located is the VPC side. When the components in the Flink framework need to communicate with other upstream and downstream components of the tenant, such as Kafka and Clickhouse, they must specify mutually accessible IP addresses and be integrated into the CR supported by the Flink framework. Therefore, the embodiment of the present application obtains the port addresses configured for each node on the operation side for the business to be switched in advance through the security operation center, and synchronizes the port addresses to the operation side.

[0043] Specifically, the port address may be an address pre-configured for each node on the operation side for the service to be switched. The port address may be synchronized to the operation side by carrying the configured port address in the switching instruction sent to the multi-cluster management system, and the multi-cluster management system may carry the configured port address when configuring the customized resource to the operation side management center. For example, a custom annotation ccos.io / port_pool may be added to the customized resource to solve the problem of each node (such as pod) accessing the VPC network.

[0044] S14. The operation-side management center receives the port address of each node and configures the port address of each node.

[0045] Among them, the node is the equipment called by the operation side when running the business.

[0046] Specifically, when receiving the port address of each node, the operation-side management center configures the port address of each node.

[0047] S15. The security operation center determines the service information of the service to be switched, and synchronizes the service information to the configuration mapping on the operation side.

[0048] Take the Flink framework as an example, Figure 3As shown in the figure, when the operation and maintenance management center in the Flink framework controls the Flink task execution component to execute specific tasks corresponding to the business, part of the required input needs to be obtained from the Mysql on the operation and maintenance side, but it is complicated and unnecessary to provide the Mysql component on the operation side. Therefore, the security operation center on the operation and maintenance side synchronizes the business information in the Mysql required to be input for the business to be switched to the configuration map (Configmap) on the operation side to avoid the problem of the inability to execute tasks due to lack of business information when executing specific tasks of the business to be processed on the operation side.

[0049] Specifically, the security operation center can synchronize business information to the configuration mapping on the operation side through the multi-cluster management system.

[0050] S16. The operation-side management center receives the service information of the service to be switched, and stores the service information in the configuration map.

[0051] Specifically, when receiving the service information of the service to be switched, the operation-side management center stores the service information in a configuration map (Configmap).

[0052] S17. The operation management center deploys and executes the tasks corresponding to the to-be-switched business in an independent operation manner through the job management component according to the business information.

[0053] Among them, the task execution corresponding to the to-be-switched business depends on the job management component, the task execution component, the archive file, the node, and the business information.

[0054] Specifically, taking the Flink framework as an example of a stream processing framework, since the archive files required for the business to be switched in step S12 are injected into the pre-created Flink framework image through initContainer. Therefore, the CR that creates the Flink job management component no longer uses the FlinkDeployment in Session mode, but uses the FlinkDeployment in Application mode. This is because the use of initContainer enables the Flink job to perform some custom initialization operations before startup. Each job in Application mode manages its life cycle independently, and can perform specific initialization operations at startup and isolate the impact of different jobs. The Session mode is a cluster-level sharing mode, which is not suitable for custom initialization each time a job starts, because it will affect the operation of the entire cluster.

[0055] In some embodiments, when the operation side management center executes the task corresponding to the business to be switched, the cached data is stored in the local storage device of the operation side. Specifically, when the Flink framework on the operation and maintenance side supports the execution of the tasks corresponding to the business, it needs to use object storage to store the data during the business runtime for snapshots and aggregated data cache. After switching to the operation side, object storage is no longer available, so the storage location of the cached data is changed to the local (LocalPV) storage device on the operation side.

[0056] In the above scheme, first, when the security operation center determines that a business switch is required, it sends a switch instruction to the operation side, the port address configured for each node on the operation side for the business to be switched, and the business information of the business to be switched. After that, when the operation side management center receives the customized resources, port address, and business information, it creates a job management component and a task execution component, injects the archive file corresponding to the business to be switched into the stream processing framework image, configures the port address for each node, stores the business information in the configuration map, and deploys and executes the tasks corresponding to the business to be switched through the job management component in an independent operation mode according to the business information. Among them, the node is the device called when the operation side runs the business; the execution of the task corresponding to the business to be switched depends on the job management component, the task execution component, the archive file, the node, and the business information. In this way, when the business of the operation and maintenance side is switched to the operation side, a pure software-level switch can be performed, and it is no longer necessary to redeploy all components of the stream processing framework on the operation side, and it is not necessary to replace the hardware on the operation side, which saves user expenses and shortens the switching process.

[0057] The embodiment of the present application can divide the functional modules of the service switching device based on the stream processing framework according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0058] like Figure 4 As shown, it is a structural diagram of a service switching device based on a stream processing framework provided in an embodiment of the present application, which is applied to a security operation center. The service switching device based on a stream processing framework includes a sending module 51, an acquisition module 52, and a determination module 53.

[0059] The sending module 51 is used to send a switching instruction to the operation side when it is determined that a business switch is required; the switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, the creation of a task execution component, and the injection of the archive file corresponding to the business to be switched into the stream processing framework image; the acquisition module 52 is used to obtain the port address configured for each node on the operation side for the business to be switched, and synchronize the port address to the operation side; the determination module 53 is used to determine the business information of the business to be switched, and synchronize the business information to the configuration mapping on the operation side.

[0060] In some embodiments, when at least one of the following switching conditions is met, it is determined that a service switching is required; the switching conditions include: reading a service switching command; the number of tenants on the operation side is greater than a preset threshold; the timer reaches the timing duration.

[0061] In some embodiments, the sending module 51 is also used to interact with the operation side through the multi-cluster management system; the sending module 51 is specifically used to configure customized resources for the operation side through the multi-cluster management system to instruct the operation side to create a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, create a task execution component, and inject the archive file corresponding to the business to be switched into the stream processing framework image.

[0062] The service switching device based on the stream processing framework provided in this embodiment can execute the service switching method based on the stream processing framework provided in the above method embodiment. Its implementation principle and technical effect are similar to the above method and will not be repeated here.

[0063] like Figure 5 As shown, a structural diagram of a business switching device based on a stream processing framework provided in an embodiment of the present application is applied to an operation side management center. The business switching device based on a stream processing framework includes a creation module 61, a configuration module 62, a storage module 63, and a processing module 64.

[0064] A creation module 61 is used to create a job management component and a task execution component when receiving customized resources configured by a multi-cluster management system, and inject the archive file corresponding to the business to be switched into the stream processing framework image; a configuration module 62 is used to receive the port address of each node and configure the port address of each node; the node is the device called when the operation side runs the business; the storage module 63 is used to receive the business information of the business to be switched, and store the business information in the configuration map; the processing module 64 is used to deploy and execute the tasks corresponding to the business to be switched through the job management component in an independent operation mode according to the business information; the execution of the tasks corresponding to the business to be switched depends on the job management component, the task execution component, the archive file, the node, and the business information.

[0065] In some embodiments, the processing module 64 is further configured to store the cached data in a local storage device on the operation side when executing a task corresponding to the service to be switched.

[0066] The service switching device based on the stream processing framework provided in this embodiment can execute the service switching method based on the stream processing framework provided in the above method embodiment. Its implementation principle and technical effect are similar to the above method and will not be repeated here.

[0067] Figure 6 An electronic device according to an exemplary embodiment may include a processor 802, and the processor 802 is used to execute application code, so as to implement the service switching method based on the stream processing framework in the present application.

[0068] The processor 802 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0069] like Figure 6 As shown, the electronic device may further include a memory 803. The memory 803 is used to store application program codes for executing the solution of the present application, and the execution is controlled by the processor 802.

[0070] The memory 803 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 803 may exist independently and be connected to the processor 802 via the bus 804. The memory 803 may also be integrated with the processor 802.

[0071] like Figure 6As shown, the electronic device may further include a communication interface 801, wherein the communication interface 801, the processor 802, and the memory 803 may be coupled to each other, for example, via a bus 804. The communication interface 801 is used to exchange information with other devices, for example, to support information exchange between the electronic device and other devices.

[0072] It should be pointed out that Figure 6 The device structure shown in the figure does not constitute a limitation on the electronic device, except Figure 6 In addition to the components shown, the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. The electronic device provided in this embodiment can execute the service switching method based on the stream processing framework provided in the above method embodiment, and its implementation principle and technical effect are similar to the above method, which will not be repeated here.

[0073] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the service switching method based on the stream processing framework in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0074] The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0075] An embodiment of the present application provides a computer program product, which stores a computer program. When the computer program is executed by a processor, each process of the service switching method based on the stream processing framework in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.

[0077] In the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0078] In this application, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0079] In this application, computer-readable media includes permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data and carrier waves.

[0080] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0081] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A service switching method based on a stream processing framework, applied to a security operation center, characterized in that: include: When it is determined that service switching is required, a switching instruction is sent to the operation side; The switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, the creation of a task execution component, and the injection of the archive file corresponding to the business to be switched into the stream processing framework image; Acquire the port address configured for each node on the operation side for the service to be switched, and synchronize the port address to the operation side; Determine the service information of the service to be switched, and synchronize the service information to the configuration map of the operation side.

2. The service switching method according to claim 1, characterized in that: The method further comprises: When at least one of the following switching conditions is met, determining that service switching is required; The switching conditions include: reading a service switching command; the number of tenants on the operation side is greater than a preset threshold; and the timer reaches the timing duration.

3. The service switching method according to claim 1, characterized in that: The method further comprises: Interact with the operations side through a multi-cluster management system; The sending of the switching instruction to the operation side includes: Customized resources are configured for the operation side through the multi-cluster management system to instruct the operation side to create a job management component, the created job management component deploys the task corresponding to the business to be switched, creates a task execution component, and injects the archive file corresponding to the business to be switched into the stream processing framework image.

4. A service switching method based on a stream processing framework, applied to an operation side management center, characterized in that: include: When receiving the customized resources configured by the multi-cluster management system, create the job management component and the task execution component, and inject the archive files corresponding to the business to be switched into the stream processing framework image; Receive the port address of each node and configure the port address of each node; the node is the device called by the operation side when running the business; Receiving service information of the service to be switched, and storing the service information in a configuration map; According to the business information, the task corresponding to the business to be switched is deployed and executed by the job management component in an independent operation mode; the execution of the task corresponding to the business to be switched depends on the job management component, the task execution component, the archive file, the node, and the business information.

5. The service switching method according to claim 4, characterized in that: The method further comprises: When executing the task corresponding to the service to be switched, the cached data is stored in a local storage device on the operation side.

6. A service switching device based on a stream processing framework, applied to a security operation center, characterized in that: include: A sending module, used to send a switching instruction to the operation side when it is determined that service switching is required; The switching instruction is used to instruct the creation of a job management component, the deployment method of the created job management component when deploying the task corresponding to the business to be switched, the creation of a task execution component, and the injection of the archive file corresponding to the business to be switched into the stream processing framework image; An acquisition module, used for acquiring the port address configured for each node on the operation side for the service to be switched, and synchronizing the port address to the operation side; The determination module is used to determine the service information of the service to be switched and synchronize the service information to the configuration map of the operation side.

7. A service switching device based on a stream processing framework, applied to an operation side management center, characterized in that: include: A creation module is used to create a job management component and a task execution component when receiving customized resources configured by a multi-cluster management system, and inject archive files corresponding to the business to be switched into the stream processing framework image; A configuration module, used to receive the port address of each node and configure the port address of each node; the node is a device called by the operation side when running the business; A storage module, used for receiving service information of the service to be switched, and storing the service information in a configuration map; A processing module is used to deploy and execute the tasks corresponding to the business to be switched in an independent running manner through the job management component according to the business information; the execution of the tasks corresponding to the business to be switched depends on the job management component, the task execution component, the archive file, the node, and the business information.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the service switching method based on a stream processing framework as described in any one of claims 1 to 3, or claims 4 to 5.

9. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the service switching method based on a stream processing framework as described in any one of claims 1 to 3, or claims 4 to 5 is implemented.

10. A computer program product, characterized in that When the computer program product runs on a computer, the computer implements the service switching method based on a stream processing framework as claimed in any one of claims 1 to 3 or claims 4 to 5.

Images