Data processing method and device, equipment and storage medium

Abstract

The invention discloses a data processing method and device, equipment and a storage medium, and relates to the technical field of data processing. The method comprises the steps of obtaining service orchestration data of M micro-services, wherein the service orchestration data comprises calling logic data, view interface data and service deployment environment data of the micro-services; based on the service orchestration data, constructing a micro-service mirror image data packet comprising the service orchestration data and data required for running M micro-services; and deploying the micro-service mirror image data packets in N containers of a container system, wherein a container isolation mechanism exists among the containers in the container system. Therefore, the micro-service mirror image data formed by the service arrangement data after micro-service arrangement is independently deployed in the container, resources among the arranged micro-service chains are ensured not to interfere with each other, stable operation of the micro-service process instances corresponding to a plurality of micro-services of each micro-service chain in the plurality of micro-service chains is effectively ensured, and the service quality of the micro-service chains is improved. The overall performance of the service system is improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a data processing method, device, equipment and storage medium. Background Art

[0002] Microservices are the process of splitting a large application into many small, independent service units, each of which performs a specific function or service task. There is a certain logical relationship between microservices, which can be organized and orchestrated to meet certain specific operation and maintenance requirements, such as executing in a preset order or orchestrating the latter to depend on the execution results of the former. This can be applied in emergency switching scenarios of multi-center application systems.

[0003] In related technologies, microservice orchestration usually publishes multiple orchestrated microservice process instances to run in the same process. However, as the complexity of services and the number of applications increase, multiple microservice process instances will simultaneously occupy public resources such as processors, memory, and networks in the service system, which will not only cause the performance of the service system to deteriorate, but also reduce the stability of the operation of microservice process instances. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium, which can solve the problem in the related technology that multiple microservice process instances simultaneously occupy service system resources, reduce the performance of the service system, and affect the stability of the operation of the microservice process instances.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, which may include:

[0006] Obtain service orchestration data of M microservices, where the service orchestration data includes call logic data, view interface data, and service deployment environment data of the microservices, where M is a positive integer;

[0007] Based on the service orchestration data, a microservice image data package is constructed. The microservice image data includes the service orchestration data and the data required to run M microservices.

[0008] The microservice image data package is deployed in N containers of the container system. There is a container isolation mechanism between the containers in the container system, and N is a positive integer.

[0009] In a second aspect, an embodiment of the present application provides a data processing device, which may include:

[0010] An acquisition module is used to acquire the service orchestration data of M microservices, where the service orchestration data includes the calling logic data, view interface data and service deployment environment data of the microservices, and M is a positive integer;

[0011] A construction module is used to construct a microservice image data package based on the service orchestration data. The microservice image data includes the service orchestration data and the data required to run M microservices.

[0012] The deployment module is used to deploy the microservice image data package in N containers of the container system. There is a container isolation mechanism between the containers in the container system, and N is a positive integer.

[0013] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory storing computer program instructions;

[0014] When the processor executes the computer program instructions, the data processing method shown in the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the data processing method shown in the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the data processing method shown in the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the data processing method shown in the first aspect.

[0018] The data processing method, apparatus, device and storage medium of the embodiments of the present application obtain the service orchestration data of M microservices, and the service orchestration data includes the calling logic data, view interface data and service deployment environment data of the microservices. In this way, based on the static preloading drive technology of memory, the service process of the operation and maintenance microservices is solidified into a memory process object, and the service process steps are abstracted into microservice data points related to the service orchestration data, so as to improve the processing performance of the microservices from seconds to milliseconds, and greatly improve the drive response time when the microservices are executed after orchestration; then, based on the service orchestration data, a microservice image data packet including the service orchestration data and the data required to run the M microservices is constructed, and the microservice image data packet is deployed in the N containers of the container system, and there is a container isolation mechanism between the containers in the container system. In this way, the service orchestration data of the operation and maintenance microservices after microservice orchestration is independently formed into microservice image data and deployed separately in the container to realize independent deployment of the image, so that the drive scheduling and operation of a series of service process steps can be independently completed based on the microservice image data in the independent container, thereby avoiding the mutual influence such as resource crowding that may occur in the unified and centralized drive scheduling of services. The container isolation mechanism is used between the multiple microservice chains after orchestration to ensure that the resources between the orchestrated microservice chains do not interfere with each other, effectively ensuring the stable execution of multiple microservices in each microservice chain in the multiple microservice chains orchestrated under sudden large traffic conditions, while improving the performance of the service system, ensuring the stability, execution timeliness and execution success rate of the microservice process instances of multiple microservices in each microservice chain. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of the structure of a data processing system provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0022] Figure 3 A flowchart of a data processing method performed by a service deployment module in a data processing system provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of comparing the execution results of a data processing method provided in an embodiment of the present application;

[0024] Figure 5A flowchart of a data processing method executed by a service calling module in a data processing system provided in an embodiment of the present application;

[0025] Figure 6 A flowchart of a data processing method performed by a service management module in a data processing system provided in an embodiment of the present application;

[0026] Figure 7 is a structural schematic diagram of a data processing device provided by an embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0030] The acquisition, storage, use, and processing of data (including but not limited to the features and information in the text) in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0031] Microservices is a software design method that splits a large application into many small, independent service units, each of which focuses on performing a specific function or service task. In the data center operation and maintenance scenario, the operation and maintenance microservice may initiate a system check script, execute an isolation instruction, or restart an application service process. These microservices generally have some logical relationship with each other and need to be organized and orchestrated to meet certain specific operation and maintenance requirements, such as: executing in a certain order or the latter depends on the execution result of the former for orchestration, which can be applied in the emergency switching scenario of multi-center application systems.

[0032] The automation engine publishes these operation and maintenance microservices through the Hypertext Transfer Protocol (HTTP) Resource Representational State Transfer (REST) ​​Application Programming Interface (API), and then orchestrates and assembles these microservices through the microservice orchestration engine to form specific operation microservices, such as: automated emergency switching service flow of multi-center application systems.

[0033] Microservice orchestration methods have attracted widespread attention for their flexibility and scalability. However, when applied to data center operation and maintenance scenarios that have very high requirements for stability, execution timeliness and execution success rate, these scenarios usually involve core key high-availability services such as multi-center switching of financial application systems. These scenarios are complex and require strong stability and fast execution efficiency. Any minor problems in the orchestration may cause the failure of multi-center switching of services or exceed the timeliness requirements for emergency processing.

[0034] In related technologies, microservice orchestration usually publishes multiple orchestrated microservice process instances to run in the same process. However, as the complexity of services and the number of applications increase, multiple microservice process instances will simultaneously occupy public resources such as processors, memory, and networks in the service system, which will not only lead to a decline in service system performance, but also reduce the stability of microservice process instance operation. Therefore, there is an urgent need for a microservice orchestration method that is suitable for complex scenarios and can fully improve the performance and stability of the service system.

[0035] Here, it should be noted that the microservice orchestration in the embodiment of the present application refers to the configuration, management and coordination of a group of microservices to implement a specific service process or function. In the microservice architecture, each microservice is responsible for a specific service function and can be independently deployed, upgraded and expanded. These microservices usually need to work together to complete more complex tasks. The unified execution engine is the core component of the microservice orchestration, which is responsible for executing and calling existing microservices in accordance with the arranged order and logic, so that the flexibility, scalability and reliability of the system can be improved.

[0036] Based on this, in order to solve the problems in the related technology, the following will be combined with the attached Figures 2 to 5 , the methods, devices, computer equipment and storage media of the embodiments of the present application are described in detail. It should be noted that these embodiments are not intended to limit the scope of the disclosure of the present application.

[0037] first, Figure 1 A schematic diagram of the structure of a data processing system provided in an embodiment of the present application.

[0038] like Figure 1 As shown, the microservice orchestration platform is a complex microservice architecture for service systems, providing a platform for unified, automated service orchestration and management across multi-cloud environments and multi-type microservices. The microservice orchestration platform uses different orchestration technology strategies in actual applications of different service systems. Here, in the embodiment of the present application, a graphical microservice assembly is used, which is driven by static preloading solidified in memory.

[0039] Based on this, the data processing system in the embodiment of the present application may include a service orchestration module, a service deployment module, a service calling module, a service management module, a service execution module and a service conversion module, as shown below.

[0040] The service orchestration module is used for automated orchestration and expansion. Based on user needs, the service orchestration module builds a service framework, designs the front-end page, and generates a deployable version through service orchestration.

[0041] The service deployment module is used for service deployment. The service deployment module can be used to compile and package the created services into container images, deploy and run them, and register them to the registration center for convenient use by users.

[0042] The service call module is used to call services and ensure efficient communication between services. To ensure high availability, the service call module distributes user service requests through the gateway and uses the load balancing mechanism to select from the registered service list to provide users with stable services.

[0043] The service management module is used to monitor the service status and handle exceptions and new services. The service management module involves the management and expansion of services, and health checks ensure the stability of service status. Once a service alarm is issued or expansion is required in advance, enhanced performance will be deployed quickly to meet the needs of high service availability.

[0044] The service execution module is used to drive the execution of microservices. Among them, the orchestrated and deployed services, process instances and process nodes (including instances and parameter objects) are pre-created in memory in advance.

[0045] The service conversion module is used to provide a standard interface to support the conversion of other dynamically loaded driver services into statically preloaded driver services.

[0046] Based on this, the modules in the data processing system involved in the embodiments of the present application work together to ensure the stability, efficiency and scalability of the microservice architecture in the data processing system, and to ensure flexible reference, efficient calling and comprehensive management through automated orchestration. Among them, the service conversion module provides an open interface to the outside, and the external system can submit its own defined process diagrammatic standard process through the open interface, which is converted into an internal service process by the service conversion module. The external automated process system takes the process structure definition and process parameters as input content through the open interface, and the conversion module converts the external system input content into the corresponding process definition, including serial, parallel, branch judgment and other process structures.

[0047] Therefore, the embodiment of the present application can be based on memory pre-allocation and object pooling management, and the orchestrated services can be mirrored and independently deployed. The core data memory objects can be pre-allocated, the execution service can be preheated, and the cold start service capability can be improved. The core data memory objects can be stored in a memory pooled manner to avoid frequent creation and release of memory to generate performance consumption, and to avoid memory fragmentation. The core execution threads are pre-allocated to improve thread reuse and reduce system consumption. In this way, the overall service responsiveness and high concurrency capabilities are improved. At the same time, middleware such as Upsql, Upredis and Upkafka are used for communication and data exchange between microservices. The embodiment of the present application aims to provide a stable, efficient and flexible microservice orchestration capability for the high-availability scenario of the financial transaction system center switching during data center operation and maintenance, so as to meet the increasingly complex and time-sensitive operation and maintenance service needs.

[0048] Combine the following Figure 2 right Figure 1 The data processing method executed by each module in the data processing system shown is described in detail.

[0049] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application.

[0050] like Figure 2 As shown, the data processing method can be applied to a data processing system, and the data processing method can specifically include the following steps:

[0051] Step 210, obtain service orchestration data of M microservices, the service orchestration data includes call logic data, view interface data and service deployment environment data of the microservices, and M is a positive integer; Step 220, build a microservice image data package based on the service orchestration data, the microservice image data package includes service orchestration data and data required to run M microservices; Step 230, deploy the microservice image data package in N containers of the container system, and there is a container isolation mechanism between the containers in the container system, and N is a positive integer.

[0052] In this way, the service orchestration data of M microservices is obtained. The service orchestration data includes the calling logic data, view interface data and service deployment environment data of the microservices. In this way, based on the static preloading drive technology of memory, the service process of the operation and maintenance microservices is solidified into a memory process object, and the service process steps are abstracted into microservice data points related to the service orchestration data, which improves the processing performance of the microservices from seconds to milliseconds, greatly improving the drive response time when the microservices are orchestrated and executed; then, based on the service orchestration data, a microservice image data package including the service orchestration data and the data required to run the M microservices is constructed, and the microservice image data package is deployed in the N containers of the container system. The containers in the container system have a container isolation mechanism. In this way, the microservice orchestration The service orchestration data of the subsequent operation and maintenance microservices is independently formed into microservice image data and deployed in the container separately to realize independent deployment of the image, so that the drive scheduling and operation of a series of service process steps can be independently completed based on the microservice image data in the independent container, thereby avoiding the mutual influence such as resource crowding that may occur in the unified and centralized drive scheduling of services. The container isolation mechanism is used between the multiple microservice chains after orchestration to ensure that the resources between the orchestrated microservice chains do not interfere with each other, and effectively guarantee the stable execution of multiple microservices in each microservice chain in the multiple microservice chains orchestrated under sudden large traffic conditions, while improving the performance of the service system, ensuring the stability, execution timeliness and execution success rate of the microservice process instances of multiple microservices in each microservice chain.

[0053] The above steps are described in detail below, as shown below.

[0054] First, regarding step 210 , in some embodiments of the present application, before step 210 , the data processing method may further include steps 2401 to 2403 .

[0055] Step 2401: Receive a first input from a user to a microservice configuration interface, where the microservice configuration interface includes P preset microservice identifiers, and the first input includes an input from a user selecting M microservice identifiers from the P preset microservice identifiers, where P is an integer greater than 1.

[0056] Step 2402: In response to the first input, according to each microservice identifier in the M microservice identifiers, determine microservice data of the microservice corresponding to each microservice identifier, where the microservice data includes data required for the operation of the microservice.

[0057] Step 2403: Generate service orchestration data of M microservices based on the microservice data of the microservice corresponding to each microservice identifier.

[0058] Furthermore, step 2403 may specifically include steps 24031 to 24034.

[0059] Step 24031, based on the microservice data of the microservice corresponding to each microservice identifier, determine whether there is a microservice that has a dependency relationship with the microservice.

[0060] Step 24032: When it is determined that there is a microservice that has a dependency relationship with the microservice, the microservice is associated with the microservice that has a dependency relationship with the microservice to obtain microservice chain information.

[0061] Step 24033, according to the connection order of the microservices in the microservice chain information, the microservice data of the microservice corresponding to each microservice identifier is associated to obtain the microservice data chain information.

[0062] Step 24034: Generate service orchestration data of M microservices based on the microservice chain information and the microservice data chain information.

[0063] Exemplarily, the service orchestration module can build a service framework through service orchestration based on user needs, design the front-end page, and generate a deployable version. Among them, first, a new service interface is added, and the API, input parameters and output parameters of the service interface, as well as the process of the service interface are defined. Create a front-end page, and create the front-end page required by the user according to the existing page elements (such as tables, buttons, input boxes, etc.). Next, data binding determines whether to introduce services. If necessary, the created services are introduced. The process nodes of the service can call existing services or call defined scripts to complete a series of logical operations. The tables, buttons, input boxes, etc. of the front-end page can be bound to the input and output of existing services to realize functions such as interaction with users and data display. Then, the service version is generated, and after the service is created, a new version can be added. The new version will associate the created service information, the configuration file and configuration environment of the service, and the front-end page into a version, which can be referenced, or provide APIs and pages to meet user needs during subsequent release and deployment.

[0064] As a result, users can use the visual interface and drag-and-drop operations to instantly transform operation and maintenance microservices into solidified executable operation microservices in static preloading mode and encapsulate them in the form of container images.

[0065] Next, step 220 is involved. In some embodiments of the present application, for example Figure 3 As shown, the service deployment module can be used to compile and package the created services into container images, deploy and run them, and register them to the registration center for convenient use by users. First, you can create a service instance and select an assembled service version. Each service version contains a set of orchestrated services, a series of front-end pages, and a service deployment environment and configuration files. Next, publish the service version and compile the version-related data of the service (including front-end pages, version data, deployment environment, etc.). Package it into a basic image package and then publish it as a version. Then, deploy the instance and publish the published version to the image repository for management to complete the deployment of the service version. When the version instance is started, it will automatically register with the registration center.

[0066] Therefore, the embodiment of the present application achieves the ultimate microservice driver scheduling efficiency based on the memory static preloading driver technology. Figure 4 As shown, the embodiment of the present application breaks the dynamic loading scheduling during the execution of traditional microservices after orchestration, and adopts static preloading scheduling technology, which reduces the time overhead of object initialization (database access and memory object instantiation) during the microservice call process, and reduces the drive scheduling time of a single orchestrated microservice from 1.5 seconds to 0.5 seconds, improving the performance by more than 3 times, which can speed up the center switching automation of the financial transaction system by 7 seconds (usually the system's center switching automation flow orchestration has an average of 7 microservice action steps).

[0067] Then, step 230 is involved, and the independent resource isolation deployment of the operation microservices of the specific scenario after orchestration can be carried out, so as to avoid the influence of mutual crowding out of resources between the operation microservices, and ensure the high stability, high reliability and high availability of the emergency switching service when low-frequency and high-availability scenarios such as center switching are triggered. The innovation in the embodiment of the present application breaks the traditional unified scheduling of microservices after orchestration and execution, and each orchestrated operation microservice is independently deployed in one (or more) containers, and does not rely on external resources during the operation-driven scheduling process. The container isolation mechanism effectively avoids resource interference between services, and realizes the independent operation of low-frequency and high-availability services.

[0068] In summary, in the embodiment of the present application, by integrating microservice orchestration with static preloading driver technology and introducing container technology, flexible configuration, rapid deployment, high availability, high stability and efficient execution of the center switching service of the data center financial transaction system are achieved. This solution provides strong support for multi-center switching and high availability scenarios in data centers, ensuring the efficiency, stability and scalability of microservice orchestration.

[0069] In addition, after step 230, the data processing method may further include a process of how to use the microservice image data in the container. Based on this, the data processing method may further include steps 2501 to 2503.

[0070] Step 2501, when the service applications corresponding to the M microservices are started, the microservice image data packets in the target containers of the N containers are loaded by static loading to obtain the service flow driving parameters.

[0071] Step 2502, when receiving a user service request from a user end forwarded by the gateway, M microservices are scheduled according to the service flow driving parameters to generate a user service result corresponding to the user service request; wherein the user end is a user end corresponding to the service application.

[0072] Step 2503: Send a forwarding instruction to the gateway, the forwarding instruction carries the user service result, and the forwarding instruction is used to instruct the gateway to forward the user service result to the user terminal.

[0073] For example, Figure 5As shown in the figure, to ensure high availability, the service call module distributes user service requests through the gateway, selects from the registered service list using the load balancing mechanism, and provides stable services for users. First, the user service requests the service interface, the gateway captures the user service request, and prepares to forward it to the registered service. Then, the gateway obtains the registered service list from the registration center, filters the latest version of the service list, and selects a suitable service instance for calling according to the preset load balancing strategy (such as polling, random, etc.). Then, the service instance completes the execution and returns the result, and the gateway collects the execution result and returns it to the caller who initiated the request.

[0074] Still available for reference Figure 4 As shown in the content, each service in the embodiment of the present application is independently deployed in the container instance. By pre-statically loading the process driving parameters, the scheduling driving efficiency is significantly improved. The start and stop nodes are statistically analyzed through 42 different process orchestration samples. The test results are as follows, which improves the efficiency by an order of magnitude compared with the traditional mode.

[0075] In some embodiments of the present application, the above step 2502 may specifically include step 25021 and step 25022.

[0076] Step 25021: upon receiving a user service request from a user terminal forwarded by the gateway, a target registration service matching the user service request is screened from the registration service list.

[0077] Specifically, step 25021 may include:

[0078] When receiving a user service request from a user end forwarded by the gateway, a target registration service matching the user service request is screened from the registration service list through a load balancing algorithm.

[0079] Step 25022, through the target service instance corresponding to the target registration service, according to the service flow driving parameters, schedule M microservices to generate a user service result corresponding to the user service request.

[0080] In other embodiments of the present application, the steps involved in the above-mentioned step 2502 or step 25022 of scheduling M microservices according to the service flow driving parameters and generating user service results corresponding to the user service request may specifically include steps 310 to 340.

[0081] Step 310, when receiving a user service request from a user end forwarded by a gateway, a first service request is sent to an i-th microservice among the M microservices through an external open interface according to a service flow driving parameter, wherein the first service request carries first processing data related to a service processing function of the i-th microservice, and the first service request is used to request the i-th microservice to determine first result data based on the first processing data.

[0082] Step 320, when the first result data is received through the callback interface corresponding to the external open interface, the first result data is deployed in the target container, and it is determined whether there is a parameter for indicating that the i+1th microservice has a dependency relationship in the service flow driving parameters.

[0083] Step 330: When it is determined that the service flow driving parameters contain a parameter of the i+1th microservice indicating that the i-th microservice has a dependency, a second service request is sent to the i+1th microservice through an external open interface, where the second service request carries the first result data and second processing data related to the service processing function of the i+1th microservice, and the second service request is used to request the i+1th microservice to determine the second result data based on the first result number and the second processing data.

[0084] Step 340: When the second result data is received through the callback interface and it is determined that the service flow driving parameter does not contain the (i+1)th microservice indicating that the i-th microservice has a dependency, the second result data is determined as a user service result corresponding to the user service request.

[0085] Exemplarily, the service conversion module provides an open interface to the outside, and the external system can submit its own defined flow chart graphical standard process through the open interface, which is converted into an internal service process by the service conversion module. The external automated process system takes the process structure definition and process parameters as input content through the open interface, and the conversion module converts the external system input content into the corresponding process definition, including serial, parallel, branch judgment and other process structures, http calls and usertask nodes. The graphical process in non-static memory mode can be converted into a process in static memory mode through this device, and the processing performance can be improved through the static process execution engine. When converting the process, the execution completion callback interface can be configured. After the process is executed, the callback interface is used to notify the external system that the process execution is completed and the execution result is returned. Without changing the existing system data and core logic, the service performance of the external system is expanded.

[0086] In addition, after step 230 , the data processing method may further include steps 410 to 450 .

[0087] Step 410: Receive an application version update request sent by a service application corresponding to the M microservices, where the application version update request carries updated version data of the service application.

[0088] Step 420 : When the updated version data includes the service data to be updated corresponding to the microservice image data in the target container of the N containers, based on the service data to be updated, generate update service orchestration data for the M microservices.

[0089] Step 430: construct an updated microservice image data package based on the updated service orchestration data, where the updated microservice image data includes the updated service orchestration data and the updated data required to run the M microservices.

[0090] Step 440: Update the microservice image data in the target container to update the microservice image data package.

[0091] Step 450: Update the microservice image data in the containers other than the target container in the N containers according to the updated microservice image data packet in the target container.

[0092] For example, Figure 6 As shown in the figure, the service management module involves the management and expansion of services, and health detection ensures the stability of service status. Once a service alarm occurs or expansion is required in advance, enhanced performance will be deployed quickly to meet the needs of high service availability. Among them, when an alarm occurs in a deployed service, or an existing service needs to be expanded in advance, a list of services that need to be expanded is determined. Then, for the services that need to be expanded, the corresponding service version is selected for deployment, and the service will be automatically registered to the registration center after successful deployment. Then, when a user service requests to access the service, the gateway will load balance the multiple registered services. To ensure high availability and performance of the service.

[0093] And, it should be noted that the service execution module in the embodiment of the present application can also be pre-created in memory in advance through the orchestration and deployment of services, process instances and process nodes (including instances and parameter objects). After startup, the service interface and service function logic will be constructed according to the service definition. The service interface provides Http services to the outside world. When an external Http request is received, the execution module loads the process instance corresponding to the process template, starts and executes the process instance to implement the execution service function logic, completes a series of service rules and operations predefined by the user, and can dynamically plan process branches and calculation results according to runtime parameter variables, and returns the results after execution is completed.

[0094] Therefore, in the embodiments of the present application, the static preloading driving technology based on memory is used to solidify the service process of the operation and maintenance microservices into a memory process object, and the service process steps (i.e., microservices: application status check, center isolation, application restart and other instruction tasks) are abstracted into microservice nodes, which improves the processing performance of the microservice nodes from seconds to milliseconds, greatly improving the driving response time when the microservices are orchestrated and executed. In addition, in the embodiments of the present application, the operation and maintenance services after the microservices are orchestrated are independently formed into container images and deployed in separate containers, which can independently complete the driving scheduling and operation of a series of service process steps, thereby avoiding the mutual influence of resource crowding and other problems that may occur in the unified and centralized driving scheduling of the usual services. The container isolation mechanism is used between the orchestrated services to ensure that the resources between the orchestrated services do not interfere with each other, effectively ensuring the stable execution of the orchestrated services under sudden large traffic conditions, and ensuring the stability, execution timeliness and execution success rate of low-frequency and high-availability key services such as center switching.

[0095] The present application also provides a data processing device, specifically in combination with Figure 7 Provide detailed explanation.

[0096] Figure 7 It is a structural diagram of a data processing device provided by an embodiment of the present application.

[0097] In some embodiments of the present application, Figure 7 The data processing device shown can be arranged in the data processing system provided in the embodiment of the present application.

[0098] like Figure 7 As shown, the data processing device 70 may specifically include:

[0099] Acquisition module 701, used to acquire service orchestration data of M microservices, the service orchestration data including call logic data, view interface data and service deployment environment data of the microservices, where M is a positive integer;

[0100] A construction module 702 is used to construct a microservice image data package based on the service orchestration data, where the microservice image data includes the service orchestration data and data required to run M microservices;

[0101] The deployment module 703 is used to deploy the microservice image data package in N containers of the container system. There is a container isolation mechanism between the containers in the container system, and N is a positive integer.

[0102] In this way, the data processing device in the embodiment of the present application obtains the service orchestration data of M microservices, and the service orchestration data includes the calling logic data, view interface data and service deployment environment data of the microservices. In this way, based on the static preloading drive technology of memory, the service process of the operation and maintenance microservice is solidified as a memory process object, and the service process steps are abstracted as microservice data points related to the service orchestration data, which improves the processing performance of the microservice from seconds to milliseconds, and greatly improves the drive response time when the microservice is orchestrated and executed; then, based on the service orchestration data, a microservice image data packet including the service orchestration data and the data required to run the M microservices is constructed, and the microservice image data packet is deployed in the N containers of the container system. There is a container isolation mechanism between the containers in the container system, so that , the service orchestration data of the operation and maintenance microservices after microservice orchestration is independently formed into microservice image data, and deployed separately in the container to realize independent deployment of the image, so that the drive scheduling and operation of a series of service process steps can be independently completed based on the microservice image data in the independent container, thereby avoiding the mutual influence such as resource crowding that may occur in the unified and centralized drive scheduling of services. The container isolation mechanism is used between the multiple microservice chains after orchestration to ensure that the resources between the orchestrated microservice chains do not interfere with each other, and effectively guarantee the stable execution of multiple microservices in each microservice chain in the multiple microservice chains orchestrated under sudden large traffic conditions, while improving the performance of the service system, ensuring the stability, execution timeliness and execution success rate of the microservice process instances of multiple microservices in each microservice chain.

[0103] The data processing device 70 in the embodiment of the present application is described in detail below.

[0104] In some embodiments of the present application, the data processing device 70 in the embodiment of the present application may further include a loading module, which is used to load the microservice image data packet in the target container of the N containers by static loading when the service application corresponding to the M microservices is started, and obtain the service flow driving parameter;

[0105] The data processing device 70 in the embodiment of the present application may further include a processing module, which is used to schedule M microservices according to the service flow driving parameters and generate a user service result corresponding to the user service request when receiving a user service request from a user end forwarded by the gateway; wherein the user end is a user end corresponding to the service application;

[0106] In the embodiment of the present application, the data processing device 70 may further include a sending module for sending a forwarding instruction to the gateway, the forwarding instruction carries the user service result, and the forwarding instruction is used to instruct the gateway to forward the user service result to the user terminal.

[0107] In some embodiments of the present application, the data processing device 70 in the embodiment of the present application may further include a screening module for screening a target registration service matching the user service request from the registration service list when receiving a user service request from a user terminal forwarded by the gateway;

[0108] In the embodiment of the present application, the data processing device 70 may further include a processing module, which is used to schedule M microservices according to the service flow driving parameters through the target service instance corresponding to the target registration service, and generate a user service result corresponding to the user service request.

[0109] In some embodiments of the present application, the data processing device 70 in the embodiments of the present application may also include a screening module for screening a target registration service that matches the user service request from a registration service list through a load balancing algorithm when a user service request from a user terminal forwarded by a gateway is received.

[0110] In some embodiments of the present application, the data processing device 70 in the embodiment of the present application may further include a sending module, which is used to send a first service request to the i-th microservice among the M microservices through an external open interface according to the service flow driving parameter when receiving a user service request from the user end forwarded by the gateway, wherein the first service request carries first processing data related to the service processing function of the i-th microservice, and the first service request is used to request the i-th microservice to determine first result data based on the first processing data;

[0111] The data processing device 70 in the embodiment of the present application may further include a determination module, which is used to deploy the first result data in the target container when the first result data is received through the callback interface corresponding to the external open interface, and determine whether there is a parameter of the i+1th microservice indicating that the i-th microservice has a dependency relationship in the service flow driving parameter;

[0112] The sending module may also be used to, when it is determined that the service flow driving parameters contain a parameter of the i+1th microservice indicating that the i-th microservice has a dependency, send a second service request to the i+1th microservice through an external open interface, the second service request carrying the first result data and second processing data related to the service processing function of the i+1th microservice, the second service request being used to request the i+1th microservice to determine the second result data based on the first result number and the second processing data;

[0113] The determination module may also be used to, when the second result data is received through the callback interface and it is determined that there is no (i+1)th microservice in the service flow driving parameter indicating that the (i)th microservice has a dependency, determine the second result data as a user service result corresponding to the user service request.

[0114] In some embodiments of the present application, the data processing device 70 in the embodiment of the present application may further include a receiving module, which is used to receive a first input of a user to a microservice configuration interface, the microservice configuration interface includes P preset microservice identifiers, the first input includes an input of a user selecting M microservice identifiers from the P preset microservice identifiers, and P is an integer greater than 1;

[0115] In the embodiment of the present application, the data processing device 70 may further include a determination module, which is used to determine, in response to the first input, microservice data of the microservice corresponding to each microservice identifier in the M microservice identifiers, where the microservice data includes data required for the operation of the microservice;

[0116] In the embodiment of the present application, the data processing device 70 may further include a generation module for generating service orchestration data of M microservices based on the microservice data of the microservice corresponding to each microservice identifier.

[0117] In some embodiments of the present application, the data processing device 70 in the embodiment of the present application may further include a determination module for determining whether there is a microservice having a dependency relationship with the microservice based on the microservice data of the microservice corresponding to each microservice identifier;

[0118] In the embodiment of the present application, the data processing device 70 may further include an association module, which is used to associate the microservice with the microservice having a dependency relationship with the microservice to obtain microservice chain information when it is determined that there is a microservice having a dependency relationship with the microservice;

[0119] The association module is also used to associate the microservice data of the microservice corresponding to each microservice identifier according to the connection order of the microservices in the microservice chain information to obtain the microservice data chain information; the data processing device 70 in the embodiment of the present application may also include a generation module for generating service orchestration data of M microservices based on the microservice chain information and the microservice data chain information.

[0120] In some embodiments of the present application, the data processing device 70 in the embodiment of the present application may further include an update module, which is used to receive an application version update request sent by a service application corresponding to the M microservices, where the application version update request carries updated version data of the service application;

[0121] In the embodiment of the present application, the data processing device 70 may further include a generating module for generating, based on the service data to be updated, update service orchestration data for the M microservices when the update version data includes the service data to be updated corresponding to the microservice image data in the target container of the N containers;

[0122] A construction module 702 is used to construct an updated microservice image data package based on the updated service orchestration data, where the updated microservice image data includes the updated service orchestration data and the updated data required to run the M microservices;

[0123] In the embodiment of the present application, the data processing device 70 may further include an update module for updating the microservice image data in the target container to an updated microservice image data packet;

[0124] The update module may also be used to update the microservice image data in containers other than the target container among the N containers according to the updated microservice image data packet in the target container.

[0125] Based on the same inventive concept, the present application also provides a computer device. Figure 8 Provide detailed explanation.

[0126] Figure 8 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application.

[0127] like Figure 8 As shown, the computer device may include at least one of the following involved in the embodiments of the present application: a user terminal device, a server terminal device. The computer device may include a processor 801 and a memory 802 storing computer program instructions.

[0128] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit (ASTC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0129] The memory 802 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 802 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 802 is a non-volatile solid-state memory. In a specific embodiment, the memory 802 includes a solid-state storage (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0130] The processor 801 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 802 .

[0131] In one example, the computer device may further include a communication interface 803 and a bus 810. Figure 8 As shown, the processor 801, the memory 802, and the communication interface 803 are connected via a bus 810 and communicate with each other.

[0132] The communication interface 803 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0133] Bus 810 includes hardware, software or both, and the parts of flow control device are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industry standard system (ETSA) bus, front side bus (FSB), hypertransport (HT) interconnection, industry standard system (TSA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel system (MCA) bus, peripheral component interconnection (PCT) bus, PCT-Express (PCT-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 810 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0134] The data processing device can execute the data processing method in the embodiment of the present application, thereby realizing the combination Figures 2 to 8 The data processing method and device described.

[0135] In addition, in combination with the data processing method in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.

[0136] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0137] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0138] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0139] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A data processing method, comprising: Obtain service orchestration data of M microservices, wherein the service orchestration data includes call logic data, view interface data, and service deployment environment data of the microservices, where M is a positive integer; Based on the service orchestration data, a microservice image data packet is constructed, where the microservice image data includes the service orchestration data and data required to run the M microservices; The microservice image data packet is deployed in N containers of a container system, and there is a container isolation mechanism between the containers in the container system, where N is a positive integer.

2. The method according to claim 1, wherein: The method further comprises: When the service applications corresponding to the M microservices are started, the microservice image data packets in the target containers of the N containers are loaded by static loading to obtain service flow driving parameters; In the case of receiving a user service request from a user end forwarded by the gateway, scheduling the M microservices according to the service flow driving parameters, and generating a user service result corresponding to the user service request; wherein the user end is a user end corresponding to the service application; A forwarding instruction is sent to the gateway, wherein the forwarding instruction carries the user service result, and the forwarding instruction is used to instruct the gateway to forward the user service result to the user terminal.

3. The method according to claim 2, wherein: In the case of receiving a user service request from a user end forwarded by a gateway, scheduling the M microservices according to the service flow driving parameters and generating a user service result corresponding to the user service request, including: In case of receiving a user service request from a user end forwarded by the gateway, screening a target registration service matching the user service request from the registration service list; The M microservices are scheduled according to the service flow driving parameters through the target service instance corresponding to the target registration service to generate a user service result corresponding to the user service request.

4. The method according to claim 3, wherein: The step of, upon receiving a user service request from a user terminal forwarded by a gateway, screening a target registration service matching the user service request from a registration service list comprises: When receiving a user service request from a user end forwarded by the gateway, a target registration service matching the user service request is screened from the registration service list through a load balancing algorithm.

5. The method according to claim 2, wherein: In the case of receiving a user service request from a user end forwarded by a gateway, scheduling the M microservices according to the service flow driving parameters and generating a user service result corresponding to the user service request, including: In the case of receiving a user service request from a user end forwarded by the gateway, sending a first service request to an i-th microservice among the M microservices through an external open interface according to the service flow driving parameter, wherein the first service request carries first processing data related to a service processing function of the i-th microservice, and the first service request is used to request the i-th microservice to determine first result data based on the first processing data; In a case where the first result data is received through the callback interface corresponding to the external open interface, the first result data is deployed in the target container, and it is determined whether there is a parameter of the (i+1)th microservice indicating that the (i)th microservice has a dependency relationship in the service flow driving parameter; When it is determined that the service flow driving parameters contain a parameter of the (i+1)th microservice indicating that the (i)th microservice has a dependency, sending a second service request to the (i+1)th microservice through the external open interface, where the second service request carries the first result data and second processing data related to the service processing function of the (i+1)th microservice, and the second service request is used to request the (i+1)th microservice to determine the second result data based on the first result number and the second processing data; When the second result data is received through the callback interface and it is determined that the service flow driving parameter does not contain an (i+1)th microservice indicating that the (i)th microservice has a dependency relationship, the second result data is determined as a user service result corresponding to the user service request.

6. The method according to claim 1, wherein: Before obtaining the service orchestration data of the M microservices, the method further includes: Receive a first input from a user on a microservice configuration interface, the microservice configuration interface including P preset microservice identifiers, the first input including an input of the user selecting M microservice identifiers from the P preset microservice identifiers, where P is an integer greater than 1; In response to the first input, according to each microservice identifier in the M microservice identifiers, determining microservice data of a microservice corresponding to each microservice identifier, wherein the microservice data includes data required for the operation of the microservice; Based on the microservice data of the microservice corresponding to each microservice identifier, the service orchestration data of the M microservices is generated.

7. The method according to claim 6, wherein: The generating the service orchestration data of the M microservices based on the microservice data of the microservice corresponding to each microservice identifier includes: Based on the microservice data of the microservice corresponding to each microservice identifier, determining whether there is a microservice having a dependency relationship with the microservice; When it is determined that there is a microservice that has a dependency relationship with the microservice, the microservice is associated with the microservice that has a dependency relationship with the microservice to obtain microservice chain information; According to the connection order of the microservices in the microservice chain information, the microservice data of the microservice corresponding to each microservice identifier is associated to obtain microservice data chain information; Based on the microservice chain information and the microservice data chain information, service orchestration data of the M microservices is generated.

8. The method according to claim 1, wherein: After deploying the microservice image data packet in N containers of the container system, the method further includes: Receive an application version update request sent by a service application corresponding to the M microservices, where the application version update request carries updated version data of the service application; In a case where the updated version data includes service data to be updated corresponding to the microservice image data in the target container of the N containers, generating update service orchestration data corresponding to the M microservices based on the service data to be updated; Based on the update service orchestration data, construct an update microservice image data package, wherein the update microservice image data includes the update service orchestration data and update data required to run the M microservices; Updating the microservice image data in the target container to the updated microservice image data packet; According to the updated microservice image data packet in the target container, the microservice image data in the containers other than the target container in the N containers are updated.

9. A data processing device, comprising: An acquisition module is used to acquire service orchestration data of M microservices, wherein the service orchestration data includes call logic data, view interface data and service deployment environment data of the microservices, and M is a positive integer; A construction module, configured to construct a microservice image data packet based on the service orchestration data, wherein the microservice image data includes the service orchestration data and data required to run the M microservices; A deployment module is used to deploy the microservice image data packet in N containers of a container system, wherein there is a container isolation mechanism between the containers in the container system, and N is a positive integer.

10. A computer device, comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the steps of the data processing method according to any one of claims 1 to 8 are implemented.

11. A storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the data processing method according to any one of claims 1 to 8.

Images