High-concurrency data processing method based on cloud computing

Abstract

The invention provides a high-concurrency data processing method based on cloud computing, and relates to the technical field of cloud computing and database management.The method comprises the steps that a Kubernetes cluster is created and used for containerization management of database processing tasks; flyway is used for database version control, and it is ensured that database scripts are executed according to a preset sequence; the database initialization task in the initContainer is executed in a Pod container of the Kubernetes, and the database refreshing scripts are executed in sequence; and after the database is refreshed, starting the application program to access and use the updated database. Through combination of Kubernetes and Flyway, efficient resource management and database version control are implemented, efficient concurrent execution of database processing tasks is achieved, the problems that in a traditional method, the error rate is high, and the processing cost is too high are solved, and the data processing efficiency and stability are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing and database management, and particularly relates to a high-concurrency data processing method based on cloud computing. BACKGROUND

[0002] With the advent of the information age, the rapid growth of data has made it an important challenge for various application platforms to efficiently and accurately process and manage large amounts of data. MySQL, as a widely used open-source relational database management system, performs excellently in small-scale data management, but its traditional flushing method gradually exposes its limitations when dealing with large-scale, high-concurrency databases.

[0003] Currently, MySQL flushing technology mainly relies on SQL script execution for database data initialization and updating. However, with the explosive growth of data, the traditional flushing method is difficult to meet the demand for high-concurrency and large-scale data processing, resulting in the following problems:

[0004] (1) Complex relationship processing between databases: In a multi-database system, the data relationship between different databases is difficult to manage, and there is a lack of effective version control means.

[0005] (2) High error rate: The traditional database flushing process lacks precise version control, leading to errors during execution.

[0006] (3) High processing cost: Under the traditional method, processing a large number of concurrent requests and complex data requires a lot of manual intervention, resulting in high operation and maintenance costs. SUMMARY

[0007] The present application aims to at least partially solve one of the technical problems in the related art.

[0008] To this end, the first purpose of the present application is to propose a high-concurrency data processing method based on cloud computing, which combines the efficient resource management of Kubernetes and the version control of Flyway, ensuring the correct execution order of database scripts while efficiently processing concurrent tasks, thereby effectively reducing error rates, improving system processing capacity, and reducing operation and maintenance costs.

[0009] The second purpose of the present application is to propose a high-concurrency data processing device based on cloud computing.

[0010] The third purpose of the present application is to propose an electronic device.

[0011] The fourth purpose of the present application is to propose a computer-readable storage medium.

[0012] The fifth purpose of the present application is to propose a computer program product.

[0013] To achieve the above purpose, the first aspect of the present application proposes a high-concurrency data processing method based on cloud computing, comprising:

[0014] Creating a Kubernetes cluster for containerized management of database processing tasks;

[0015] Using Flyway for database version control to ensure that database scripts are executed in a predetermined order;

[0016] Executing database initialization tasks in initContainer within the Pod container of Kubernetes, and executing library flushing scripts in order;

[0017] After the database library flushing is completed, starting the application to access and use the updated database.

[0018] Optionally, the creation of the Kubernetes cluster for containerized management of database processing tasks comprises:

[0019] Configuring the nodes of the Kubernetes cluster to ensure high availability and load balancing;

[0020] Deploying database containers in the Kubernetes cluster and using containerization technology to manage database processing tasks;

[0021] Configuring an automatic expansion function to dynamically expand or reduce the number of containers according to load requirements;

[0022] Configuring resource scheduling strategies to ensure efficient allocation of resources for database processing tasks within the cluster;

[0023] Configuring a Kubernetes cluster monitoring system to monitor the execution status and resource usage of database processing tasks in real time.

[0024] Optionally, the use of Flyway for database version control to ensure that database scripts are executed in a predetermined order comprises:

[0025] Assigning a unique version number to each database script and managing the execution order of the scripts through Flyway;

[0026] During database initialization, Flyway automatically detects and executes unexecuted scripts according to the version number, ensuring that scripts are executed in version order;

[0027] During the execution of database scripts, record the version and status of each executed script to ensure traceability of database changes;

[0028] Configure Flyway's rollback mechanism to automatically roll back to the last stable state when script execution fails.

[0029] Ensure the consistency and integrity of the database structure and data through Flyway's verification mechanism after all scripts are executed.

[0030] Optionally, execute the database initialization task in initContainer within the Kubernetes Pod container, and execute the library flushing script in sequence, including:

[0031] Configure initContainer in the Kubernetes Pod container as a preprocessing stage for database initialization, ensuring that the database environment is ready before the main container starts;

[0032] Execute database connection and initialization scripts in initContainer to create necessary database structures and configurations, preparing the environment needed for library flushing;

[0033] Execute library flushing scripts in sequence according to Flyway-managed script versions, check the database state before each library flushing script execution to ensure that database changes do not conflict;

[0034] Use Kubernetes' container management capabilities to ensure that initContainer completes the library flushing task before the main container starts, and automatically terminates after the task is completed;

[0035] Configure the container log system to record detailed logs of each script execution to track the script execution process and diagnose possible errors.

[0036] Optionally, after the database library flushing is completed, start the application to access and use the updated database

[0037] After initContainer completes the database library flushing task and exits, Kubernetes automatically triggers the start of the main container, starting the Pod container containing the application;

[0038] The application connects to the database to ensure that the application can access and operate the latest database structure and data;

[0039] During startup, Kubernetes dynamically allocates appropriate resources based on the system's load balancing strategy to ensure that the application can run stably;

[0040] The application performs database connection verification when starting to ensure that the library flushing task is successfully executed and the database is in a usable state;

[0041] The application program detects the database state in real time through a monitoring mechanism to ensure that the application program is not affected by database changes during operation;

[0042] Once the application program is successfully started and the database connection is verified, the system begins to process user requests and provide services.

[0043] To achieve the above purpose, the second aspect embodiment of the present application proposes a high-concurrency data processing device based on cloud computing, comprising:

[0044] A Kubernetes cluster creation module is used to create a Kubernetes cluster, which realizes containerized management of database processing tasks;

[0045] A Flyway version control module is used to use Flyway for database version control to ensure that database scripts are executed in a predetermined order;

[0046] An initialization task execution module is used to execute database initialization tasks in initContainer within the Pod container of Kubernetes and execute the library flushing script in sequence;

[0047] An application program startup module is used to start the application program to access and use the updated database after the database library flushing is completed.

[0048] Optionally, the Kubernetes cluster creation module is used to:

[0049] Configure the nodes of the Kubernetes cluster to ensure high availability and load balancing;

[0050] Deploy database containers in the Kubernetes cluster and use containerization technology to manage database processing tasks;

[0051] Configure an automatic expansion function to dynamically expand or reduce the number of containers according to load requirements;

[0052] Configure a resource scheduling strategy to ensure efficient allocation of resources for database processing tasks within the cluster;

[0053] Configure a Kubernetes cluster monitoring system to monitor the execution status and resource usage of database processing tasks in real time.

[0054] To achieve the above purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: a processor, and a memory in communication connection with the processor;

[0055] The memory stores computer execution instructions;

[0056] The processor executes computer-executed instructions stored in the memory to implement the method of any one of the first aspect.

[0057] To achieve the above object, the fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores computer-executed instructions, the computer-executed instructions are executed by a processor to implement the method of any one of the first aspect.

[0058] To achieve the above object, the fifth aspect of the present application provides a computer program product, the computer program is executed by a processor to implement the method of any one of the first aspect.

[0059] Additional aspects and advantages of the present application will be made apparent by the following description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0060] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, wherein:

[0061] Figure 1 A flowchart of a high-concurrency data processing method based on cloud computing provided by an embodiment of the present application;

[0062] Figure 2 A structural diagram of a high-concurrency data processing device based on cloud computing provided by an embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0064] To solve the problems of the prior art, the embodiments of the present application provide a high-concurrency data processing method based on cloud computing, Figure 1 A flowchart of a high-concurrency data processing method based on cloud computing provided by an embodiment of the present application.

[0065] As shown in the figure, the method comprises the following steps: Figure 1

[0066] Step 101, create a Kubernetes cluster for containerized management of database processing tasks.

[0067] ​In this example, the first step is to create a Kubernetes cluster, which is primarily tasked with providing a highly available and scalable environment for containerized management of database processing tasks. The specific steps are as follows:

[0068] First, configure the nodes of the Kubernetes cluster to ensure high availability and load balancing of the system. By configuring the Kubernetes cluster with multiple nodes, you can effectively distribute the load and achieve fault tolerance, ensuring the stability of the cluster. In the cluster, each node is responsible for specific computing tasks, and multiple nodes work together to ensure that resources can be dynamically allocated when the load is high, avoiding single-point failures or overloads.

[0069] Next, deploy database containers in the Kubernetes cluster. Through containerization technology, database services are packaged into independent containers that can run on different computing nodes, which can improve the flexibility and scalability of database processing tasks. The benefits of containerized databases include isolating the database runtime environment, quickly starting, stopping, and migrating, reducing dependence on hardware resources, and making database processing more efficient.

[0070] To cope with changing load requirements, you also need to configure the automatic scaling function of Kubernetes. Through the automatic scaling function, you can automatically increase or decrease the number of containers based on the actual load requirements of the database. For example, when the load of database processing tasks is high, Kubernetes will automatically deploy more database containers to meet the high-concurrency processing requirements; when the load decreases, Kubernetes will automatically reduce the number of containers to save resources and avoid unnecessary waste of resources.

[0071] In terms of resource allocation, it is important to configure resource scheduling policies. Kubernetes cluster determines the resource allocation of containers through scheduling policies to ensure efficient execution of database processing tasks. The scheduling policy will dynamically schedule based on the needs of different containers and the load of the nodes to ensure that each container can obtain sufficient resources while avoiding overloading or idling of some containers, ensuring optimal use of cluster resources.

[0072] In addition, to monitor the execution status and resource usage of database processing tasks in real time, the Kubernetes cluster needs to be configured with a monitoring system. Through the cluster monitoring system, you can track the running status, resource usage, fault reports, etc. of each container in real time, and timely discover potential problems and make adjustments. For example, if the monitoring system detects that the resource usage of a container exceeds expectations, it will automatically trigger an alarm, and administrators can adjust resource allocation or start more containers based on the alarm.

[0073] Through the above configuration and management, the Kubernetes cluster can effectively containerize the management of database processing tasks, not only improving the flexibility and efficiency of database processing, but also ensuring the high availability, scalability and resource utilization of the system, providing a stable running environment for subsequent database processing tasks.

[0074] Step 102, use Flyway for database version control to ensure that database scripts are executed in a predetermined order.

[0075] In this embodiment, Flyway is used as a database version control tool to ensure that database scripts are executed in a predetermined order and to ensure that the database remains consistent and complete during execution. The specific steps are as follows:

[0076] First, assign a unique version number to each database script. Flyway manages the execution order of database scripts by using the version number of the script file. These version numbers are usually arranged in chronological order, and each script must ensure the uniqueness of the version number before submission. In this way, Flyway can determine the execution order of scripts based on version numbers, ensuring that database updates are performed in a predetermined sequence, avoiding version confusion or conflicts.

[0077] During database initialization, Flyway automatically detects and executes scripts that have not been executed based on version numbers. Whenever the database starts or initializes, Flyway scans the script versions in the database and compares them with the scripts that have already been executed. If it finds scripts that have not been executed, Flyway will execute these scripts in order according to the version sequence. In this way, Flyway ensures that database scripts are executed in a version-controlled order and can automatically skip already executed scripts, avoiding repeated execution.

[0078] During the execution of database scripts, Flyway records the version and status of each executed script. Flyway maintains a special history table (usually schema_version table) in the database to record the execution status of each script (such as executed, rolled back, etc.). This history table not only records the version information of each script, but also includes execution time, execution status, and execution logs. Through this recording mechanism, Flyway ensures the traceability of database changes, facilitating later viewing and auditing, especially when problems occur, allowing quick localization to specific scripts and execution status.

[0079] To ensure the stability of script execution, Flyway is configured with a rollback mechanism. If a script fails during execution, Flyway can automatically roll back to the last stable state, ensuring that the database is always consistent and stable. This means that during the execution of database scripts, if an error or exception occurs, Flyway will automatically recover the database to the state before execution according to the rollback mechanism, preventing erroneous database changes from affecting the entire system.

[0080] In addition, Flyway also provides a verification mechanism to ensure the consistency and integrity of the database structure and data after all scripts are executed. Flyway will verify whether the database structure is consistent with the expected state after each database script is executed. If the database has undergone unexpected changes (such as table structure errors, missing indexes, etc.), Flyway will mark these inconsistencies and prevent subsequent script execution, ensuring that the database is always in the expected state. This verification mechanism helps to avoid potential data corruption or structural abnormalities, ensuring that database changes do not compromise data consistency and integrity.

[0081] Through these operations, Flyway ensures that database script execution is orderly and stable, avoiding database problems caused by incorrect or inconsistent script execution order. At the same time, the rollback mechanism and verification function provided by Flyway enhance the reliability of database version control, greatly improving the security and stability of the database change process.

[0082] Step 103: Execute the database initialization task in initContainer within the Pod container of Kubernetes, and execute the library flushing script in order.

[0083] In this embodiment, step 103 describes how to use the Pod container and initContainer of Kubernetes to execute the database initialization task and ensure that the library flushing script is executed in the correct order. The specific steps are as follows:

[0084] First, configure initContainer in the Pod container of Kubernetes as a preprocessing stage for database initialization. initContainer runs before the main application container starts, mainly responsible for initializing the database environment, executing necessary scripts, etc. In this way, it can ensure that the database environment is fully prepared for the start of the main application container, avoiding problems encountered by the main container after starting.

[0085] In the initContainer, the first task is to execute the database connection and initialization scripts. Through these initialization scripts, the system will create the basic structure required for the database, including tables, indexes, stored procedures, and so on. Configuration settings for the database also need to be made, such as setting up the database connection pool, user permissions, and other related configurations. This process ensures that the database has the environment to support the application before the main container starts. In addition, the initContainer also needs to prepare the environment for library flushing, such as installing necessary database extensions, loading initial data, or performing data migration, and other tasks.

[0086] Next, the library flushing scripts are executed in order according to the Flyway-managed script version. Flyway version control ensures that all database scripts are executed in the correct order. In the initContainer, Flyway checks and executes each unexecuted library flushing script according to the script version number. Before each library flushing script is executed, the initContainer checks the database state to ensure that the current database environment can carry the content of the script to be executed. The purpose of this is to avoid version conflicts, data corruption, and other problems during database changes, and to ensure that the database structure remains consistent.

[0087] Using the container management function of Kubernetes, we can ensure that the initContainer completes the database initialization task before the main container starts. Kubernetes manages the execution process of the initContainer to ensure that the main container will not start until the database initialization task is completed. This mechanism ensures that the database environment is fully prepared and the main application container can start smoothly in an error-free database environment. After the database initialization task is completed, the initContainer will automatically terminate, and the Pod container will enter the startup phase of the main application container.

[0088] In addition, in order to facilitate subsequent troubleshooting and diagnosis, the initContainer also needs to configure the container log system to record detailed logs of each library flushing script execution. The log system will record the execution time, execution status (success or failure), error information, and other information to provide a complete execution history. These logs are of great significance for tracking the library flushing process, diagnosing possible errors, and analyzing execution performance. Through the log system, developers can clearly understand the execution of each script, identify and solve problems in a timely manner, and ensure the smooth progress of the database change process.

[0089] Through the above operations, initContainer can efficiently and securely perform database initialization tasks within the Pod container of Kubernetes, execute the library brushing script in order, and ensure the correctness and consistency of the database environment before the main container starts. This mechanism not only improves the reliability of the database initialization process, but also ensures the smooth execution of the library brushing script through Flyway version control, avoiding conflicts or inconsistencies in the database structure.

[0090] Step 104, after the database library brushing is completed, start the application to access and use the updated database.

[0091] In this embodiment, step 104 describes how to start the application after the database library brushing is completed, and ensure that the application can smoothly access and use the updated database. The specific steps are as follows:

[0092] First, when initContainer completes the database library brushing task and exits, Kubernetes automatically triggers the start of the main container, starting the Pod container containing the application. Kubernetes ensures that the main application container can only start after the database library brushing is completed through the automatic scheduling mechanism. This way ensures that the application starts with the latest structure and data of the database, avoiding inconsistent or unupdated database content when the application starts.

[0093] After the application starts, it will connect to the database to ensure that it can access and manipulate the latest database structure and data. The application's database connection is configured to connect to the latest database instance, ensuring that the application can access all updated tables, fields, indexes, and other structures in the library brushing process, and correctly handle the latest data in the database. In this way, the application can fully utilize the updates and optimizations brought by the database library brushing, ensuring the smooth operation of business logic.

[0094] During the startup process, Kubernetes dynamically allocates appropriate resources according to the system's load balancing strategy to ensure that the application can run stably. The load balancing strategy will automatically adjust the resources required by the application according to the current resource status of the cluster (such as CPU, memory usage, etc.). Kubernetes can automatically expand or reduce the number of containers according to the load situation, and dynamically allocate computing resources in the cluster to ensure that the application can run stably under high concurrency requests, avoiding resource overload or response delay.

[0095] To ensure that database changes do not negatively impact the application, the application performs a database connection verification at startup. The application verifies that the connection to the database is successful and confirms that the database flushing task has been executed correctly, and that the database is in a usable state. If the connection verification fails, the application will report an error and take appropriate measures to prevent the application from starting in an abnormal database state, avoiding potential business errors.

[0096] In addition, the application uses a monitoring mechanism to detect the database state in real-time during operation, ensuring that the application is not affected by database changes during runtime. The application can periodically check the connection health of the database, monitor response time, data consistency, and other key indicators to ensure normal operation and efficient response of the database. Through this monitoring mechanism, the application can detect database abnormalities in a timely manner and take measures to address potential problems, ensuring high availability of the system.

[0097] Once the application successfully starts and verifies the database connection, the system begins processing user requests and providing services. At this time, the application can execute business logic based on the latest database structure and data to process requests from users. The entire process ensures that the application can seamlessly access the database after it is updated and will not be affected by changes to the database, ensuring the stability and business continuity of the system.

[0098] Through the above operations, Kubernetes ensures that the application can successfully start and access the updated database after the database flushing is completed. At the same time, through load balancing, database verification, and monitoring mechanisms, the stable operation of the application and the high consistency of data are guaranteed. This process effectively avoids potential risks caused by database changes and improves the reliability of the system and the continuity of business services.

[0099] To achieve the above embodiments, the application also provides a high-concurrency data processing device based on cloud computing. Figure 2 A structural diagram of a high-concurrency data processing device based on cloud computing is provided for the embodiments of the application. As shown in Figure 2 The device includes:

[0100] A Kubernetes cluster creation module 100 is used to create a Kubernetes cluster, which implements containerized management of database processing tasks.

[0101] A Flyway version control module 200 is used to use Flyway for database version control to ensure that database scripts are executed in a predetermined order.

[0102] The initialization task execution module 300 is used to execute the database initialization task in the initContainer in the Pod container of Kubernetes and execute the database refresh script in sequence;

[0103] The application startup module 400 is used to start the application to access and use the updated database after the database refresh is completed.

[0104] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0105] In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0106] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0107] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0108] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0109] It is important to note that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold beyond these legitimate uses. Furthermore, such collection / sharing should be conducted only after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes the relevant user information before using the feature. Furthermore, any necessary steps must be taken to safeguard and secure access to such personal information and ensure that others with access to personal information comply with its privacy policy and procedures.

[0110] The present application contemplates an implementation that provides users with the ability to selectively opt in or opt out of permitting the collection and / or use of their personal information data. That is, the present disclosure contemplates providing users with the ability to prevent or limit the collection and / or use of their personal information data. For example, the present disclosure contemplates providing users with the ability to prevent or limit the collection and / or use of their personal information data by, for example, blocking or deleting cookies. In addition, the present disclosure contemplates providing users with the ability to determine whether and how to interact with the present disclosure by, for example, blocking web beacons. Further, the present disclosure contemplates providing users with the ability to access and / or edit their personal information data when such data is collected by the present disclosure. In addition, the present disclosure contemplates that the collection and / or use of personal information data can be limited to only those users who expressly consent or give permission to the collection and / or use of their personal information data.

[0111] In the foregoing detailed description, the description used with respect to the terms "one embodiment", "some embodiments”, "an example”, "a specific example” or "some examples” etc. means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Illustrative appearances of the above terms are not necessarily referred to the same embodiment or example throughout the description. Moreover, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. Additionally, the description given with respect to the processes described in the flow charts, or otherwise described herein, can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing the specified logical function(s) or process(es) and / or other implementation, as will be appreciated by those skilled in the art. The scope of preferred embodiments of the present application encompasses also other implementations wherein additional, not shown, devices can execute some or all of the processes described, or some combination thereof.

[0112] In addition, the terms "first”, "second”, etc. are used herein only to describe various steps in a method, process, or algorithm, and are not intended to refer to relative importance or a number of the identified steps. Accordingly, a feature defined with "first” or "second” can implicitly or explicitly include at least one of the feature. The meaning of "a plurality” herein is at least two, for example, two, three, etc., unless expressly specified otherwise.

[0113] Any processes or methods described in the flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the processes. The scope of preferred embodiments of the present application encompasses also other implementations wherein, for example, additional or fewer processes are performed, the processes are performed in a different order, or the processes are performed concurrently or with partial concurrence. Particularly, a plurality of processes can be performed at one time, and / or modules or components can be combined or divided.

[0114] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For the purposes of this specification, a "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can specifically include the following, which are non-exhaustive list: electrical connection (electrical device having one or more wires), portable computer diskette (magnetic device), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber device, and portable compact disc read-only memory (CDROM). In addition, the computer readable medium can even be paper or other suitable medium upon which the program is printed, as the program can be electronically captured, for example, by optically scanning the paper or other suitable medium, then electronically converted into a form that can be edited, compiled, or interpreted, or otherwise processed in electronic form into an executable form suitable for use in the instruction execution system, apparatus or device.

[0115] It should be understood that parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As in another embodiment, if implemented in hardware, any of the following technologies known in the art or their combination can be used: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0116] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0117] In addition, each of the function units in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0118] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

[0119] It should be understood that the various forms of flow shown above can be reordered, added or deleted steps. For example, each step described in the present application can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0120] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A high-concurrency data processing method based on cloud computing, characterized in that: The following steps are involved: Create a Kubernetes cluster for containerized management of database processing tasks; Use Flyway for database version control to ensure that database scripts are executed in the predetermined order; Execute the database initialization task in the initContainer in the Kubernetes Pod container and execute the database refresh script in sequence; After the database refresh is complete, start the application to access and use the updated database.

2. The method according to claim 1, characterized in that Creating a Kubernetes cluster for containerized management of database processing tasks includes: Configure the nodes of the Kubernetes cluster to ensure high availability and load balancing; Deploy database containers in a Kubernetes cluster and use containerization technology to manage database processing tasks; Configure automatic scaling to dynamically expand or reduce the number of containers based on load demand; Configure resource scheduling strategies to ensure efficient resource allocation for database processing tasks within the cluster; Configure the Kubernetes cluster monitoring system to monitor the execution status and resource usage of database processing tasks in real time.

3. The method according to claim 2, characterized in that The use of Flyway for database version control ensures that database scripts are executed in the predetermined order, including: Assign a unique version number to each database script and manage the execution order of the scripts through Flyway; When the database is initialized, Flyway automatically detects and executes unexecuted scripts based on the version number, ensuring that the scripts are executed in the order of the versions. During the database script execution process, the version and status of each executed script are recorded to ensure the traceability of database changes; Configure Flyway's rollback mechanism to automatically roll back to the last stable state when the script fails to execute; Flyway's verification mechanism ensures the consistency and integrity of the database structure and data after all scripts are executed.

4. The method according to claim 3, characterized in that The database initialization task in the initContainer is executed in the Kubernetes Pod container, and the database refresh script is executed in sequence, including: Configure initContainer in the Kubernetes Pod container as a pre-processing stage for database initialization to ensure that the database environment is ready before the main container starts; Execute the database connection and initialization scripts in the initContainer, create the necessary database structure and configuration, and prepare the environment required for refreshing the database; Execute database refresh scripts sequentially according to the script versions managed by Flyway. Check the database status before executing each refresh script to ensure that there are no conflicts in database changes. Use Kubernetes' container management function to ensure that the initContainer completes the database refresh task before the main container starts and automatically terminates after the task is completed; Configure the container logging system to record detailed logs for each script execution to track the script execution process and diagnose possible errors.

5. The method according to claim 4, characterized in that After the database is refreshed, start the application to access and use the updated database After the initContainer completes the database flushing task and exits, Kubernetes automatically triggers the startup of the main container and the Pod container containing the application; The application connects to the database to ensure that the application can access and operate the latest database structure and data; During the startup process, Kubernetes dynamically allocates appropriate resources based on the system's load balancing strategy to ensure stable operation of the application. When the application starts, it verifies the database connection to ensure that the database refresh task is successfully executed and the database is available. The application uses a monitoring mechanism to detect the database status in real time to ensure that the application is not affected by database changes during operation; Once the application is successfully started and the database connection is verified, the system begins to process user requests and provide services.

6. A high-concurrency data processing device based on cloud computing, characterized in that: include: Kubernetes cluster creation module, used to create Kubernetes clusters and implement containerized management of database processing tasks; Flyway version control module, used to use Flyway for database version control, ensuring that database scripts are executed in the predetermined order; The initialization task execution module is used to execute the database initialization task in the initContainer within the Kubernetes Pod container and execute the database refresh script in sequence; The application startup module is used to start the application to access and use the updated database after the database refresh is completed.

7. The device according to claim 6, characterized in that The Kubernetes cluster creation module is used to: Configure the nodes of the Kubernetes cluster to ensure high availability and load balancing; Deploy database containers in a Kubernetes cluster and use containerization technology to manage database processing tasks; Configure automatic scaling to dynamically expand or reduce the number of containers based on load demand; Configure resource scheduling strategies to ensure efficient resource allocation for database processing tasks within the cluster; Configure the Kubernetes cluster monitoring system to monitor the execution status and resource usage of database processing tasks in real time.

8. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when executed by a processor.

Images