Data persistence method and device for cloud computing platform and computer program product

Abstract

The embodiment of the invention provides a data persistence method and device for a cloud computing platform and a computer program product. The data persistence method comprises the steps that a persistence configuration file for a cloud computing platform is obtained, the cloud computing platform comprises a plurality of to-be-detected task nodes, and the persistence configuration file comprises detection information of the to-be-detected task nodes and plug-in information and storage information corresponding to the to-be-detected task nodes; according to the detection information, detecting the plurality of task nodes to be detected; when it is detected that the states of the task nodes in the to-be-detected task nodes are changed, determining a target analysis plug-in and a target storage component corresponding to the task nodes based on the persistent configuration file; analyzing the task node by using the target analysis plug-in to obtain analysis data of the task node; and carrying out persistence on the analysis data by utilizing the target storage component.

Description

Technical Field

[0001] This invention relates to the field of data persistence technology, and in particular to a data persistence method, device and computer program product for a cloud computing platform. Background Technology

[0002] Cloud-native refers to a collection of technologies and practices optimized for cloud environments, enabling software development and operations to fully leverage the elasticity, scalability, and distributed nature of cloud computing. Cloud-native technologies empower organizations to build and run scalable applications in modern, dynamic environments. Persistence refers to saving data to non-volatile storage media, such as hard drives, solid-state drives (SSDs), or cloud storage, so that this data can still be recovered and accessed after a system restart or crash.

[0003] Currently, in cloud-native computing platforms, to improve data processing efficiency, computationally intensive tasks are often broken down into multiple subtasks, which are then distributed across different computers for parallel execution. Finally, the computational tasks are aggregated. Since the processing logic and workflow of each task running on a cloud computing platform often differ, a set of persistence components is typically developed for each task's processing flow. These components can then be used to implement the respective persistence logic for each task.

[0004] However, due to the large variety and number of tasks running in cloud computing platforms, not only is the number of persistence logics used to implement persistence operations excessive, but the complexity of maintaining persistence logic is also increased. Summary of the Invention

[0005] This invention provides a data persistence method, device, and computer program product for a cloud computing platform, which can solve the problems of excessive persistence logic and high complexity in maintaining parsing logic in cloud computing platforms.

[0006] In a first aspect, embodiments of the present invention provide a data persistence method for a cloud computing platform, comprising:

[0007] Obtain a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple task nodes to be detected, the persistent configuration file including: detection information of the multiple task nodes to be detected, plugin information and storage information corresponding to each of the multiple task nodes to be detected;

[0008] Based on the detection information, the plurality of task nodes to be detected are detected;

[0009] When a change in the state of a task node is detected among the plurality of task nodes to be detected, the target parsing plugin and target storage component corresponding to the task node are determined based on the persistent configuration file.

[0010] The task node is parsed using the target parsing plugin to obtain the parsed data of the task node;

[0011] The parsed data is persisted using the target storage component.

[0012] Secondly, embodiments of the present invention provide a data persistence device for a cloud computing platform, comprising:

[0013] The first acquisition module is used to acquire a persistent configuration file for a cloud computing platform, wherein the cloud computing platform includes multiple task nodes to be detected, and the persistent configuration file includes: detection information of the multiple task nodes to be detected, plugin information and storage information corresponding to each of the multiple task nodes to be detected;

[0014] The first processing module is used to detect the plurality of task nodes to be detected based on the detection information;

[0015] The first determining module is used to determine the target parsing plugin and target storage component corresponding to the task node based on the persistent configuration file when it is detected that the state of the task node among the plurality of task nodes to be detected has changed.

[0016] The first processing module is further configured to use the target parsing plugin to parse the task node and obtain the parsed data of the task node;

[0017] The first persistence module is used to persist the parsed data using the target storage component.

[0018] Thirdly, embodiments of the present invention provide an electronic device, including: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the data persistence method of the cloud computing platform described in the first aspect is implemented.

[0019] Fourthly, embodiments of the present invention provide a computer storage medium for storing a computer program that, when executed by a computer, implements the data persistence method for the cloud computing platform described in the first aspect above.

[0020] Fifthly, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, wherein when the computer instructions are executed by one or more processors, the one or more processors cause the one or more processors to perform the steps in the data persistence method of the cloud computing platform described in the first aspect above.

[0021] Sixthly, embodiments of the present invention provide a method for persisting image data, comprising:

[0022] Obtain a persistent configuration file for a cloud computing platform, which includes multiple image processing nodes to be detected. The persistent configuration file includes: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected.

[0023] Based on the detection information, the plurality of image processing nodes to be detected are detected;

[0024] When it is detected that the state of one of the plurality of image processing nodes to be detected has changed, the target parsing plugin and target storage table corresponding to the image processing node are determined based on the persistent configuration file.

[0025] The image processing node is parsed using the target parsing plugin to obtain the image parsing data of the image processing node;

[0026] The image parsing data is persisted using the target storage table.

[0027] In a seventh aspect, embodiments of the present invention provide an image data persistence device, comprising:

[0028] The second acquisition module is used to acquire a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple image processing nodes to be detected, and the persistent configuration file including: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected;

[0029] The second processing module is used to detect the plurality of image processing nodes to be detected based on the detection information;

[0030] The second determining module is used to determine the target parsing plugin and target storage table corresponding to the image processing node based on the persistent configuration file when it is detected that the state of the image processing node among the plurality of image processing nodes to be detected has changed.

[0031] The first processing module also uses the target parsing plugin to parse the image processing node to obtain the image parsing data of the image processing node;

[0032] The second persistence module is used to persist the image parsing data using the target storage table.

[0033] Eighthly, embodiments of the present invention provide an electronic device, including: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image data persistence method in the sixth aspect above.

[0034] In a ninth aspect, embodiments of the present invention provide a computer storage medium for storing a computer program that, when executed by a computer, implements the image data persistence method described in the sixth aspect above.

[0035] In a tenth aspect, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the image data persistence method described in the sixth aspect above.

[0036] The data persistence method, device, and computer program product for cloud computing platforms provided in this invention effectively enable data persistence operations on multiple task nodes based on a single persistence logic of the cloud computing platform. Specifically, by obtaining a persistence configuration file for the cloud computing platform, the multiple task nodes to be detected are detected according to the detection information of multiple task nodes to be detected included in the persistence configuration file. When a change in the state of a task node is detected, the target parsing plugin and target storage component corresponding to the task node can be determined based on the persistence configuration file. Then, the target parsing plugin is used to parse the task node to obtain the parsed data of the task node, and the parsed data can be persisted using the target storage component. This not only effectively reduces the number of persistence logic operations for each task node, but also allows for the reuse of target parsing plugins for different task nodes, further reducing the complexity of maintaining the persistence logic. This effectively improves the flexibility and reliability of the method and ensures its practicality. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A schematic diagram illustrating a data persistence method for a cloud computing platform provided in an embodiment of the present invention;

[0039] Figure 2 A flowchart illustrating a data persistence method for a cloud computing platform provided in an embodiment of the present invention;

[0040] Figure 3 A schematic diagram illustrating the generation of a persistent configuration file as provided in an embodiment of the present invention;

[0041] Figure 4 A flowchart illustrating a data persistence method for a cloud computing platform provided in an embodiment of the present invention;

[0042] Figure 5 A schematic diagram illustrating the principle of a plugin-based resource persistence method provided for an application embodiment of the present invention;

[0043] Figure 6 A flowchart illustrating an image data persistence method provided in an embodiment of the present invention;

[0044] Figure 7 A schematic diagram of the structure of a data persistence device for a cloud computing platform provided in an embodiment of the present invention;

[0045] Figure 8 To and Figure 7 A schematic diagram of the electronic device corresponding to the data persistence device of the cloud computing platform provided in the embodiment;

[0046] Figure 9 This is a schematic diagram of the structure of an image data persistence device provided in an embodiment of the present invention;

[0047] Figure 10 To and Figure 9 A schematic diagram of the electronic device corresponding to the image data persistence device provided in the embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise; “multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0050] It should be understood that the term "and / or" used in this document is merely a description of the relationship between associated devices, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated devices are in an "or" relationship.

[0051] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0052] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes that element.

[0053] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0054] Terminology Explanation:

[0055] Cloud Native: Cloud Native refers to a collection of technologies and practices optimized for cloud environments, enabling software development and operations to fully leverage the elasticity, scalability, and distributed nature of cloud computing. Cloud Native technologies empower organizations to build and run scalable applications in modern, dynamic environments. The following are some key characteristics and concepts that constitute cloud-native applications:

[0056] (1) Microservice architecture: The application is designed as a set of small, independent and loosely coupled services. Each service implements a specific function and can be deployed, scaled and updated independently.

[0057] (2) Containerization: Applications and their runtime environments are encapsulated in lightweight containers that can run in a consistent manner in any cloud environment.

[0058] (3) Dynamic management and automation: Automated deployment, management, scaling and self-healing of application containers using container orchestration tools such as Kubernetes.

[0059] (4) DevOps and continuous delivery: Integrating development, testing and operations processes to make the building, testing and release of software faster, more frequent and more reliable.

[0060] (5) Observability: In distributed systems, it is necessary to monitor the performance, errors and operations of applications in real time in order to respond quickly to problems.

[0061] (6) Resilience and fault tolerance: Build applications that can automatically respond to changes, failures and changing demand pressures in the underlying infrastructure.

[0062] Persistence refers to saving data to non-volatile storage media, such as hard drives, solid-state drives (SSDs), or cloud storage, so that the data can still be recovered and accessed after a system restart or crash. Specifically, persistence has the following characteristics: (1) Reliability: Data persistence can ensure that data will not be lost over a long period of time; (2) Stability: Even if the system fails, the persisted data can be recovered; (3) Performance: Persistence operations are usually slower than memory operations because they involve disk read and write, but modern systems can minimize the performance impact through caching and optimization. Common persistence technologies and methods can include: databases (relational databases such as MySQL, NoSQL databases such as MongoDB), file systems (text files, binary files), serialization (serializing objects in memory into byte streams and saving them to disk), and log files.

[0063] Parser Plugin: A parser plugin is a pluggable module used to parse specific types of data. Parser plugins can be dynamically loaded and used to convert the input data format into another data structure or format used internally by the computer; it has the following characteristics: (1) Scalability: Through the plugin mechanism, the system can be extended to support new data formats or protocols at any time without modifying the main program; (2) Flexibility: Users can select and download the corresponding plugins as needed to meet different parsing requirements; (3) Modularity: Parser plugins are usually independent modules, which are easy to develop, test and maintain.

[0064] To facilitate understanding of the specific implementation process and effects of the data persistence method, device, and computer program product of the cloud computing platform in this embodiment, the relevant technologies are briefly described below:

[0065] In distributed computing platforms, to improve the efficiency of data processing, it is often necessary to break down computationally intensive tasks into multiple subtasks, then distribute these subtasks across different computers to run in parallel, and finally aggregate the computational tasks. Furthermore, in distributed computing platforms, it is not necessary to record the state of parent and child tasks.

[0066] To ensure that tasks in a distributed computing platform can still be recovered and accessed after a system restart or crash, persistence operations can be performed on the computing tasks (including larger tasks, subtasks, etc.). Since the processing logic and workflow of each task running in a cloud computing platform often differ, a persistence component is typically developed for each task's processing workflow. This component can then be used to implement the persistence logic for each task. For example, multiple tasks might include Task 1 and Task 2. The processing logic of Task 1 and Task 2 differs, requiring a persistence process 1 for Task 1 and a persistence process 2 for Task 2. These processes can then be used to perform persistence operations on Task 1 and Task 2 respectively. However, the above persistence method has the following drawbacks:

[0067] (1) Creating different persistent components for different node resources or task resources can easily lead to a large number of persistent components in the distributed computing platform;

[0068] (2) Different persistent components have their own resource parsing logic, which not only increases the complexity of maintaining persistent components, but also reduces the efficiency of developing persistent components because the repetitive logic in persistent components cannot be reused and the personalized logic in personalized persistent components is complex.

[0069] To address the aforementioned technical problems, embodiments of the present invention provide a data persistence method, device, and computer program product for a cloud computing platform, wherein, with reference to the appendix... Figure 1 As shown, the execution entity of the data persistence method of the cloud computing platform is the data persistence device 200 of the cloud computing platform (hereinafter referred to as "data persistence device 200"). The data persistence device 200 of the cloud computing platform is communicatively connected to the client 100. When the data persistence device 200 is implemented as a server in the cloud, the data persistence method can be executed in the cloud. Several computing nodes (cloud servers) can be deployed in the cloud, each with computing, storage, and other processing resources. In the cloud, multiple computing nodes can be organized to provide a certain service; of course, a single computing node can also provide one or more services. The cloud can provide this service by providing a service interface, which users call to use the corresponding service. Service interfaces include Software Development Kits (SDKs), Application Programming Interfaces (APIs), etc.

[0070] The client 100 can be a mobile phone, a personal computer (PC), a tablet computer, a settings application, a wearable device, etc. Furthermore, the basic structure of the client 100 may include at least one processor. The number of processors depends on the configuration and type of the client 100. The client 100 may also include memory, which can be volatile, such as Random Access Memory (RAM), or non-volatile, such as Read-Only Memory (ROM), flash memory, etc., or both types may be included. The memory typically stores the operating system (OS), one or more applications, and may also store program data. In addition to the processing unit and memory, the client 100 also includes some basic configurations, such as a network interface card (NIC) chip, an I / O bus, a display component, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, a mouse, a stylus, a printer, etc. Other peripheral devices are well known in the art and will not be described in detail here.

[0071] A data persistence device 200 refers to a device that can provide data persistence operations in a network virtual environment, typically referring to a device that utilizes a network for information planning and data persistence operations. In physical implementation, the data persistence device 200 can be any device capable of providing computing services and responding to data persistence requests to perform data persistence operations, such as a cluster server, a regular server, a cloud server, a cloud host, or a virtual data center. The data persistence device 200 mainly consists of a processor, hard disk, memory, system bus, etc., similar to a general computer architecture.

[0072] In this embodiment described above, a network connection is established between the client 100 and the data persistence device 200. This network connection can be a wireless or wired network connection. If the connection between the client 100 and the data persistence device 200 is a communication connection, the mobile network standard can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, 6G, etc.

[0073] Client 100 is used by users to implement applications that meet their data persistence needs. Specifically, when a user has a data persistence requirement, they can obtain a data persistence request. In some instances, the data persistence request can be obtained through human-computer interaction. In this case, client 100 can display a human-computer interaction interface and obtain the execution operation input by the user on the human-computer interaction interface; based on the execution operation, it obtains the data persistence request used to implement the data persistence operation; in order to implement the data persistence operation, the obtained data persistence request can be sent to data persistence device 200.

[0074] The data persistence device 200 is used to obtain data persistence requests through the client 100, and then analyze and process the data persistence requests to obtain a persistence configuration file for the cloud computing platform. The cloud computing platform may include multiple task nodes to be detected. The persistence configuration file includes: detection information for the multiple task nodes to be detected, as well as plugin information and storage information corresponding to each of the multiple task nodes to be detected. Since the state of the task nodes directly affects the data persistence operation, after obtaining the detection information of the multiple task nodes to be detected, the multiple task nodes to be detected can be detected based on the detection information. If the state of the multiple task nodes to be detected has not changed, then no data persistence operation is required for the multiple task nodes to be detected.

[0075] When a change in the state of one or more task nodes among multiple task nodes to be detected is detected, data persistence operations can be performed on the task nodes. Specifically, after obtaining the persistence configuration file, the persistence configuration file can be analyzed and processed to determine the target parsing plugin and target storage component corresponding to the task node. Then, the target parsing plugin can be used to parse the task node to obtain the parsed data of the task node. After that, the target storage component can be used to persist the parsed data, thereby effectively realizing the persistence operation of data in the cloud computing platform.

[0076] In this embodiment, a set of persistence logic based on a cloud computing platform is effectively implemented to perform persistence operations on data of multiple task nodes. This not only effectively reduces the number of persistence logic operations for each task node, but also allows the target parsing plugin used for parsing operations on each task node to be reused, thereby reducing the complexity of maintaining the persistence logic and further improving the flexibility and reliability of the method.

[0077] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0078] Figure 2 A flowchart illustrating a data persistence method for a cloud computing platform provided in an embodiment of the present invention; see attached diagram. Figure 2 As shown, this embodiment provides a data persistence method for a cloud computing platform. The execution subject of this method is a data persistence device for the cloud computing platform. This data persistence device can be implemented as software or a combination of software and hardware. When the data persistence device is implemented as hardware, it can be various electronic devices with data persistence operations, including but not limited to personal computers, servers, etc. When the data persistence device is implemented as software, it can be installed in the electronic devices listed above. Specifically, based on the above-mentioned data persistence device, the data persistence method for the cloud computing platform in this embodiment may include:

[0079] Step S201: Obtain a persistent configuration file for the cloud computing platform, which includes multiple task nodes to be detected. The persistent configuration file includes: detection information of the multiple task nodes to be detected, as well as plugin information and storage information corresponding to each of the multiple task nodes to be detected.

[0080] Step S202: Detect the multiple task nodes to be detected based on the detection information.

[0081] Step S203: When it is detected that the state of a task node among multiple task nodes to be detected has changed, the target parsing plugin and target storage component corresponding to the task node are determined based on the persistent configuration file.

[0082] Step S204: Use the target parsing plugin to parse the task nodes and obtain the parsed data of the task nodes.

[0083] Step S205: Persist the parsed data using the target storage component.

[0084] The specific implementation process and effects of each of the above steps are explained in detail below:

[0085] Step S201: Obtain a persistent configuration file for the cloud computing platform, which includes multiple task nodes to be detected. The persistent configuration file includes: detection information of the multiple task nodes to be detected, as well as plugin information and storage information corresponding to each of the multiple task nodes to be detected.

[0086] When users have persistent requirements for cloud computing platforms, the data persistence device can obtain a persistent configuration file for the cloud computing platform. This persistent configuration file can include detection information for multiple task nodes to be detected (two or more), as well as plugin information and storage information corresponding to each of these task nodes. The cloud computing platform can include multiple nodes, each with different node types and different data processing logic. The multiple task nodes to be detected can be at least a subset of these nodes, and different application scenarios may correspond to task nodes used to implement different data processing operations.

[0087] For example, a cloud computing platform may include: node 1, node 2, node 3, node 4 and node 5; multiple task nodes to be detected as determined in application scenario 1 may include: node 2, node 4 and node 5; multiple task nodes to be detected as determined in application scenario 2 may include: node 1, node 2 and node 3; and multiple task nodes to be detected as determined in application scenario 3 may include: node 1, node 3, node 4 and node 5.

[0088] Furthermore, for multiple task nodes to be detected, the detection information can be used to identify the running status or data status of the task nodes. The running status information can include at least one of the following: normal status, interrupted status, fault status, etc. The data status can include at least one of the following: data addition status, reduction status, deletion status, update status, etc. The plugin information corresponding to each of the multiple task nodes to be detected is used to identify the parsing plugins required by each task node. A task node can require one or more parsing plugins. Plugin information can include at least one of the following: the identity information of each parsing plugin, the name information of each parsing plugin, the storage address of each parsing plugin, etc. The storage information corresponding to each of the multiple task nodes to be detected is used to identify the storage components required by each task node. Different task nodes can correspond to the same or different storage components, and a task node can correspond to one or more storage components. Users can flexibly adjust or configure the storage information corresponding to task nodes according to application requirements, which can effectively ensure the reliability of data persistence operations.

[0089] Furthermore, this embodiment does not limit the specific method of obtaining the persistent configuration file. In some instances, the persistent configuration file can be obtained by accessing a preset device. In this case, obtaining the persistent configuration file for the cloud computing platform may include: determining a preset device that is communicatively connected to the data persistence device, wherein the preset device may store the persistent configuration file; and obtaining the persistent configuration file for the cloud computing platform by actively or passively accessing the preset device.

[0090] In other instances, persistent configuration files can be obtained not only by accessing preset devices but also through configuration operations. In this case, obtaining a persistent configuration file for a cloud computing platform can include: obtaining multiple task nodes to be detected based on node configuration operations input to the cloud computing platform; obtaining plugin information corresponding to each task node based on plugin configuration operations input to the multiple task nodes to be detected; obtaining storage information corresponding to each task node based on storage configuration operations input to the multiple task nodes to be detected; and generating a persistent configuration file based on the multiple task nodes to be detected, the plugin information corresponding to each task node, and the storage information.

[0091] Among them, reference appendix Figure 3As shown, in order to obtain a persistent configuration file through configuration operations, the data persistence device can display a configuration page for the persistent configuration file. Then, node configuration operations, plugin configuration operations, and storage configuration operations can be entered on the configuration page. After obtaining the node configuration operations entered for the cloud computing platform, multiple task nodes to be detected can be obtained directly based on the node configuration operations entered for the cloud computing platform. The multiple task nodes obtained can be at least a part of all task platforms for the cloud computing platform.

[0092] In some instances, node configuration operations can be implemented as node selection operations input to the cloud computing platform. In this case, multiple task nodes can be determined through node configuration operations. Specifically, the configuration page can display all task nodes in the cloud computing platform. Users can input node selection operations for multiple task nodes in the cloud computing platform, and then directly determine the task nodes corresponding to the node selection operations as the task nodes to be detected. This can effectively ensure the accuracy and reliability of obtaining multiple task nodes.

[0093] In other instances, multiple task nodes to be detected can be determined by the type of the task nodes. In this case, obtaining multiple task nodes to be detected based on the node configuration operation input to the cloud computing platform may include: obtaining the node type of each task node in the cloud computing platform based on the node configuration operation input to the cloud computing platform; and determining multiple task nodes to be detected based on the node type of each task node.

[0094] After obtaining the node configuration operations input to the cloud computing platform, the node type of each task node in the cloud computing platform can be obtained based on the node configuration operations. The node type of a task node can include at least one of the following: job node, node node, pod node, etc. In some instances, the node type can be determined by the node's identity identifier. In this case, after obtaining the node configuration operations, the identity identifier corresponding to each of the multiple task nodes to be detected can be determined based on the node configuration operations; then, the node type of each task node can be determined based on the identity identifier corresponding to each of the multiple task nodes.

[0095] Specifically, each task node in a cloud computing platform can correspond to different types, such as pod, job, and node. Different types of task nodes can be used to perform different data processing operations. After obtaining the node type of each task node, multiple task nodes to be detected can be determined based on the node type. For example, in scenario 1, multiple pod-type task nodes can be identified as multiple task nodes to be detected; or in scenario 2, multiple node-type and pod-type task nodes can be identified as multiple task nodes to be detected. This effectively ensures the accuracy and reliability of determining multiple task nodes.

[0096] After identifying multiple task nodes to be detected, it is also necessary to determine the plugin information corresponding to each task node. Specifically, this can be achieved by first obtaining the plugin configuration operations input for multiple task nodes. These plugin configuration operations can be user selection operations for any one of the multiple parsing plugins, or editing operations for plugin identifiers input for multiple task nodes. Then, the plugin information corresponding to each task node is obtained based on these plugin configuration operations. In some instances, plugin information can be determined through a preset mapping relationship. In this case, obtaining the plugin information corresponding to each task node based on the plugin configuration operations can include: obtaining the identity information of the plugins corresponding to each task node based on the plugin configuration operations; and then determining the plugin information corresponding to each task node based on the identity information of the plugins corresponding to each task node and the preset mapping relationship. This effectively ensures the accuracy and reliability of obtaining the plugin information corresponding to each task node.

[0097] In other instances, plugin information can be determined not only through preset mapping relationships, but also by combining a plugin library to determine the plugin information corresponding to each task node. In this case, obtaining the plugin information corresponding to each task node based on the plugin configuration operation input for multiple task nodes to be detected can include: obtaining a plugin library, which includes multiple parsing plugins for implementing data parsing operations; and obtaining the plugin information corresponding to each task node based on the plugin configuration operation among the multiple parsing plugins provided in the plugin library.

[0098] Specifically, to accurately obtain the plugin information corresponding to each of the multiple task nodes and ensure that the parsing plugins used by each task node are those provided by the data persistence device, a plugin library can be obtained first when configuring plugin information. This plugin library can include multiple parsing plugins used to implement data parsing operations; that is, the plugin library provides all the parsing plugins that can be downloaded or invoked. After obtaining the plugin library, the plugin information corresponding to each task node can be obtained from the multiple parsing plugins provided in the plugin library based on plugin configuration operations. In some instances, plugin configuration operations can be plugin selection operations or plugin marking operations, and then the plugin information corresponding to each task node can be obtained based on plugin selection operations or plugin marking operations. This effectively ensures the accuracy and reliability of obtaining the plugin information corresponding to each task node.

[0099] In addition to determining the plugin information corresponding to each task node, it is also necessary to determine the storage information corresponding to each task node. At this time, the storage configuration operation input for multiple task nodes can be obtained first, and then the storage information corresponding to each task node can be obtained based on the storage configuration operation. The storage information may include at least one of the following: the name of the storage component corresponding to each task node, the access address of the storage component, the type of the storage component, etc.

[0100] After obtaining multiple task nodes, their corresponding plugin information, and storage information, these can be analyzed and processed to generate a persistent configuration file. In some instances, the information can be integrated to generate a persistent configuration file that includes all the necessary details, thus ensuring the accuracy and reliability of obtaining the persistent configuration file.

[0101] Step S202: Detect the multiple task nodes to be detected based on the detection information.

[0102] For a persistent configuration file containing detection information for multiple task nodes to be detected, each task node can correspond to different status information. For example, a task node can correspond to status information 'a' and status information 'b'. Status information 'a' indicates that the running status of the task node has not changed, while status information 'b' indicates that the running status of the task node has changed. Since the node status of each task node can directly affect the persistence operation of the data of each task node, in order to perform real-time or near real-time persistence operations on the data of multiple task nodes, status detection can be performed on the aforementioned multiple task nodes based on the persistent configuration file. Specifically, status detection operations can be performed on multiple task nodes to be detected based on the detection information in the persistent configuration file.

[0103] Step S203: When it is detected that the state of a task node among multiple task nodes to be detected has changed, the target parsing plugin and target storage component corresponding to the task node are determined based on the persistent configuration file.

[0104] If the status detection results for multiple task nodes to be detected show that none of the task nodes have changed, then no data persistence operation will be triggered on any of the task nodes; that is, no operation needs to be performed on the multiple task nodes. However, if the status detection results show that some of the task nodes have changed, in order to ensure that the changed data can still be recovered and accessed after a system restart or crash, the target parsing plugin and target storage component corresponding to the task node can be determined based on the persistence configuration file.

[0105] In some instances, the target parsing plugin corresponding to a task node can be determined by accessing a preset plugin storage address. In this case, determining the target parsing plugin corresponding to a task node based on a persistent configuration file may include: determining the plugin storage address corresponding to at least one parsing plugin required by the task node based on the plugin information corresponding to each of the multiple task nodes included in the persistent configuration file; and then accessing the plugin storage address corresponding to at least one parsing plugin to obtain the target parsing plugin corresponding to the task node.

[0106] In other instances, the target parsing plugin corresponding to a task node can be determined not only by accessing the plugin storage address, but also by a pre-defined plugin library. In this case, determining the target parsing plugin corresponding to a task node based on a persistent configuration file may include: determining at least one plugin identifier corresponding to the task node based on the persistent configuration file; and identifying at least one parsing plugin corresponding to at least one plugin identifier as the target parsing plugin in the pre-defined plugin library, wherein the plugin library includes multiple parsing plugins for implementing data parsing operations.

[0107] Specifically, since the persistent configuration file includes plugin information corresponding to each task node, and the plugin information is used to identify the parsing plugins required by the task node, after obtaining the persistent configuration file, at least one plugin identifier corresponding to the task node can be determined based on the plugin information corresponding to the task node in the persistent configuration file. That is, when the task node requires multiple parsing plugins, the number of identified plugin identifiers is also multiple; when the task node requires only one parsing plugin, the number of identified plugin identifiers is also one.

[0108] After obtaining at least one plugin identifier, at least one parsing plugin corresponding to the at least one plugin identifier can be identified as the target parsing plugin in a preset plugin library. This effectively ensures the accuracy and reliability of the target parsing plugin identification. The preset plugin library can be a pre-configured plugin library for implementing data persistence operations. It can be configured in a preset area of ​​the data persistence device or communicate with the data persistence device. When data persistence operations are required, the preset database can be called to determine the target parsing plugin for parsing the task node. This effectively ensures the accuracy and reliability of the target parsing plugin identification.

[0109] It should be noted that, for the preset plugin library, the multiple parsing plugins included in the plugin library may not include the target parsing plugin required by the task node. In this case, in order to accurately perform data persistence operations on the task node, a plugin update operation can be performed on the preset plugin library. Specifically, the method in this embodiment may also include: obtaining plugin configuration information when the specific parsing plugin required by the task node does not exist in the plugin library; determining the specific parsing plugin required by the task node based on the plugin configuration information; updating the specific parsing plugin to the plugin library to obtain the updated plugin library.

[0110] After obtaining a specific parsing plugin required by the task node, it's possible to check if the plugin library contains such a plugin. If the plugin library doesn't contain the required plugin, to ensure stable data persistence for the task node, the plugin library can be updated with the configuration for that specific parsing plugin. Specifically, this involves first obtaining the plugin configuration information for both the plugin library and the specific parsing plugin. This configuration information can include at least one of the following: the logic code required by the specific parsing plugin, the name of the specific parsing plugin, the identifier of the specific parsing plugin, the plugin storage address of the specific parsing plugin, etc. This configuration information can be obtained through human-computer interaction. Obtaining this information can include: displaying a configuration interface for configuring the plugin library; obtaining the configuration operations entered by the user in the configuration interface for the specific parsing plugin; and obtaining the corresponding plugin configuration information based on these configuration operations.

[0111] After obtaining the plugin configuration information, it can be analyzed and processed to determine the specific parsing plugin required by the task node. Then, the specific parsing plugin can be updated to the plugin library to obtain the updated plugin library, which includes the specific parsing plugin required by the task node. Then, the specific parsing plugin can be used to perform persistent operations on the task node's data.

[0112] To accurately persist data on task nodes, it is necessary not only to determine the target parsing plugin used by the task node, but also the target storage component. Specifically, after obtaining the persistence configuration file, the target storage component corresponding to the task node can be determined directly based on the storage information in the persistence configuration file. The number of target storage components can be one or more. Specifically, the target storage component can be at least one of the following: database (relational database such as MySQL, NoSQL database such as MongoDB, etc.), file system (text file, binary file), log file, etc.

[0113] In other instances, the number of target storage components can be one. In this case, determining the target storage component corresponding to the task node based on the persistent configuration file can include: determining the storage component that the task node needs to use based on the persistent configuration file; when there is only one storage component, determining the storage component as the target storage component corresponding to the task node; when there are multiple storage components, determining one target storage component corresponding to the task node from among the multiple storage components.

[0114] Since the persistent configuration file includes storage information corresponding to the task node, after obtaining the persistent configuration file, the storage component required by the task node can be determined directly based on the storage information in the persistent configuration file. The number of storage components can be one or more. Since the target storage component is only one, different strategies can be used to determine the target storage component for different numbers of storage components. Specifically, when there is only one storage component, the obtained storage component can be directly determined as the target storage component required by the task node.

[0115] Correspondingly, when there are multiple storage components, a target storage component corresponding to the task node can be determined from among them. In some instances, a random selection algorithm can be used to randomly select one of the multiple storage components as the target storage component corresponding to the task node. In other instances, a load balancing algorithm and the data storage size of each storage component can be used to determine the target storage component corresponding to the task node. Specifically, the target storage component can be the storage component with the smallest data storage size among the multiple storage components. In still other instances, a load balancing algorithm and the storage space size of each storage component can be used to determine the target storage component corresponding to the task node. Specifically, the target storage component can be the storage component with the largest storage space among the multiple storage components. This effectively ensures the flexibility and reliability of determining the target storage component.

[0116] Step S204: Use the target parsing plugin to parse the task nodes and obtain the parsed data of the task nodes.

[0117] After obtaining the target parsing plugins, you can use them to parse the task nodes, thereby obtaining their parsed data. Different task nodes may require different target parsing plugins, resulting in different parsed data. For example, the target parsing plugins needed for a task node might include: Plugin 1 (for extracting node name information), Plugin 2 (for extracting node latency information), Plugin 3 (for extracting node network traffic), and Plugin 4 (for extracting node data transmission rate). Then, you can use these plugins to perform status parsing on the task nodes, obtaining their parsed data, which may include: the task node's name information, latency information, network traffic, and data transmission rate.

[0118] Similarly, the target resolution plugins required for each task node include: Plugin 1 (for extracting node name information), Plugin 4 (for extracting node data transfer rate), Plugin 5 (for extracting node resource usage), and Plugin 6 (for extracting node disk information). These plugins can then be used to perform status resolution operations on the task node, thereby obtaining its resolved data. This resolved data can include: the task node's name information, data transfer rate, resource usage, and disk information. Alternatively, the target resolution plugins required for each task node may include: Plugin 1 (for extracting node name information), Plugin 2 (for extracting node latency information), Plugin 3 (for extracting node network traffic), and Plugin 7 (for extracting node hardware information). These plugins can then be used to perform status resolution operations on the task node, thereby obtaining its resolved data. This resolved data can include: the task node's name information, latency information, network traffic, and hardware information. This effectively ensures the accuracy and reliability of obtaining the resolved data for each task node.

[0119] Step S205: Persist the parsed data using the target storage component.

[0120] After obtaining the parsed data, the parsed data can be persisted using the target storage component. This means storing the parsed data in the target storage component so that the stored parsed data can be recovered and accessed after a system restart or crash, thereby ensuring the stability and reliability of the data processing operations performed by the task nodes.

[0121] The data persistence method for cloud computing platforms provided in this embodiment effectively enables data persistence operations on multiple task nodes based on a single persistence logic of the cloud computing platform. Specifically, by obtaining a persistence configuration file for the cloud computing platform, the multiple task nodes to be detected are detected according to the detection information of multiple task nodes to be detected included in the persistence configuration file. When a change in the state of a task node is detected among the multiple task nodes to be detected, the target parsing plugin and target storage component corresponding to the task node can be determined based on the persistence configuration file. Then, the target parsing plugin is used to parse the task node to obtain the parsed data of the task node, and the parsed data can be persisted using the target storage component. This not only effectively reduces the amount of persistence logic for persistence operations on each task node, but also allows the target parsing plugin for different task nodes to be reused, further reducing the complexity of maintaining the persistence logic, further improving the flexibility and reliability of the method, and ensuring its practicality.

[0122] Figure 4A flowchart illustrating a data persistence method for a cloud computing platform provided by an embodiment of the present invention; based on any of the above embodiments, refer to the appendix. Figure 4 As shown, since the state of a task node directly affects whether data persistence is required, this embodiment provides a technical solution for determining whether the state of a task node has changed. Specifically, after obtaining the persistence configuration file for the cloud computing platform, the method may further include:

[0123] Step S401: Detect multiple task nodes based on persistent configuration files.

[0124] For task nodes, when their state changes, corresponding state change information (or event information) is generated; when their state remains unchanged, no such information is generated. Therefore, the state of a task node is closely related to whether it generates corresponding state change information. Thus, the state of a task node can be determined based on whether or not state change information is detected or obtained. In some instances, after obtaining a persistent configuration file, multiple task nodes can be detected based on this file to obtain the detection results for each node. In some examples, detecting multiple task nodes based on a persistent configuration file may include: using detection information from multiple task nodes in the persistent configuration file; performing real-time or near-real-time detection of the running status of multiple task nodes, and then determining the task node's state information by whether state change information used to identify the task node's state is obtained.

[0125] In other instances, not only can multiple task nodes be detected directly based on the persistent configuration file to obtain the detection results for each task node, but multiple task nodes can also be detected by combining their detection priorities. In this case, detecting multiple task nodes based on the persistent configuration file can include: obtaining the detection priorities for each task node; and detecting multiple task nodes based on the persistent configuration file and the detection priorities.

[0126] In this process, for multiple task nodes to be detected, different task nodes can have different detection priorities. These priorities indicate the importance of each task node; for example, important task nodes have higher priorities than less important ones. To ensure flexibility and reliability in detecting multiple task nodes, the detection priorities for each task node can be obtained first. In some instances, the detection priorities can be determined by the node identifiers of the task nodes, or by analyzing the task information to be processed by the task nodes using preset recognition rules. After obtaining the detection priorities for each task node, detection operations can be performed based on the persistent configuration file and the detection priorities. Specifically, the task nodes can first be sorted according to their detection priorities to obtain sorting information, and then the detection operations can be performed sequentially based on the persistent configuration file and the sorting information to obtain the detection results for each task node.

[0127] Step S402: When the status change information of the task node is obtained, it is determined that the status of the task node has changed.

[0128] Step S403: If no status change information of the task node is obtained, it is determined that the status of the task node has not changed.

[0129] When information about changes in the status of a task node is obtained, it can be determined that the status of the task node has changed. For example, the data in the task node may have increased, decreased, or changed, or the running status of the task node may have changed (from normal to faulty, or from faulty to normal). When no information about changes in the status of a task node is obtained, it can be determined that the status of the task node has not changed. This effectively detects whether the status of a task node has changed, and then data persistence operations can be performed based on the detection results, further improving the stability and reliability of the method.

[0130] In some other instances, after obtaining the state change information of the task node, corresponding processing operations can be performed on the data of the task node. In this case, the method in this embodiment may also include: parsing the state change information to obtain the parsing result; determining the operation instruction corresponding to the task node based on the parsing result; and executing the data processing operation corresponding to the task node based on the operation instruction.

[0131] Specifically, since the state change information of a task node can identify changes in its state, in order to accurately perform corresponding data processing operations, after obtaining the state change information, a parsing operation can be performed. This involves converting the obtained state change information into computer language that the data persistence device can recognize, thereby obtaining the parsing result of the state change information. After obtaining the parsing result, the operation instruction corresponding to the task node can be determined based on the parsing result. The operation instruction can include at least one of the following: data write instruction, data delete instruction, data update instruction, etc. After obtaining the operation instruction, the data processing operation corresponding to the task node can be executed based on the operation instruction. For example, when the operation instruction is a data write instruction, a data write operation corresponding to the task node can be executed based on the data write instruction; when the operation instruction is a data delete instruction, a data delete operation corresponding to the task node can be executed based on the data delete instruction; when the operation instruction is a data update instruction, a data update operation corresponding to the task node can be executed based on the data update instruction. This effectively ensures the stable and reliable operation of the task node.

[0132] In this embodiment, multiple task nodes are detected based on a persistent configuration file. When the state change information of a task node is obtained, it is determined that the state of the task node has changed. When the state change information of a task node is not obtained, it is determined that the state of the task node has not changed. This effectively realizes a stable detection operation of the state of the task nodes, and then facilitates the determination of whether the data of the task nodes needs to be persisted based on the detected state of the task nodes, further improving the practicality of the method.

[0133] In practical applications, to address the issue that the complexity of persistence logic for each node increases proportionally with the addition of nodes with different characteristics, please refer to the appendix. Figure 5 As shown, this application embodiment provides a plug-in-configurable data persistence method for a cloud computing platform. The execution entity of this data persistence method can be a persistence component, providing a unified persistence component to offer general data persistence capabilities for multiple task nodes. It can also accumulate a plug-in library including parsing plug-ins required by each task node. The parsing logic of each parsing plug-in in the plug-in library is isolated from each other, and it can also include specific or customized parsing plug-ins required by the task nodes. This not only reduces the complexity of the persistence process but also improves the maintainability of the persistence logic, while supporting configurability for different storage media. Specifically, the method may include the following steps:

[0134] Step 1: Create a configuration file.

[0135] When users have data persistence requirements for cloud computing platforms, in order to achieve data persistence operations, they can first use a persistence component or configure a configuration file corresponding to the cloud computing platform within the persistence component. In some instances, the configuration file can be configured based on human-computer interaction. Specifically, the configuration file can include: detection information of multiple task nodes to be detected (e.g., task node 1, task node 2, task node 3, etc.), as well as the plugin information and storage information corresponding to each task node. The multiple task nodes to be detected can be determined by the type of task node, which can include at least one of the following: Pod node, job node, node node; the storage information is used to identify the storage component required by each task node. The storage component can be implemented as a database, data table, etc., for storing data. Users can flexibly configure or adjust the specific implementation of the storage component according to specific application scenarios or application requirements.

[0136] In different application scenarios, task nodes can have different implementation principles. For example, task nodes can be machine learning task nodes, image processing task nodes, etc. In addition, the plugin information corresponding to each task node defined in the configuration file can identify the parsing plugins required by each task node. The parsing plugins required by each task node can be some of the parsing plugins provided by the preset plugin library. If the parsing plugin required by the task node is not stored in the preset plugin library, the preset plugin library can be configured with new parsing plugins first, thereby realizing the update operation of the plugin library. Then, the parsing plugin required by the task node can be determined from the updated plugin library.

[0137] Step 2: Based on the information in the configuration file (Watch-Config), perform status detection operations on multiple task nodes to be detected.

[0138] In some instances, the persistent component may include a detection module for performing status checks on task nodes. After obtaining the configuration file, the detection module can directly determine multiple task nodes to be checked based on the information in the configuration file, and then perform status checks on these task nodes directly. Alternatively, after determining multiple task nodes to be checked, the detection priority for each task node can be determined, and then multiple detection queues can be generated based on these priorities. Different detection queues can have different detection priorities, and then the task nodes in each detection queue can be checked sequentially based on their detection priorities. It is understood that detection queues with higher detection priorities should be checked first, and detection queues with lower detection priorities should be checked later.

[0139] Step 3: When the event information of the task node (corresponding to the state change information in the above embodiment) is obtained, the event information can be processed in a general way to determine the operation instruction corresponding to the task node; and then the data processing operation corresponding to the task node is executed based on the operation instruction.

[0140] Upon obtaining the event information of a task node, it indicates a change in the task node's state. This allows for corresponding processing logic for state changes, such as setting default values, adding, deleting, modifying, and querying data. Therefore, after obtaining the task node's event information, the corresponding operation instructions can be determined. Specifically, the general processing module included in the persistence component can be used to analyze and process the event information to obtain the corresponding operation instructions. These instructions can include at least one of the following: data addition, data deletion, data modification, and data query instructions. Then, data processing operations corresponding to the task node can be executed based on these operation instructions.

[0141] Step 4: When the event information of the task node is obtained, the target parsing plugin corresponding to the task node is determined based on the persistent configuration file.

[0142] The persistent component may also include a plugin selection module for determining the target parsing plugin. When the state of a task node changes, the plugin selection module can analyze and process the plugin information in the configuration file to determine the target parsing plugin corresponding to the task node. Specifically, a plugin library can be obtained first. This plugin library can be configured in the persistent component or in a device that communicates with the persistent component. The plugin library includes multiple parsing plugins for implementing data parsing operations. After obtaining the plugin library, the plugin information corresponding to each task node can be obtained from the multiple parsing plugins provided in the plugin library based on the plugin configuration operation. Then, the target parsing plugin corresponding to the task node can be determined based on the plugin information corresponding to each task node. Specifically, the number of target parsing plugins can be one or more.

[0143] Step 5: Use the target parsing plugin to parse the data of the task nodes and obtain the parsed data of the task nodes.

[0144] The number of target parsing plugins can be one or more, and the target parsing plugins corresponding to different task nodes can be reused. In this case, the target parsing plugins that can be reused can be ordinary plugins in the plugin library, while the target parsing plugins that cannot be reused can be personalized plugins or specific plugins in the plugin library. For example, for task node 1, the corresponding target parsing plugins may include: ordinary plugin 1, ordinary plugin 2, ordinary plugin 3, and specific plugin 1; for task node 2, the corresponding target parsing plugins may include: ordinary plugin 2, ordinary plugin 3, ordinary plugin 4, and specific plugin 2. Then, the determined target parsing plugins can be used to parse the data of each task node, thereby obtaining the parsed data of the task node.

[0145] Step 6: Determine the target storage component corresponding to the task node based on the persistent configuration file, and then use the target storage component to perform persistent operations on the parsed data.

[0146] The node persistence solution based on plug-in configuration provided in this application embodiment has advantages such as a unified persistence component, a built-in plug-in library, and configurable storage media. Specifically, by abstracting the persistence process, a unified persistence component is formed. This component not only provides general data persistence capabilities but also builds a plug-in library. Through the unified persistence process provided by the persistence component, the introduction of a data parsing plug-in library, and configuration files, data persistence configuration operations are performed for multiple different task nodes, achieving unified storage of information from different nodes, thereby effectively solidifying / simplifying the data persistence process. Specifically, this method has the following advantages:

[0147] (1) A plugin library has been formed, which includes general plugins and personalized plugins. This not only allows personalized logic to be precipitated into plugins and isolated, but also allows personalized logic to be precipitated for reuse. This not only reduces the complexity of personalized logic, but also improves the reusability of logic. In addition, users can flexibly update and configure the plugins included in the plugin library according to their needs. Ordinary plugins and personalized plugins in the database are stored in isolation from each other, which facilitates the improvement of the quality and efficiency of parsing plugin dimensions.

[0148] (2) By abstracting the persistence process, a unified persistence component is formed, which makes the data persistence solution universal; and the four key links of detection, field mapping, parsing operation and data storage operation are configured, which solves the problem in related technologies that different persistence components are required for different task nodes, resulting in too many persistence components.

[0149] (3) It is applicable to distributed computing platforms and can be applied in distributed scenarios in cloud-native environments. For example, it can be applied in scenarios such as cloud computing platforms, artificial intelligence platforms, and data processing pipelines (such as data parsing in streaming data processing systems), which effectively expands the scope of application and practicality of the method.

[0150] Figure 6 A flowchart illustrating an image data persistence method provided in an embodiment of the present invention; see attached diagram. Figure 6 As shown, this embodiment provides a method for persisting image data. The execution subject of this method is an image data persistence device, which can be implemented as software or a combination of software and hardware. When the image data persistence device is implemented as hardware, it can be various electronic devices with image data persistence operations, including but not limited to personal computers, servers, etc. When the image data persistence device is implemented as software, it can be installed in the electronic devices exemplified above. Specifically, based on the above-mentioned image data persistence device, the image data persistence method in this embodiment may include:

[0151] Step S601: Obtain a persistent configuration file for the cloud computing platform, which includes multiple image processing nodes to be detected. The persistent configuration file includes: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected.

[0152] Step S602: Detect the plurality of image processing nodes to be detected based on the detection information;

[0153] Step S603: When it is detected that the state of one of the plurality of image processing nodes to be detected has changed, the target parsing plugin and target storage table corresponding to the image processing node are determined based on the persistent configuration file.

[0154] Step S604: Use the target parsing plugin to parse the image processing node and obtain the image parsing data of the image processing node.

[0155] Step S605: Persist the image parsing data using the target storage table.

[0156] The specific implementation method, implementation principle, and implementation effect of the above steps in this embodiment are the same as those described above. Figure 2 The specific implementation methods, implementation principles and implementation effects of steps S201-S205 in the illustrated embodiments are similar. For details, please refer to the above description and will not be repeated here.

[0157] In addition, the method in this embodiment can be executed. Figures 1-5 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figures 1-5 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figures 1-5 The descriptions in the illustrated embodiments will not be repeated here.

[0158] The image data persistence method provided in this embodiment effectively enables data persistence operations on multiple image processing nodes based on a single persistence logic of a cloud computing platform. Specifically, it obtains a persistence configuration file for the cloud computing platform, and then performs state detection on the image processing nodes based on the detection information of multiple image processing nodes to be detected in the persistence configuration file. When a state change of an image processing node is detected among the multiple image processing nodes to be detected, the target parsing plugin and target storage table corresponding to the image processing node are determined based on the persistence configuration file. Then, the target parsing plugin is used to parse the image processing node to obtain the image parsing data of the image processing node, and the target storage table is used to persist the image parsing data. This not only effectively reduces the number of persistence logic operations for each image processing node, but also reduces the complexity of maintaining the persistence logic, further improving the flexibility and reliability of the method and ensuring its practicality.

[0159] Figure 7 This is a schematic diagram of the structure of a data persistence device for a cloud computing platform provided in an embodiment of the present invention; see attached diagram. Figure 7As shown, this embodiment provides a data persistence device for a cloud computing platform, which is used to perform the above-described... Figure 2 The data persistence method for the cloud computing platform shown may include:

[0160] The first acquisition module 11 is used to acquire a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple task nodes to be detected, and the persistent configuration file including: detection information of the multiple task nodes to be detected, plugin information and storage information corresponding to each of the multiple task nodes to be detected;

[0161] The first processing module 13 is used to detect the plurality of task nodes to be detected based on the detection information;

[0162] The first determining module 12 is used to determine the target parsing plugin and target storage component corresponding to the task node based on the persistent configuration file when the state of a task node among the plurality of task nodes to be detected changes.

[0163] The first processing module 13 is also used to parse the task nodes using the target parsing plugin to obtain the parsed data of the task nodes;

[0164] The first persistence module 14 is used to persist the parsed data using the target storage component.

[0165] In some instances, when the first acquisition module 11 acquires the persistent configuration file for the cloud computing platform, the first acquisition module 11 is used to: obtain multiple task nodes to be detected based on the node configuration operation input for the cloud computing platform; obtain the plugin information corresponding to each task node based on the plugin configuration operation input for the multiple task nodes to be detected; obtain the storage information corresponding to each task node based on the storage configuration operation input for the multiple task nodes to be detected; and generate a persistent configuration file based on the multiple task nodes to be detected, the plugin information corresponding to each task node, and the storage information.

[0166] In some instances, when the first acquisition module 11 obtains multiple task nodes to be detected based on the node configuration operation input to the cloud computing platform, the first acquisition module 11 is used to: obtain the node type of each task node in the cloud computing platform based on the node configuration operation input to the cloud computing platform; and determine multiple task nodes to be detected based on the node type of each task node.

[0167] In some instances, when the first acquisition module 11 obtains plugin information corresponding to each task node based on the plugin configuration operation input for multiple task nodes to be detected, the first acquisition module 11 is used to: acquire a plugin library, which includes multiple parsing plugins for implementing data parsing operations; and, based on the plugin configuration operation, obtain the plugin information corresponding to each task node from among the multiple parsing plugins provided in the plugin library.

[0168] In some instances, after obtaining the persistent configuration file for the cloud computing platform, the first processing module 13 in this embodiment performs the following steps: detecting multiple task nodes based on the persistent configuration file; when the state change information of the task node is obtained, it is determined that the state of the task node has changed; when the state change information of the task node is not obtained, it is determined that the state of the task node has not changed.

[0169] In some instances, when the first processing module 13 detects multiple task nodes based on a persistent configuration file, the first processing module 13 is used to perform: obtaining the detection priority corresponding to each of the multiple task nodes; and detecting the multiple task nodes based on the persistent configuration file and the detection priority.

[0170] In some instances, after obtaining the state change information of the task node, the first processing module 13 in this embodiment performs the following steps: parses the state change information to obtain the parsing result; determines the operation instruction corresponding to the task node based on the parsing result; and performs the data processing operation corresponding to the task node based on the operation instruction.

[0171] In some instances, when the first determining module 12 determines the target parsing plugin corresponding to the task node based on the persistent configuration file, the first determining module 12 is used to: determine at least one plugin identifier corresponding to the task node based on the persistent configuration file; and determine at least one parsing plugin corresponding to at least one plugin identifier as the target parsing plugin in a preset plugin library, wherein the plugin library includes multiple parsing plugins for implementing data parsing operations.

[0172] In some instances, the first acquisition module 11 and the first determination module 12 in this embodiment are used to perform the following steps:

[0173] The first acquisition module 11 is used to acquire plugin configuration information when the specific parsing plugin required by the task node does not exist in the plugin library;

[0174] The first determining module 12 is used to determine the specific parsing plugin required by the task node based on the plugin configuration information; update the specific parsing plugin to the plugin library to obtain the updated plugin library.

[0175] In some instances, when the first determining module 12 determines the target storage component corresponding to the task node based on the persistent configuration file, the first determining module 12 is used to perform the following steps: determining the storage component required by the task node based on the persistent configuration file; when there is only one storage component, determining the storage component as the target storage component corresponding to the task node; when there are multiple storage components, determining a target storage component corresponding to the task node from among the multiple storage components.

[0176] Figure 7 The device shown can perform Figures 1-5 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figures 1-5 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figures 1-5 The descriptions in the illustrated embodiments will not be repeated here.

[0177] In one possible design, Figure 7 The data persistence device of the cloud computing platform shown can be implemented as an electronic device, which can be a controller, personal computer, server, or other similar devices. Figure 8 As shown, the electronic device may include a first processor 21 and a first memory 22. The first memory 22 is used to store data executed by the corresponding electronic device. Figures 1-5 In the illustrated embodiment, the data persistence method of the cloud computing platform is provided in a program where the first processor 21 is configured to execute the program stored in the first memory 22.

[0178] The program includes one or more computer instructions, wherein when executed by the first processor 21, the one or more computer instructions can perform the following steps: obtaining a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple task nodes to be detected, the persistent configuration file including: detection information of the multiple task nodes to be detected, plugin information and storage information corresponding to each of the multiple task nodes to be detected; detecting the multiple task nodes to be detected according to the detection information; when a change in the state of a task node is detected among the multiple task nodes to be detected, determining the target parsing plugin and target storage component corresponding to the task node based on the persistent configuration file; parsing the task node using the target parsing plugin to obtain parsed data of the task node; and persisting the parsed data using the target storage component.

[0179] Furthermore, the first processor 21 is also used to perform the aforementioned Figures 1-5 All or part of the steps in the illustrated embodiments.

[0180] The structure of the electronic device may also include a first communication interface 23 for communication between the electronic device and other devices or communication networks.

[0181] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by an electronic device, which includes instructions for executing the above-described... Figures 1-5 The program involved in the data persistence method of the cloud computing platform in the illustrated embodiment.

[0182] Furthermore, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the aforementioned... Figures 1-5 The steps in the data persistence method of the cloud computing platform shown in the method embodiment are illustrated.

[0183] Figure 9 This is a schematic diagram of the structure of an image data persistence device provided in an embodiment of the present invention; see attached diagram. Figure 9 As shown, this embodiment provides an image data persistence device, which is used to perform the above-described... Figure 6 The image data persistence method shown may include:

[0184] The second acquisition module 31 is used to acquire a persistent configuration file for a cloud computing platform. The cloud computing platform includes multiple image processing nodes to be detected. The persistent configuration file includes: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected.

[0185] The second processing module 33 is used to detect the plurality of image processing nodes to be detected based on the detection information;

[0186] The second determining module 32 is used to determine the target parsing plugin and target storage table corresponding to the image processing node based on the persistent configuration file when the state of an image processing node among multiple image processing nodes to be detected changes.

[0187] The second processing module 33 is used to parse the image processing node using the target parsing plugin to obtain the image parsing data of the image processing node;

[0188] The second persistence module 34 is also used to persist the image parsing data using the target storage table.

[0189] Figure 9 The device shown can perform Figure 6For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 6 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 6 The descriptions in the illustrated embodiments will not be repeated here.

[0190] In one possible design, Figure 9 The structure of the image data persistence device shown can be implemented as an electronic device, which can be a controller, personal computer, server, or other similar devices. Figure 10 As shown, the electronic device may include a second processor 41 and a second memory 42. The second memory 42 is used to store data executed by the corresponding electronic device. Figure 6 In the illustrated embodiment, the program for the image data persistence method is provided, and the second processor 41 is configured to execute the program stored in the second memory 42.

[0191] The program includes one or more computer instructions, wherein when executed by the second processor 41, the one or more computer instructions can perform the following steps: obtaining a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple image processing nodes to be detected, the persistent configuration file including: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected; detecting the multiple image processing nodes to be detected according to the detection information; when a change in the state of an image processing node is detected among the multiple image processing nodes to be detected, determining the target parsing plugin and target storage table corresponding to the image processing node based on the persistent configuration file; parsing the image processing node using the target parsing plugin to obtain image parsing data of the image processing node; and persisting the image parsing data using the target storage table.

[0192] Furthermore, the second processor 41 is also used to perform the aforementioned... Figure 6 All or part of the steps in the illustrated embodiments.

[0193] The structure of the electronic device may also include a second communication interface 43 for communication between the electronic device and other devices or communication networks.

[0194] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by an electronic device, which includes instructions for executing the above-described... Figure 6 The procedure involved in the image data persistence method in the illustrated embodiment.

[0195] Furthermore, embodiments of the present invention provide a computer program product, comprising: a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause one or more processors to perform the aforementioned... Figure 6 The steps in the image data persistence method shown in the embodiment are illustrated.

[0196] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0197] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0198] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0200] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0201] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data persistence method for a cloud computing platform, characterized in that, include: Obtain a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple task nodes to be detected, the persistent configuration file including: detection information of the multiple task nodes to be detected, plugin information and storage information corresponding to each of the multiple task nodes to be detected; Based on the detection information, the plurality of task nodes to be detected are detected; When a change in the state of a task node is detected among the plurality of task nodes to be detected, the target parsing plugin and target storage component corresponding to the task node are determined based on the persistent configuration file. The task node is parsed using the target parsing plugin to obtain the parsed data of the task node; The parsed data is persisted using the target storage component.

2. The method according to claim 1, characterized in that, Obtain persistent configuration files for the cloud computing platform, including: Based on the node configuration operation input to the cloud computing platform, the plurality of task nodes to be detected are obtained; Based on the plugin configuration operations input for the multiple task nodes to be detected, the plugin information corresponding to each task node is obtained; Based on the storage configuration operations input to the multiple task nodes to be detected, the storage information corresponding to each task node is obtained; Based on the multiple task nodes to be detected, the plugin information corresponding to each task node, and the storage information, the persistent configuration file is generated.

3. The method according to claim 2, characterized in that, Based on the node configuration operations input to the cloud computing platform, the plurality of task nodes to be detected are obtained, including: Based on the node configuration operation input to the cloud computing platform, the node type of each task node in the cloud computing platform is obtained; Based on the node type of each task node, the plurality of task nodes to be detected are determined.

4. The method according to claim 2, characterized in that, Based on the plugin configuration operations input for the multiple task nodes to be detected, the plugin information corresponding to each task node is obtained, including: Obtain a plugin library, which includes multiple parsing plugins for implementing data parsing operations; Among the multiple parsing plugins provided in the plugin library, plugin information corresponding to each task node is obtained based on the plugin configuration operation.

5. The method according to any one of claims 1-4, characterized in that, After obtaining the persistent configuration file for the cloud computing platform, the method further includes: The multiple task nodes are detected based on the persistent configuration file. When the status change information of the task node is obtained, it is determined that the status of the task node has changed; If no status change information for the task node is obtained, it is determined that the status of the task node has not changed.

6. The method according to claim 5, characterized in that, The detection of the multiple task nodes is performed based on the persistent configuration file, including: Obtain the detection priority of each of the multiple task nodes; The multiple task nodes are detected based on the persistent configuration file and the detection priority.

7. The method according to claim 5, characterized in that, After obtaining the state change information of the task node, the method further includes: The state change information is parsed to obtain the parsing results; Based on the analysis results, determine the operation instructions corresponding to the task node; Based on the operation instructions, perform the data processing operation corresponding to the task node.

8. The method according to any one of claims 1-4, characterized in that, Based on the persistent configuration file, the target parsing plugin corresponding to the task node is determined, including: Based on the persistent configuration file, at least one plugin identifier corresponding to the task node is determined; In a preset plugin library, at least one parsing plugin corresponding to the at least one plugin identifier is identified as the target parsing plugin, wherein the plugin library includes multiple parsing plugins for implementing data parsing operations.

9. The method according to claim 8, characterized in that, The method further includes: If the specific parsing plugin required by the task node is not found in the plugin library, obtain the plugin configuration information; Based on the plugin configuration information, determine the specific parsing plugin required by the task node; Update the specific parsing plugin to the plugin library to obtain the updated plugin library.

10. The method according to any one of claims 1-4, characterized in that, Based on the persistent configuration file, the target storage component corresponding to the task node is determined, including: Based on the persistent configuration file, determine the storage components required by the task node; When the number of storage components is one, the storage component is determined as the target storage component corresponding to the task node; When there are multiple storage components, a target storage component corresponding to the task node is determined from among the multiple storage components.

11. A method for persisting image data, characterized in that, include: Obtain a persistent configuration file for a cloud computing platform, which includes multiple image processing nodes to be detected. The persistent configuration file includes: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected. Based on the detection information, the plurality of image processing nodes to be detected are detected; When it is detected that the state of one of the plurality of image processing nodes to be detected has changed, the target parsing plugin and target storage table corresponding to the image processing node are determined based on the persistent configuration file. The image processing node is parsed using the target parsing plugin to obtain the image parsing data of the image processing node; The image parsing data is persisted using the target storage table.

12. A data persistence device for a cloud computing platform, characterized in that, include: The first acquisition module is used to acquire a persistent configuration file for a cloud computing platform, wherein the cloud computing platform includes multiple task nodes to be detected, and the persistent configuration file includes: detection information of the multiple task nodes to be detected, plugin information and storage information corresponding to each of the multiple task nodes to be detected; The first processing module is used to detect the plurality of task nodes to be detected based on the detection information; The first determining module is used to determine the target parsing plugin and target storage component corresponding to the task node based on the persistent configuration file when it is detected that the state of the task node among the plurality of task nodes to be detected has changed. The first processing module is further configured to use the target parsing plugin to parse the task node and obtain the parsed data of the task node; The first persistence module is used to persist the parsed data using the target storage component.

13. A device for persistent image data, characterized in that, include: The second acquisition module is used to acquire a persistent configuration file for a cloud computing platform, the cloud computing platform including multiple image processing nodes to be detected, and the persistent configuration file including: detection information of the multiple image processing nodes to be detected, and plugin information and storage table information corresponding to each of the multiple image processing nodes to be detected; The second processing module is used to detect the plurality of image processing nodes to be detected based on the detection information; The second determining module is used to determine the target parsing plugin and target storage table corresponding to the image processing node based on the persistent configuration file when it is detected that the state of the image processing node among the plurality of image processing nodes to be detected has changed. The second processing module is further configured to use the target parsing plugin to parse the image processing node and obtain the image parsing data of the image processing node; The second persistence module is used to persist the image parsing data using the target storage table.

14. An electronic device, characterized in that, include: A memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method of any one of claims 1-11.

15. A computer storage medium, characterized in that, Used to store a computer program that, when executed by a computer, implements the method of any one of claims 1-11.

16. A computer program product, characterized in that, include: A computer program, when executed by a processor of an electronic device, causes the processor to perform the steps of the method of any one of claims 1-11.

Images