Data lake task management method, device, equipment, storage medium and product

Abstract

This application provides a data inbound task management method, apparatus, device, storage medium, and product. The method includes: in response to the initiation of a data inbound task for a preset data lake, monitoring the target logs of the data source of the data inbound task to determine whether a table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification; when it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation, invoking a metadata management service to perform the target table structure modification operation on the original data lake table corresponding to the data inbound task; after the target table structure modification operation is completed, starting a new data inbound task and generating a new data lake table corresponding to the new data inbound task.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, storage medium and product for managing data import tasks. Background Technology

[0002] With the continuous increase in data volume and the diversification of data types, more and more customers are adopting data lake architectures for data storage. A data lake is a centralized repository that stores data in its native format (also known as natural format). When using a data lake to store data, users do not need to perform structuring on the data, and it supports users to store structured, unstructured, or semi-structured data at any scale. Currently, during the data ingestion task, when the table structure of the source table in the data source changes, there are problems such as the inability to adapt to changes in the source table structure and poor reusability. Summary of the Invention

[0003] This application provides a data inflow task management method, apparatus, device, storage medium, and product that can solve the problems of inability to adapt to changes in source table structure and poor reusability in related technologies.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a data inflow task management method, the method including: In response to the initiation of a data ingestion task for a preset data lake, the target logs of the data source for the data ingestion task are monitored to determine whether the table structure modification operation of the source table for the data source is a target table structure modification operation that the preset data lake does not support adaptive modification. When it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation, the metadata management service is invoked to perform the target table structure modification operation on the original data lake table corresponding to the data inflow task. After the target table structure modification operation is completed, a new data inflow task is started to generate a new data lake table corresponding to the new data inflow task.

[0005] In some exemplary embodiments, the target table structure modification operation includes: deleting a column, renaming the table, or modifying the column type that does not meet the preset type conversion conditions, wherein the preset type conversion conditions are used to indicate the range of type conversions that the preset data lake supports for adaptive modification.

[0006] In some exemplary embodiments, the step of monitoring the target logs of the data source for the data inflow task to determine whether a table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification includes: Monitor whether a preset statement exists in the target log; wherein, the preset statement is a statement used to indicate a table structure modification operation performed on the source table of the data source; When the preset statement is detected in the target log, the target log is parsed to determine whether the modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification.

[0007] In some exemplary embodiments, performing the target table structure modification operation on the original data lake table corresponding to the data inflow task includes: The target table structure modification operation is performed on the original data lake table corresponding to the data inflow task to obtain the modified data lake table; Before the new data lake table is fully generated, queries are performed using the modified data lake table; After the new data lake table is generated, the metadata management service is invoked to delete the modified data lake table, and the new data lake table is used for querying.

[0008] In some exemplary embodiments, each data inflow task runs using Kubernetes application mode; the method further includes: By using Kubernetes' listening mechanism, the behavior status of the deployment object corresponding to each data inflow task is determined. The status of each data ingestion task is determined based on the behavior of the deployment object corresponding to each data ingestion task.

[0009] In some exemplary embodiments, determining the state of each data inbound task based on the behavioral state of the deployment object corresponding to each data inbound task includes: For each data ingestion task, the status of the data ingestion task is determined based on the behavior status of the deployment object corresponding to the data ingestion task, the status of the container group managed by the deployment object, and the task status in the job manager of the computing engine obtained by calling the monitoring tool.

[0010] This application embodiment provides a data inflow task management device, the device comprising: The response unit is used to respond to the start of a data inbound task for a preset data lake by monitoring the target log of the data inbound task's data source to determine whether the table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification. An execution unit is used to call the metadata management service to perform the target table structure modification operation on the original data lake table corresponding to the data lake task when it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation. The generation unit is used to start a new data lake task after the target table structure modification operation is completed, and generate a new data lake table corresponding to the new data lake task.

[0011] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data inflow task management method provided in the embodiments of this application.

[0012] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, the data inflow task management method provided in this application is implemented.

[0013] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the data inflow task management method provided in this application.

[0014] The embodiments of this application have the following beneficial effects: This application provides a data ingestion task management method, apparatus, device, storage medium, and product. In response to the initiation of a data ingestion task, it monitors the target logs of the data source corresponding to the data ingestion task to identify target table structure modification operations in the source table of the data source that are not supported by the preset data lake for adaptive modification. Then, for target table structure modification operations in the source table of the data source that are not supported by the preset data lake for adaptive modification, it calls the metadata management service to perform the target table structure modification operation on the original data lake table corresponding to the data ingestion task. This enables the synchronization of table structure changes in the source table of the data source to the data lake. Thus, it achieves automatic synchronization of table structure modifications that are not supported by the preset data lake for adaptive modification to the data lake, thereby solving the problems of related solutions being unable to adapt to changes in source table structure and having poor reusability. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the first process of a data inflow task management method provided in an embodiment of this application; Figure 2This is a schematic diagram of the second process of a data inflow task management method provided in an embodiment of this application; Figure 3 A flowchart illustrating an application embodiment of the data inflow task management method provided in this application; Figure 4 A schematic diagram of a data inflow task management device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0016] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, references are made to "some exemplary embodiments," which describe a subset of all possible embodiments. However, it is understood that "some exemplary embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0019] In the description of this application, the terms "first," "second," etc., are used merely to distinguish similar objects and do not represent a specific order of objects, nor should they be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated or described herein.

[0020] In the description of this application, "multiple" means two or more. Terms such as "one or more," "one or more items," or "one or more items" indicate any one, any two, or more than two of a plurality. For example, including one or more of A, B, and C can mean including any one, any two, or more elements selected from the set consisting of A, B, and C.

[0021] In the description of this application, the terms "module," "unit," or "component" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented, wholly or partially, using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of a larger module or unit that includes the functionality of that module or unit.

[0022] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0023] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0024] To facilitate a better understanding of the technical solutions of this application by those skilled in the art, the nouns and terms involved in the embodiments of this application are explained below, and the nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0025] 1. Data Lake: A data lake is a centralized repository or storage system that stores data in its native format (also known as natural format). When using a data lake to store data, it supports storing structured, unstructured, or semi-structured data of any size. This includes copies of raw data from the original system and transformed data generated for various tasks. This includes structured data (rows and columns) from relational databases, semi-structured data (such as comma-separated values ​​(CSV), logs, Extensible Markup Language (XML), JavaScript Object Notation (JSON), etc.), and unstructured data (such as emails, documents, Portable Document Format (PDF), images, audio, and video). Data lake integration refers to the collection, integration, and storage of data from various sources into a centralized data lake.

[0026] 2. Apache Paimon is a streaming data lake storage technology that provides users with high-throughput, low-latency data ingestion, streaming subscription, and real-time query capabilities. It supports batch writes and reads, as well as streaming writes and reads. It offers advantages such as low cost, high reliability, and scalable metadata.

[0027] 3. Kubernetes (K8S) is an open-source container orchestration, scheduling, and management tool, considered the operating system of the cloud-native era. A cluster implemented using K8S is also called a K8S cluster. A POD (Container Group) is the smallest unit that can be created and managed in Kubernetes. A POD can include one or more containers, which implement corresponding business logic. A container can be understood as a microservice application. A Deployment Object is a commonly used cluster-oriented object in Kubernetes for deploying and managing stateless applications. A Deployment is a high-level object used for declaratively managing applications and is a commonly used fundamental component in Kubernetes. It is responsible for creating and managing a set of replicas of Pods, thereby enabling application version control, rolling updates, rollback, and other functionalities.

[0028] 4. Apache Flink is a big data computing engine that combines batch and stream processing. Its core is a stream data processing engine that provides data distribution and parallel computation, characterized by high throughput, high performance, low latency, and high scalability. The Flink runtime architecture mainly consists of four different components: JobManager, ResourceManager, TaskManager, and Dispatcher, which work together when running stream processing applications. The JobManager is the core control component of Flink, responsible for the global management and coordination of submitted stream processing jobs, managing all TaskManagers, and deciding which TaskManagers execute user tasks. A job is a series of program instructions executed sequentially by the computing engine, such as executing query statements to retrieve data. One deployment method for Flink is based on Kubernetes. In this method, Kubernetes features are utilized to containerize Flink's JobManager and TaskManager components. Multiple data sources are connected to Flink through the Flink Connector for real-time data acquisition. Flink processes data from multiple data sources in real time, performing data cleaning, format conversion, and filtering, and storing the processed data in Paimon.

[0029] 5. Metadata, also known as intermediary data or relay data, is data about data, primarily describing data properties. It supports functions such as indicating storage location, historical data, resource lookup, and file records. Metadata management (MetaStore) services refer to services that manage metadata, such as HiveMetastore (HMS). HMS is the de facto standard for metadata management, capable of supporting integrated stream and batch architectures, and is a component that can manage metadata for real-time data.

[0030] With the development of big data lake warehouse technology, data ingestion has become a crucial step for enterprises in building big data platforms. Data ingestion refers to the collection, integration, and storage of data from various sources into a centralized data lake. Through data ingestion, enterprises can break down data silos, achieve data interconnection and sharing, and extract valuable information from massive amounts of data to provide data support for business decisions and strategic formulation. Current data ingestion solutions require data exploration of the source data before ingestion, first obtaining the source data's metadata, then constructing a data resource catalog based on the collected metadata and creating data lake tables (such as Hudi tables) with a structure consistent with the source tables. During the data ingestion process, if the table structure of the source tables changes, the data ingestion task cannot adaptively modify the target tables in the data lake, causing the data ingestion task to fail. Therefore, current data ingestion solutions suffer from problems such as inability to adapt to changes in source table structure and poor reusability.

[0031] In this embodiment, the data ingestion task management method can be applied to a data management system, which may include: multiple data sources, a computing engine, and a data lake. The computing engine is used to connect multiple data sources to the computing engine via connectors and perform real-time data acquisition; it also processes the data acquired from the multiple data sources in real-time and stores the acquired data in the data lake.

[0032] In some exemplary embodiments, the data lake can be built using Paimon, and the computing engine can be Flink. Thus, data ingestion tasks can be implemented based on the officially provided paimon-flink-action jar package. The PaimonAction Jar is a command-line tool framework provided by Paimon for table maintenance operations, and the paimon-flink-action jar package is a module within the Paimon Action Jar. Furthermore, considering that the paimon-flink-action jar package currently does not support data integration with SQL Server and TiDB, when the default data source includes SQL Server and TiDB, the Paimon source code can be modified to implement data integration capabilities for SQL Server and TiDB based on Flink CDC. Flink CDC is a streaming-based data integration tool that uses Change Data Capture (CDC) technology based on database logs (such as Binlog or redolog) to achieve unified incremental and full data reading. Flink CDC can be used as a data synchronization tool to connect to multiple data sources. Combined with Flink's excellent pipeline capabilities and rich upstream and downstream ecosystem, Flink CDC can efficiently achieve real-time integration of massive amounts of data.

[0033] This application provides a data ingestion task management method, apparatus, device, storage medium, and product. In response to the initiation of a data ingestion task, it monitors the target logs of the data source corresponding to the data ingestion task to identify target table structure modification operations in the source table of the data source that are not adaptively modified by the preset data lake. Then, for target table structure modification operations in the source table of the data source that are not adaptively modified by the preset data lake, it calls the metadata management service to execute the target table structure modification operation on the original data lake table corresponding to the data ingestion task. This enables the synchronization of table structure changes in the source table of the data source to the data lake. Thus, it automatically synchronizes table structure modifications that are not adaptively modified by the preset data lake to the data lake, thereby solving the problems of related solutions being unable to adapt to changes in the source table structure and having poor reusability during the operation of the data ingestion task.

[0034] In some exemplary embodiments, this data ingestion task management method can be applied to various scenarios in the big data field that require the management of data ingestion tasks, such as the scenario of managing multi-source heterogeneous data ingestion tasks using Zero-ETL (Zero Extract-Transform-Load). Here, Zero-ETL refers to the ability to directly and one-stop complete data synchronization and management without requiring complex ETL processing of the data.

[0035] In some exemplary embodiments, this data ingestion task management method can be applied to a data management system, including: a preset data source, a preset computing engine, and a preset data lake connected to the preset computing engine. As an example, the preset computing engine can be a big data computing engine, such as Flink. As an example, the preset data lake can be a streaming data lake warehouse, such as Paimon. As an example, the preset data source can include one or more of the following: MySQL, SQL Server, TiDB, MongoDB, PostgreSQL, etc.

[0036] Figure 1 This is a schematic diagram of the first process of a data inflow task management method provided in an embodiment of this application. The following will be combined with... Figure 1 This will be explained further. As an example, the execution entity for this data ingestion task management method can be a computing engine, such as Flink (e.g., a Flink deployment based on Kubernetes). Figure 1 As shown, the data inflow task management method may include the following steps S101 to S103: S101, in response to the start of the data inbound task for the preset data lake, by monitoring the target log of the data source of the data inbound task, determine whether the table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification. S102, when it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation, the metadata management service is called to perform the target table structure modification operation on the original data lake table corresponding to the data lake task. S103, after the target table structure modification operation is completed, start a new data lake task and generate a new data lake table corresponding to the new data lake task.

[0037] The "data ingestion task for the preset data lake" refers to the process of starting a task that stores data from the data source connected to the preset data lake through a preset computing engine.

[0038] As an example, taking a data lake using Paimon as an example, the table format of the original data lake table and the new data lake table can be Paimon table. The table format defines how the data and metadata are organized, providing a unified semantic meaning for the "table" above.

[0039] In some exemplary embodiments, table structure modification operations for the source table of the data source may include, but are not limited to, one or more of the following: adding table parameters, modifying table parameters (such as SET operation); renaming the table (such as RENAME operation); adding columns (such as ADD operation); renaming columns (such as RENAME operation); deleting columns (such as DROP operation); deleting partitions (such as DROP operation); modifying column comments; modifying column order (such as MODIFY operation); modifying column type (such as specific type change, i.e., modification upwards).

[0040] The provision that the preset data lake does not support adaptive modification of the target table structure refers to specific pattern changes that may affect the storage and processing of data in the preset data lake. This is related to the data lake framework used to construct the preset data lake and can be set by those skilled in the art based on the data lake framework used to construct the data lake. Here, this application embodiment does not limit this.

[0041] In some exemplary embodiments, the target table structure modification operations that the preset data lake does not support adaptive modification may include, but are not limited to: deleting columns, renaming tables, and modifying column types that do not meet preset type conversion conditions. The preset type conversion conditions are used to indicate the range of type conversions that the preset data lake supports for adaptive modification.

[0042] As an example, taking Paimon as an example of a data lake, the modified column type that meets the preset type conversion conditions can include, but is not limited to, one or more of the following: a shorter string type is modified to a longer string type (e.g., varchar is modified to text); a shorter binary type is modified to a longer binary type (e.g., binary is modified to blob); a shorter integer type is modified to a broader integer type (e.g., int is modified to bigint); a floating-point type with a smaller range is modified to a floating-point type with a broader range (e.g., float is modified to double), etc.

[0043] As an example, taking Paimon as an example of a data lake, the column type that does not meet the preset type conversion conditions can be modified to include, but is not limited to, one or more of the following: changing the string type to the binary type (e.g., changing varchar to binary); changing the longer string type to the shorter string type (e.g., changing text to varchar); changing the broader integer type to the shorter integer type (e.g., changing bigint to int), etc.

[0044] In some exemplary embodiments, the target table structure modification operation that the preset data lake does not support adaptive modification can refer to other table structure modification operations that may affect the storage and processing of data in the preset data lake, besides those table structure modification operations that the preset data lake supports adaptive modification. As an example, table structure modification operations that the preset data lake supports adaptive modification may include, but are not limited to: adding columns, renaming columns, and modifying column types that meet preset type conversion conditions (also known as modifying column types upwards). Therefore, the target table structure modification operation that the preset data lake does not support adaptive modification can refer to other table structure modification operations besides adding columns, renaming columns, and modifying column types that meet preset type conversion conditions (also known as modifying column types upwards), such as deleting columns, renaming tables, and modifying column types that do not meet preset type conversion conditions.

[0045] In some exemplary embodiments, invoking the metadata management service can be achieved by calling the Hive Metastore interface. Of course, other metadata management services can also be used, which can be determined by those skilled in the art based on the framework adopted by the preset computing engine and the preset data lake.

[0046] In some exemplary embodiments, the target log can be a binary log (Binlog). A binary log is a database log file used to record data modification operations in the database, including insert, update, and delete operations. Specifically, insert indicates adding a new row to a table in the database. Update indicates updating an existing row in a table in the database. Delete indicates deleting an existing row from a table in the database.

[0047] Thus, in this embodiment, in response to the start of the data lake ingestion task, the target logs of the data source corresponding to the data lake ingestion task are monitored to identify target table structure modification operations in the source table of the data source that the preset data lake does not support adaptive modification. Then, for target table structure modification operations in the source table of the data source that the preset data lake cannot adaptively modify, the target table structure modification operation is performed on the original data lake table corresponding to the data lake ingestion task by calling the metadata management service. This can realize the synchronization of the table structure changes of the source table in the data source to the data lake. In this way, the table structure modifications that the preset data lake cannot adaptively modify are automatically synchronized to the data lake, thereby solving the problem that related solutions cannot adapt to changes in the source table structure and have poor reusability during the operation of the data lake ingestion task. Therefore, when the solution of this embodiment is applied to build a preset data lake based on Paimon, the schema evolution function of Paimon can be extended to solve the problem that Paimon cannot adapt to changes in the source table structure and has poor reusability.

[0048] As an example, since the Rename operation is not an atomic operation, when the target table structure modification operation is to rename the table, a metadata management service (such as Hive Metastore (HMS)) can be called to perform the Rename operation on the original data lake table corresponding to the data lake task. In this way, the target table structure modification operation is performed on the original data lake table corresponding to the data lake task.

[0049] In some exemplary embodiments, S102, when it is determined that the table structure modification operation for the source table of the data source is a target table structure modification operation, the metadata management service is invoked to perform the target table structure modification operation on the original data lake table corresponding to the data lake entry task. This may include: S1021, when the table structure modification operation for the source table of the data source is any one of the following: deleting a column, renaming the table, or modifying a column type that does not meet the preset type conversion conditions, it is determined that the table structure modification operation for the source table of the data source is a target table structure modification operation; the metadata management service is called to perform the target table structure modification operation on the original data lake table corresponding to the data ingestion task. The preset type conversion conditions are used to indicate the range of type conversions that the preset data lake supports for adaptive modification. As an example, taking Paimon as the data lake, the modified column type that does not meet the preset type conversion conditions can include, but is not limited to, one or more of the following: changing a string type to a binary type (e.g., changing varchar to binary); changing a longer string type to a shorter string type (e.g., changing text to varchar); changing a broader integer type to a shorter integer type (e.g., changing bigint to int), etc.

[0050] In some other exemplary embodiments, S102, when it is determined that the table structure modification operation for the source table of the data source is a target table structure modification operation, the metadata management service is invoked to perform the target table structure modification operation on the original data lake table corresponding to the data lake entry task, which may include: S1022, when the table structure modification operation for the source table of the data source is not adding columns, renaming columns, or modifying column types that meet the preset type conversion conditions, it is determined that the table structure modification operation for the source table of the data source is a target table structure modification operation; the metadata management service is called to perform the target table structure modification operation on the original data lake table corresponding to the data ingestion task. The preset type conversion conditions are used to indicate the range of type conversions that the preset data lake supports for adaptive modification. As an example, taking Paimon as an example of a data lake, the modified column types that meet the preset type conversion conditions can include, but are not limited to, one or more of the following: modifying a shorter string type to a longer string type (e.g., varchar to text); modifying a shorter binary type to a longer binary type (e.g., binary to blob); modifying a shorter integer type to a broader integer type (e.g., int to bigint); modifying a smaller floating-point type to a broader floating-point type (e.g., float to double), etc.

[0051] In some exemplary embodiments, S101, in response to the initiation of a data ingestion task for a preset data lake, by monitoring the target logs of the data source for the data ingestion task, determining whether a table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification, may include: S1011, Monitor whether there is a preset statement in the target log; where the preset statement is a statement used to indicate the table structure modification operation performed on the source table of the data source; S1012, when a preset statement is detected in the target log, the target log is parsed to determine whether the table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification.

[0052] In some exemplary embodiments, the preset statement can be a statement based on a preset query language. For example, the preset query language can be Structured Query Language (SQL), and the preset statement can be a statement in SQL used to indicate modifications to the table structure. For instance, taking SQL as the preset query language, the preset statement could be the `alter table` statement in SQL. The `alter table` statement is a standard SQL command used to modify the structure of an existing table.

[0053] As an example, let's consider a modification operation on the source table of the data source as "alter table t_1 RENAME to t_2", which changes the name of the source table from "t_1" to "t_2". Executing S1011 and monitoring the target log confirms the presence of the predefined statement "alter table". Then, executing S1012 and parsing the target log reveals that the table structure modification operation on the source table of the data source is a table renaming operation (such as a RENAME operation). Renaming a table is a target table structure modification operation that the data lake pre-supports for adaptive modification. Therefore, it can be determined that the table structure modification operation on the source table of the data source is a target table structure modification operation.

[0054] In some other exemplary embodiments, S101, in response to the initiation of a data ingestion task for a preset data lake, by monitoring the target logs of the data source for the data ingestion task, determining whether a table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification, may include: S1013, Monitor whether there is a preset statement in the target log; where the preset statement is a statement used to indicate the table structure modification operation performed on the source table of the data source; S1014 When a preset statement is detected in the target log, the target log is parsed to determine whether the table structure modification operation of the source table of the data source is another table structure modification operation that the preset data lake supports adaptive modification. S1015, when the table structure modification operation of the source table of the data source is a table structure modification operation of another table, it can be determined that the table structure modification operation of the source table of the data source is not a target table structure modification operation; or, when the table structure modification operation of the source table of the data source is not a table structure modification operation of another table, it can be determined that the table structure modification operation of the source table of the data source is a target table structure modification operation.

[0055] As an example, let's take the modification operation on the source table of the data source as "alter table t_1 RENAME to t_2", which means changing the name of the source table of the data source from "t_1" to "t_2". At this time, executing S1013, by monitoring the target log, we can determine that the preset statement "alter table" exists in the target log; then, executing S1014, by parsing the target log, we can determine that the table structure modification operation on the source table of the data source is a table renaming operation (such as a RENAME operation). Renaming a table is not another table structure modification operation that the preset data lake supports adaptive modification (such as adding columns, renaming columns, and modifying column types that meet preset type conversion conditions). Therefore, we can determine that the table structure modification operation on the source table of the data source is a target table structure modification operation.

[0056] In some exemplary embodiments, step S102, modifying the target table structure of the original data lake table corresponding to the data ingestion task, may include: Perform a target table structure modification operation on the original data lake table corresponding to the data ingestion task to obtain the modified data lake table; Before the new data lake table is fully generated, use the modified data lake table for queries; After the new data lake table is generated, call the metadata management service to delete the modified data lake table and then use the new data lake table for querying.

[0057] As an example, if the modification operation on the source table of the data source is "alter table t_1 RENAME to t_2", then it can be determined that the table structure modification operation on the source table of the data source is a table renaming operation (such as the RENAME operation). A table renaming operation is a target table structure modification operation that is not supported by the default data lake for adaptive modification. Therefore, a table renaming operation (such as the RENAME operation) can be performed on the original data lake table corresponding to the data inflow task to obtain the renamed data lake table (i.e., the modified data lake table).

[0058] Therefore, in this embodiment, considering that the generation of a new data lake table (such as a new Paimon table) takes some time after starting a new data lake task, during this period, before the new data lake table is fully generated, queries are performed using the modified data lake table (such as the original data lake table after a Rename operation) until the new data lake table is fully generated. Then, the metadata management service is invoked to delete the modified data lake table, and queries are performed using the new data lake table. This ensures the stability of the data lake task operation.

[0059] In some exemplary embodiments, the pre-defined computing engine (such as Flink) is deployed in Kubernetes (K8S) in application mode, and each data ingestion task runs using K8S application mode. This ensures isolation between data ingestion tasks. However, running data ingestion tasks in K8S application mode makes monitoring task status challenging. Typically, the task status of the pre-defined computing engine (such as Flink) is obtained through the JobManager within the engine. However, in application mode, the JobManager only starts running after the data ingestion task is successfully submitted, and it terminates as soon as the task ends, making it impossible to distinguish whether the task ended abnormally, completed normally, or failed to submit. Therefore, when each data ingestion task runs in K8S application mode, such as... Figure 2 As shown, after S101 or S103, once the data inbound task is initiated, the data inbound task management method provided in this application embodiment may further include: S201 uses the K8S Watcher mechanism to determine the behavior (Action) status of the deployment object corresponding to each data inflow task. S202, based on the behavior (Action) state of the deployment object corresponding to each data inbound task, determine the state of each data inbound task.

[0060] Thus, in this embodiment of the application, by setting each data inbound task to run in Kubernetes Application mode, the isolation between data inbound tasks can be guaranteed; and by using Kubernetes Watcher mechanism to determine the behavior (Action) state of the deployment object corresponding to each data inbound task, the status of each data inbound task can be determined, thereby ensuring the isolation between data inbound tasks while monitoring the task status in real time.

[0061] In some exemplary embodiments, S202, determining the state of each data inbound task based on the behavior state of the deployment object corresponding to each data inbound task may include: A1. When the Action state of the Deployment object corresponding to the data inbound task is DELETED, determine whether there is a user operation for the data inbound task that indicates the task to be paused. A2, when there is a user action for the data inflow task that indicates the task to be paused, determine that the data inflow task is in a paused state. A3. When there is no user action to indicate the pause of the data inflow task for this data inflow task, the status of the data inflow task is determined to be abnormal.

[0062] Therefore, since the Deployment object's Action state is DELETED, it could be due to either a normal pause by the user of the data ingestion task or an anomaly in the data ingestion task causing the Deployment to be deleted. Thus, it's necessary to further determine whether the user paused the data ingestion task. If the user paused the data ingestion task, it can be determined that the data ingestion task is in a paused state; otherwise, it can be determined that the data ingestion task is in an abnormal state.

[0063] In some other exemplary embodiments, S202, determining the state of each data inbound task based on the behavior state of the deployment object corresponding to each data inbound task may include: B1. When the Action status of the Deployment object corresponding to the data ingestion task is ERROR, the data ingestion task is determined to be in an abnormal state. Therefore, when Kubernetes DeploymentWatcher detects an ERROR Action, the data ingestion task must have encountered an anomaly, and thus, it can be directly determined that the data ingestion task is in an abnormal state.

[0064] In some further exemplary embodiments, S202, determining the state of each data inbound task based on the behavior state of the deployment object corresponding to each data inbound task, may include: S2021. For each data inbound task, the status of the data inbound task is determined based on the behavior status of the deployment object corresponding to the data inbound task, the status of the container group managed by the deployment object, and the task status in the job manager of the computing engine obtained by calling the monitoring tool.

[0065] In some exemplary embodiments, in S2021, for each data inbound task, the state of the data inbound task is determined based on the behavior state of the deployment object corresponding to the data inbound task, the state of the container group managed by the deployment object, and the task state in the job manager of the computing engine obtained by calling the monitoring tool. This may include: C1, When the Action state of the Deployment object corresponding to the data inflow task is MODIFIED, determine whether there is a restarted container group (POD) among the one or more container groups (PODs) managed by the Deployment object. C2, when a container group (POD) has restarted, determine whether there is a container group (POD) in the running state. C3: When there is no container group (POD) in the running state, the task state is not modified. Instead, the state of the deployment object corresponding to the data inbound task is monitored through the K8S Watcher mechanism to redetermine the state of the data inbound task. C4, when there are container groups (PODs) in the running state, call monitoring tools (such as Prometheus) to obtain the task status in the compute engine's JobManager; C5 determines the status of the data ingestion task based on the task status in the JobManager of the computing engine.

[0066] In some exemplary embodiments, C5, determining the status of the data inbound task based on the task status in the JobManager of the computing engine, may include: C51, when the task status in the JobManager of the computing engine is RUNNING, determining the status of the data inbound task as normal; or, C52, when the task status in the JobManager of the computing engine is not RUNNING, continuing to monitor the behavior (Action) status of the deployment object corresponding to the data inbound task through the Kubernetes Watcher mechanism, and re-determining the status of the data inbound task.

[0067] In some exemplary embodiments, in C52 and C3, the task status in the JobManager of the computing engine can be obtained through Prometheus to obtain a more detailed status of the data ingestion task.

[0068] Thus, in this embodiment of the application, by setting each data inbound task to run in K8S application mode, the isolation between data inbound tasks can be guaranteed; and by combining K8S's watcher mechanism with monitoring tools (such as Prometheus) to determine the status of each data inbound task, the isolation between data inbound tasks can be guaranteed while the task status can be monitored in real time.

[0069] For ease of understanding, the system architecture of the data inbound task management method applicable to the embodiments of this application will be described below with reference to exemplary application embodiments, and the data inbound task management method of the embodiments of this application will be further described in detail based on the architecture of the communication system.

[0070] This application provides an application embodiment of a data ingestion task management method. This embodiment proposes a data ingestion system, which may include: a data source management module, a data ingestion module, a structure change evolution module, and a task monitoring module. The data source management module manages the connection information of the data source, tests connectivity, and displays the databases / tables that can be ingested from the data source. The data ingestion module performs full incremental ingestion of data from the data source. The structure change evolution module monitors the data source logs (e.g., Binlog), parses the logs (e.g., Binlog), calls the data ingestion module to rerun the task, performs target table structure modification operations (e.g., Rename operations) on the original Paimon table, and deletes the modified original Paimon table after the new Paimon table is generated. The task monitoring module monitors the status of the data ingestion task and its operational metrics.

[0071] In some exemplary embodiments, this application embodiment also proposes a method for ingesting multi-source heterogeneous data into a lake based on Paimon's Zero-ETL, which may include the following steps: D1, Add a data source.

[0072] As an example, D1, adding a data source, can include: selecting the data source type, adding the data source, and entering the data source's connection information (such as address, port number, password, etc.). After entering the connection information, a connectivity test can be performed; the data source can only be successfully added after the connectivity test passes. For example, preset data sources can include: MySQL, SQL Server, TiDB, MongoDB, Postgres SQL, etc. Of course, in addition to the listed exemplary data sources, support for other data sources can be flexibly implemented by developing corresponding data source plugins.

[0073] D2, Select the source database / table.

[0074] As an example, D2, selecting the source database / table, can include: After adding the data source in step D1, a list of databases for the data source can be displayed. For the database, you can choose either database synchronization or table synchronization. Selecting database synchronization uses that database as the source database, while selecting table synchronization requires selecting one or more tables in the database as source data tables. This facilitates subsequent writing of data from the data source to the data lake on a database-by-database basis, or on a table-by-table basis within the database.

[0075] D3, Select the target database / table.

[0076] As an example, D3, selecting the destination database / table, can include: After selecting the source database / table in step D2, the database name of the source database is generally used to create the database in the data lake by default, and the destination database does not need to be selected. If a database with the same name exists in the data lake, the system will prompt the user that a database with the same name exists, so that the user can choose to overwrite the original destination database or change the destination database. When table synchronization is required, after selecting the source database and source table according to step D2, the destination database is selected as needed. If a table with the same name as the source table exists in the destination database, the system will prompt the user that a table with the same name exists, so that the user can choose to overwrite or change the destination database or table.

[0077] D4, Initiate the data import task.

[0078] As an example, after completing steps D1 to D3, all configurations for data import into the data lake are complete. The user can click "Start Task". At this time, the system receives the instruction to start the data import task. In response to the start of the data import task, the data import task can begin and steps S101 to S103 in this application can be executed to synchronize the table structure changes of the source table to the data lake.

[0079] As an example, let's assume the default data lake is Paimon, the data source is MySQL, the target log is Binlog, and the default statement is "alter table". Then, taking the MySQL data ingestion task as an example... Figure 3As shown, S1, the data ingestion task starts; S2, after the data ingestion task starts, the structure change evolution module continuously monitors logs, such as the Binlog; S3, it determines whether a preset statement, such as "alter table", is matched in the Binlog; S4, when "alter table" is matched in the Binlog, the Binlog information is parsed; S5, based on the parsed information, it determines whether the alter table operation on the source table of the data source is a preset adaptive table structure modification operation of the data lake (Paimon) (such as adding a column or modifying a column upwards). S6, taking the alter table operation specifically as a table rename as an example, when the alter table operation is specifically a table rename, it can be determined that the alter table operation is not an adaptive table structure modification operation (such as adding columns or modifying columns upwards). The structure change evolution module will call the Hive Metastore interface to perform a table rename operation on the original Paimon table corresponding to the data lake entry task; S7, after the rename operation on the original Paimon table corresponding to the data lake entry task is completed, the data lake entry module is called to start a new data lake entry task and generate a new Paimon table.

[0080] As an example, after starting a new data ingestion task, it takes some time to generate a new Paimon table. During this period, the renamed Paimon table is used for querying. After the new Paimon table is generated, the structure change evolution module calls the HiveMetastore interface to delete the renamed Paimon table.

[0081] D5, monitoring the status and operational indicators of the data entering the lake.

[0082] As an example, after starting a data ingestion task, users can view metrics such as task status and data ingestion speed on the task monitoring page. The system's underlying computing engine uses Flink. To ensure isolation between data ingestion tasks, the system runs these tasks in Kubernetes Application mode. This makes monitoring task status challenging: typically, Flink task status is obtained through the JobManager, but in Application mode, the JobManager only starts running after a task is successfully submitted, and it also terminates once the task ends. Therefore, it's impossible to distinguish whether a task terminated abnormally, completed normally, or failed to submit. To address this, the system uses a combination of Kubernetes Deployment Watcher and Prometheus to monitor task status. Specifically, after a data ingestion task is submitted (i.e., started), the system starts a Kubernetes Deployment Watcher for each task to monitor its Kubernetes Deployment status. When the Watcher detects that the Kubernetes Deployment object corresponding to a data inbound task is in a DELETED state, it could be due to a user pausing the task normally or an exception causing the Deployment to be deleted. The Watcher then determines if there are any user actions indicating that the data inbound task should be paused. If such user actions exist, the data inbound task is determined to be in a paused state. If no such user actions exist, the data inbound task is determined to be in an abnormal state. When the Watcher detects an ERROR action, the data inbound task is determined to be in an abnormal state. When the Watcher detects that the Action status of a data inbound task is MODIFIED, it determines whether any of the PODs managed by the Deployment object corresponding to the data inbound task have restarted. If a POD has restarted, it indicates that the data inbound task has encountered an error. If no POD has restarted, it further determines whether any POD is in the Running state. If no POD is in the Running state, the task status is not modified and monitoring continues. If a POD is in the Running state, it indicates that the data inbound task is running normally. The Watcher calls the Prometheus interface to obtain the task status in the JobManager. If the task status in the JobManager obtained from Prometheus is RUNNING, it indicates that the data inbound task is running normally. If the task status in the JobManager obtained from Prometheus is not RUNNING, monitoring continues.(Here, when it is necessary to distinguish the task status of the data entry into the lake in more detail, it can be obtained through Prometheus.)

[0083] Thus, in this embodiment, a data lake is built based on Paimon, and data ingestion and multi-source heterogeneous data are achieved with a single "one-click" input using the paimon-flink-actionjar package and Flink CDC, eliminating the need for any ETL (Extract-Transform-Load) operations, simplifying the process. For the same data source, only one data source connection configuration is required to ingest different databases and tables from that data source. During the data ingestion task, by monitoring the data source's Binlog, changes in the source table structure can be identified. For table structure modifications that Paimon cannot adaptively perform (such as deleting columns or renaming tables), automatic updates can be implemented, allowing the data ingestion task to adaptively modify the target data lake table, resulting in good reusability. Flink is deployed on Kubernetes to implement the computing engine, and the data ingestion task is run in Kubernetes Application mode, ensuring isolation between data ingestion tasks. By combining Kubernetes Deployment Watcher and Prometheus to monitor task status, both isolation between data ingestion tasks and monitoring task status can be ensured.

[0084] Embodiments of this application provide a data inflow task management device. Figure 4 This application provides a schematic diagram of the structure of a data inflow task management device, as shown in the embodiments of this application. Figure 4 As shown, the data import task management device may include: The response unit 401 is used to respond to the start of the data inbound task for the preset data lake. By monitoring the target log of the data source of the data inbound task, it determines whether the table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification. Execution unit 402 is used to call the metadata management service to perform target table structure modification operation on the original data lake table corresponding to the data lake task when it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation. The generation unit 403 is used to start a new data lake task after the target table structure modification operation is completed, and generate a new data lake table corresponding to the new data lake task.

[0085] In some exemplary embodiments, when the target table structure modification operation includes any one of: deleting a column, renaming the table, or modifying the column type that does not meet the preset type conversion conditions, the preset type conversion conditions are used to indicate the range of type conversions that the preset data lake supports for adaptive modification.

[0086] In some exemplary embodiments, the response unit 401 is used to monitor whether a preset statement exists in the target log; wherein, the preset statement is a statement used to indicate a table structure modification operation performed on the source table of the data source; when the preset statement is detected in the target log, the response unit 401 is used to parse the target log to determine whether the modification operation on the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification.

[0087] In some exemplary embodiments, the execution unit 402 is used to perform a target table structure modification operation on the original data lake table corresponding to the data lake entry task to obtain a modified data lake table; before the new data lake table is generated, the modified data lake table is used for querying; after the new data lake table is generated, the metadata management service is called to delete the modified data lake table and the new data lake table is used for querying.

[0088] In some exemplary embodiments, each data ingestion task runs using Kubernetes application mode; the device also includes: The first determining unit is used to determine the behavior status of the deployment object corresponding to each data inflow task through the K8S listening mechanism. The second determining unit is used to determine the status of each data inbound task based on the behavior status of the deployment object corresponding to each data inbound task.

[0089] In some exemplary embodiments, the second determining unit is configured to determine the state of each data inbound task based on the behavior state of the deployment object corresponding to the data inbound task, the state of the container group managed by the deployment object, and the task state in the job manager of the computing engine obtained by calling the monitoring tool.

[0090] Embodiments of this application provide an electronic device that may include: Memory is used to store executable instructions or computer programs. When a processor executes computer-executable instructions or computer programs stored in memory, it implements one or more of the data inflow task management methods provided in the embodiments of this application.

[0091] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. This electronic device can be applied to... Figure 1A corresponding embodiment provides a data ingestion task management method. For example... Figure 5 As shown, the electronic device 500 may include a processor 501, a memory 502, and a bus 503. The various components in the electronic device 500 are coupled together via the bus 503. Wherein: Bus 503 is used to realize the communication connection between processor 501 and memory 502; Memory 502 is used to store computer-executable instructions or computer programs; When the processor 501 executes computer-executable instructions or computer programs stored in the memory 502, it implements the data inflow task management method in one or more exemplary embodiments described above.

[0092] In some exemplary embodiments, bus 503 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. For example, bus 503 may be divided into a data bus, a power bus, an address bus, a control bus, a status signal bus, etc. For ease of illustration, in... Figure 5 The bus 503 is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0093] In some exemplary embodiments, the electronic device can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or it can be implemented as a server. For example, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0094] In some exemplary embodiments, the processor may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or any conventional processor, etc.

[0095] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the data ingestion task management method provided in this application can be implemented. For example, ... Figure 1 The data entry task management method is shown.

[0096] This application provides a computer program product, which includes a computer program or computer-executable instructions. When the computer-executable instructions or the computer program are executed by a processor, the data inflow task management method provided in this application can be implemented. For example, ... Figure 1 The illustrated method for managing data inbound tasks is as follows. For example, the computer program or computer-executable instructions are stored in a computer-readable storage medium. The processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium and executes the computer program or computer-executable instructions, causing the electronic device to perform the data inbound task management method provided in the embodiments of this application.

[0097] In some exemplary embodiments, the aforementioned computer-readable storage medium / memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; or it may be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0098] In some embodiments, a computer program or computer-executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0099] As an example, a computer program or computer-executable instructions may, but not necessarily, correspond to a file in a file system. It may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0100] As an example, a computer program or computer-executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected by a communication network.

[0101] It should be noted that the descriptions of the above embodiments of the apparatus, devices, storage media, or products are similar to the descriptions of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device, storage medium, or product embodiments of this application, those skilled in the art should refer to the descriptions of the method embodiments of this disclosure for understanding. Further details will not be repeated here.

[0102] The features disclosed in the several methods, apparatuses, devices, storage media or product embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments, apparatuses, devices, storage media or product embodiments.

[0103] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for managing data import tasks into a lake, characterized in that, The method includes: In response to the initiation of a data ingestion task for a preset data lake, the target logs of the data source for the data ingestion task are monitored to determine whether the table structure modification operation of the source table for the data source is a target table structure modification operation that the preset data lake does not support adaptive modification. When it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation, the metadata management service is invoked to perform the target table structure modification operation on the original data lake table corresponding to the data inflow task. After the target table structure modification operation is completed, a new data inflow task is started to generate a new data lake table corresponding to the new data inflow task.

2. The method according to claim 1, characterized in that, The target table structure modification operation includes: deleting a column, renaming the table, or modifying the column type if it does not meet the preset type conversion conditions. The preset type conversion conditions are used to indicate the range of type conversions that the preset data lake supports for adaptive modification.

3. The method according to claim 1, characterized in that, The step of monitoring the target logs of the data source for the data inflow task to determine whether the table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification includes: Monitor whether a preset statement exists in the target log; wherein, the preset statement is a statement used to indicate a table structure modification operation performed on the source table of the data source; When the preset statement is detected in the target log, the target log is parsed to determine whether the modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification.

4. The method according to claim 1, characterized in that, The step of modifying the target table structure on the original data lake table corresponding to the data inflow task includes: The target table structure modification operation is performed on the original data lake table corresponding to the data inflow task to obtain the modified data lake table; Before the new data lake table is fully generated, queries are performed using the modified data lake table; After the new data lake table is generated, the metadata management service is invoked to delete the modified data lake table, and the new data lake table is used for querying.

5. The method according to any one of claims 1 to 4, characterized in that, Each data ingestion task runs using Kubernetes application mode; the method also includes: By using Kubernetes' listening mechanism, the behavior status of the deployment object corresponding to each data inflow task is determined. The status of each data ingestion task is determined based on the behavior of the deployment object corresponding to each data ingestion task.

6. The method according to claim 5, characterized in that, The process of determining the state of each data inbound task based on the behavior state of the deployment object corresponding to each data inbound task includes: For each data ingestion task, the status of the data ingestion task is determined based on the behavior status of the deployment object corresponding to the data ingestion task, the status of the container group managed by the deployment object, and the task status in the job manager of the computing engine obtained by calling the monitoring tool.

7. A data import task management device, characterized in that, The device includes: The response unit is used to respond to the start of a data inbound task for a preset data lake by monitoring the target log of the data inbound task's data source to determine whether the table structure modification operation of the source table of the data source is a target table structure modification operation that the preset data lake does not support adaptive modification. An execution unit is used to call the metadata management service to perform the target table structure modification operation on the original data lake table corresponding to the data lake task when it is determined that the table structure modification operation of the source table of the data source is the target table structure modification operation. The generation unit is used to start a new data lake task after the target table structure modification operation is completed, and generate a new data lake table corresponding to the new data lake task.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, the method as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, the method as described in any one of claims 1 to 6 is implemented.

Images