A data table management method, apparatus, storage medium, and device

Abstract

This application provides a data table management method, apparatus, storage medium, and device. In this method, a big data platform acquires a target table synchronized by a business system, determines its corresponding mapping table, collects information from the target table, and marks the table name of the mapping table based on the collection results to establish a relationship between the two tables. Furthermore, the big data platform analyzes the relationships between data tables in various business systems through key fields, and then caches the collection and analysis results to a cloud platform, allowing operators to access relevant information from the cloud platform. In this way, operators can quickly obtain the required data without the need for intervention from business-related personnel, effectively saving their time and improving their work efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data table management method, apparatus, storage medium, and device. Background Technology

[0002] In the financial industry, all of an institution's business systems are often connected to a big data platform. Key data tables from each business system are typically synchronized to the big data platform for backup or data cleansing. As the number of clients and business systems increases, the number of tables the big data platform needs to manage also increases exponentially. When relevant personnel need to access the big data platform to process target tables based on business system scenarios, if the data source of that target table is unrelated to them, they need to consult relevant business personnel who will then provide the necessary metadata tables, fields, and other relationships. This approach consumes a significant amount of the business personnel's time and is inefficient. Summary of the Invention

[0003] The purpose of this application is to provide a data table management method, apparatus, storage medium and device, which aims to solve the problem in related technologies that when operators access big data platforms to process tables, it easily takes up a lot of time for business-related personnel and results in low work efficiency.

[0004] Firstly, this application provides a data table management method applied to a big data platform, including:

[0005] Obtain the target table synchronized by the business system, and determine the mapping table corresponding to the target table;

[0006] Information about the target table is collected, and the table names of the mapping table are marked based on the collection results. The association relationships between the target table and each associated table are statistically analyzed using key fields. The marked table names are used to characterize the correspondence between the mapping table and the target table. The associated tables are other data tables that are related to the target table.

[0007] The collected and statistical results are cached on the cloud platform.

[0008] In the above implementation process, the big data platform acquires the target table synchronized by the business system, determines its corresponding mapping table, collects information from the target table, and marks the table name of the mapping table based on the collection results to achieve the association between the two tables. Furthermore, the big data platform analyzes the relationships between data tables in various business systems through key fields, and then caches the collection and analysis results to the cloud platform, allowing operators to access relevant information from the cloud platform. In this way, operators can quickly obtain the required data without the need for intervention from business personnel, effectively saving their time and improving their work efficiency.

[0009] Furthermore, in some embodiments, the acquisition results include at least one of the following:

[0010] The target table's system ID, table structure comments, and filename.

[0011] During the above implementation process, information is collected from the target table, including system ID, table structure comments, and file names, which can identify the system owner and business scenario, to facilitate subsequent tagging and analysis.

[0012] Furthermore, in some embodiments, the key fields include primary keys and foreign keys; the step of statistically analyzing the relationships between the target table and each associated table using the key fields includes:

[0013] If the primary key ID of the target table is a foreign key of the associated table, a master-slave relationship is determined to exist between the target table and the associated table, and the master-slave relationship is used as a statistical result.

[0014] In the above implementation process, a specific method is provided to analyze the relationships between tables through primary keys and foreign keys.

[0015] Furthermore, in some embodiments, the step of statistically analyzing the relationships between the target table and each associated table using key fields further includes:

[0016] Based on the business scenario corresponding to the target table, the target table and the associated table are tagged, and the tagging results are used as statistical results; the tagging results are used to mark the behavioral operations targeting the target table and the associated table.

[0017] In the above implementation process, combined with the scenario, it is possible to perform annotation on the target table and related tables, so that the annotation results can intuitively display the corresponding scenario information.

[0018] Furthermore, in some embodiments, the method further includes:

[0019] The target table is categorized according to preset attributes.

[0020] In the above implementation process, when managing the data tables of various business systems, the big data platform can divide multiple table scenarios according to preset attributes. In this way, when a new data table is received, it is classified into the corresponding table scenario according to the preset attributes of the table, thereby improving the convenience and efficiency of data analysis for other personnel.

[0021] Furthermore, in some embodiments, the preset attribute includes any one of the following:

[0022] Archive time, synchronization time, query statement.

[0023] In the above implementation process, the target table is categorized into the corresponding table scenario according to the time range, so that other personnel can analyze the data as needed, or the target table is categorized into the corresponding table scenario according to the common query statements in the scenario, so as to provide relevant personnel with reference or reuse.

[0024] Furthermore, in some embodiments, the method further includes:

[0025] The collected and statistical results are displayed on an interface, and search terms are provided; the search terms are extracted based on the collected and statistical results.

[0026] In the above implementation process, the big data platform provides the function of displaying the results data cached on the cloud platform, and provides search terms. Operators only need to search according to their own needs to obtain the required data, thus improving the work efficiency of operators.

[0027] Secondly, this application provides a data table management device applied to a big data platform, comprising:

[0028] The determination module is used to obtain the target table synchronized by the business system and determine the mapping table corresponding to the target table.

[0029] The statistics module is used to collect information from the target table, mark the table names of the mapping table based on the collection results, and count the relationships between the target table and each associated table through key fields; wherein, the marked table names are used to characterize the correspondence between the mapping table and the target table; the associated tables are other data tables that are related to the target table;

[0030] The caching module is used to cache the collected and statistical results to the cloud platform.

[0031] Thirdly, this application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method described in any of the first aspects.

[0032] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in any of the first aspects.

[0033] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in any of the first aspects.

[0034] Other features and advantages disclosed in this application will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the above-described technology disclosed in this application.

[0035] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 A flowchart illustrating a data table management method provided in this application embodiment;

[0038] Figure 2 A schematic diagram illustrating the workflow of a distributed system metadata acquisition scheme provided in an embodiment of this application;

[0039] Figure 3 A block diagram of a data table management device provided in an embodiment of this application;

[0040] Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0042] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0043] As described in the background section, related technologies suffer from the problem that when operators access big data platforms to process tables, it easily consumes a significant amount of time for business-related personnel, resulting in low work efficiency. Based on this, embodiments of this application provide a data table management solution to address the aforementioned problems.

[0044] The embodiments of this application will be described below:

[0045] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a data table management method provided in an embodiment of this application. The method is applied to a big data platform. A big data platform is a network platform that provides services through content sharing, resource sharing, channel co-construction, and data sharing. Enterprise systems relying on big data platforms can achieve more precise data management models, while also enabling rapid data sharing. Various dynamic information and real-time data can be accurately shared, preventing information silos between departments. In application, the method can be implemented as middleware integrated into the big data platform, such as an SDK (Software Development Kit) or JAR (Java Archive).

[0046] The method includes:

[0047] In step 101, obtain the target table for synchronization with the business system and determine the mapping table corresponding to the target table;

[0048] In this embodiment, the business system is a system that interfaces with a big data platform and relies on the big data platform to manage its metadata. For example, in the fintech field, a bank's business systems include credit systems, customer service systems, and auto finance systems. These business systems typically synchronize their respective data tables to the big data platform to achieve data backup, data cleansing, and other functions.

[0049] The target table mentioned in this step refers to the data table of the business system. Different business systems may have different target tables. For example, the target tables for a credit card back-end system might include installment payment agreement tables and user change record tables, while the target tables for a customer service system might include visitor statistics tables and after-sales issue record tables. This target table can be provided to the big data platform by the business system via file transfer, or it can be obtained by the big data platform actively accessing the business system's database.

[0050] When various business systems synchronize data tables to the big data platform, the big data platform processes the data tables according to predefined data standards, metrics, dimensions, data specifications, naming conventions, and data quality requirements to obtain a mapping table. Therefore, after obtaining the target table, the middleware determines the mapping table summarized and organized by the big data platform for that target table, facilitating subsequent processing.

[0051] In step 102, information about the target table is collected, and the table names of the mapping table are marked based on the collection results. The association relationships between the target table and each associated table are statistically analyzed using key fields. The marked table names are used to characterize the correspondence between the mapping table and the target table. The associated tables are other data tables that are associated with the target table.

[0052] In related technologies, when personnel access a big data platform to process data tables from other business systems, it often consumes a significant amount of the development time of the developers of those business systems. For example, system A needs to access synchronized data from system B through a big data platform for processing. However, the big data platform maps data tables in system B, which may result in differences in table names and field names. Therefore, operators in system A need to consult the developers of system B, who then access the big data platform to match the relationships and provide the relevant business logic and query statements to the operators in system A. This process consumes the time of the developers in system B and leads to low work efficiency for the operators in system A. In contrast, in this embodiment, middleware collects and statistically analyzes data from business systems and the big data platform, generating collection and statistical results which are then cached on the cloud platform. This allows data operators to view relevant information and understand the background without needing to consult the developers of each business line, thereby improving work efficiency.

[0053] Specifically, in some embodiments, the collection results mentioned in this step may include at least one of the following: the system ID of the target table, comments on the table structure, and filename. That is, when the business system synchronizes the target table to the big data platform, the middleware of the big data platform can collect information about the target table, such as the system ID, comments on the table structure, and filename—information that identifies the system owner and the business scenario—facilitating subsequent tagging and analysis.

[0054] Generally, business systems and big data platforms belong to different departments. Because big data platforms need to process and manipulate data, the corresponding table names often differ. In this embodiment, after information collection, the middleware marks the table names of the mapping table to represent the correspondence between the mapping table and the target table. That is, the purpose of this marking is to associate tables that have the same meaning, the same business, and the same data. These tables can be defined in the form of key-value pairs, where the key is a custom enumerated value, such as 1 representing a business table and 2 representing a configuration table, and the value is the table name of the target table in the business system combined with the table name of the mapping table in the big data platform. This facilitates subsequent association calculations.

[0055] For data tables in various business systems, the middleware performs statistical correlation analysis on key fields between tables. Through these key fields, it analyzes which tables have what kind of relationships. In some embodiments, these key fields may include primary keys and foreign keys. A primary key generally refers to a primary key, which is one or more fields in a table whose value is used to uniquely identify a record in the table. If a common key is a primary key in one relation, then this common key is called a foreign key in another relation. Through primary and foreign keys, relationships can be statistically analyzed, including master-child relationships, dependent relationships, and table partitioning relationships between tables.

[0056] Optionally, in some embodiments, if the primary key ID of the target table is a foreign key of the associated table, a master-slave relationship is determined between the target table and the associated table, and this master-slave relationship is used as a statistical result. For example, the data tables obtained by the big data platform include an installment contract table and a user change record table. If the primary key ID of the installment contract table is used as a foreign key of the user change record table, that is, the two tables can be associated by data fields, then a master-slave relationship can be determined between the two tables, and this master-slave relationship is used as a statistical result. That is, the statistical result records that a master-slave relationship exists between the installment contract table and the user change record table.

[0057] Furthermore, in some embodiments, the step of statistically analyzing the relationships between the target table and related tables using key fields may further include: tagging the target table and related tables according to the business scenario corresponding to the target table, and using the tagging results as statistical results; the tagging results are used to annotate behavioral operations targeting the target table and related tables. In other words, by combining scenarios, behavioral operations targeting the target table and related tables can be annotated. Using the previous example, for the installment contract table and user change record table, the corresponding scenarios are identified as having user installment contract signing operations and any user change record operations related to those installment contracts. Tagging these two tables, the tagging results indicate that user installment contract signing behavior involves the installment contract table and user change record table. Thus, based on the tagging results, all installment contract signing scenario information for the user can be intuitively displayed through these two tables, without the need for intervention from the credit card backend system developers, effectively saving their time.

[0058] Furthermore, the tagging results can be further categorized and extracted for more detailed labeling. Typically, business personnel are unaware of the various backend systems, only understanding the business scenarios. Therefore, by subdividing and tagging scenarios based on the corresponding tables and interfaces of each system, for example, for data tables related to customer account opening and transaction scenarios, the tagging results could include account opening tags, account closing tags, transaction tags, return tags, etc. This allows business personnel to more clearly understand the relationships between the required data and the business systems.

[0059] In step 103, the collected results and statistical results are cached to the cloud platform.

[0060] In this embodiment, the middleware caches the collected data and the analyzed relationships in the cloud platform. This allows data operators to view relevant information from the cloud platform and obtain the data they need.

[0061] Generally, big data encompasses the entire business system, resulting in a large volume of data. Therefore, to further improve the work efficiency of data operators, in some embodiments, the above method may further include: classifying the target table according to preset attributes. That is, when managing data tables from various business systems, the big data platform can divide multiple table scenarios according to preset attributes. Thus, when a new data table is received, it is classified into the corresponding table scenario according to its preset attributes, thereby improving the convenience and efficiency of data analysis for other personnel. Optionally, the preset attributes may include any of the following: archiving time, synchronization time, and query statement. The archiving time may refer to the time when the big data platform summarizes and organizes the target table, and the synchronization time may refer to the time when the business system synchronizes the target table to the big data platform. The archiving time and synchronization time can be a year, month, or specific date, etc., classifying the target table into the corresponding table scenario according to the time range facilitates data analysis by other personnel as needed. The query statement can be obtained by collecting and analyzing relevant query statements in the corresponding scenario. For example, the query statement for the installment contract scenario can include statements representing payment methods, installment periods, etc. When the target table contains content that matches these query statements, the target table is classified into the installment contract scenario. This makes it easier to provide reference or reuse for relevant personnel.

[0062] Furthermore, in some embodiments, the above method may further include: displaying the collected results and statistical results on an interface, and providing search terms; the search terms are extracted based on the collected results and statistical results. In other words, the big data platform provides the function of displaying the results data cached on the cloud platform on an interface, and provides search terms, allowing operators to search according to their specific needs, thus improving operator efficiency.

[0063] In this embodiment, the big data platform acquires the target table synchronized by the business system, determines its corresponding mapping table, collects information from the target table, and marks the table name of the mapping table based on the collection results to achieve the association between the two tables. Furthermore, the big data platform analyzes the relationships between data tables in various business systems through key fields, and then caches the collection and analysis results to the cloud platform, allowing operators to access relevant information from the cloud platform. In this way, operators can quickly obtain the required data without the need for intervention from business-related personnel, effectively saving their time and improving their work efficiency.

[0064] To provide a more detailed explanation of the solution in this application, a specific embodiment is described below:

[0065] This embodiment relates to a metadata management scenario in the fintech field. In the financial industry, big data platforms include metadata software to uniformly manage the metadata of various business systems within an organization. As the number of clients and business systems increases, the number of tables that the big data platform needs to manage also increases exponentially. In practice, operators frequently need to access the big data platform for related processing based on the business system's context, such as data cleaning and report manipulation. In this case, if the data source of the required table is unrelated to the operator, the operator often needs to consult relevant business personnel who then access the big data platform to match and verify relationships before providing the operator with the relevant business logic and query statements. This consumes a significant amount of time for business personnel and also affects the operator's work efficiency. Therefore, this embodiment provides a distributed system metadata acquisition solution to address the aforementioned problems.

[0066] This solution can be implemented as middleware, integrated into a big data platform. This middleware can detect data interactions and scenario collection between business systems and the big data platform. The workflow of this solution is as follows: Figure 2 As shown, it includes:

[0067] S201. Collect information from the data tables of each business system and cache the collected data to the cloud platform;

[0068] Specifically, when various business systems distribute or synchronize data to the big data platform, the middleware collects information from the data tables of the business systems, including system IDs, table structure comments, or file names, as well as other information that can identify the system owner and business scenario.

[0069] S202. Mark the mapping table of the big data platform;

[0070] Specifically, the purpose of this tag is to link tables together to represent tables with the same meaning, the same business, and the same data. It is defined in the form of key-value pairs, where the key is a custom enumeration value, such as 1 representing the business table and 2 representing the configuration table, and the value is a combination of the table name of the original table in the business system and the table name of the mapping table summarized and organized by the big data platform.

[0071] S203. Analyze the relationships between tables through key fields, then label each table according to the scenario, and cache the analysis results to the cloud platform.

[0072] Specifically, the middleware performs key field statistics and correlations on the data tables of various business systems, especially primary keys and foreign keys. Through key fields, it analyzes which tables have what kind of relationship. For related tables, it tags them according to the scenario to mark the behavior operations of the current table and related tables. Furthermore, the middleware can perform vocabulary categorization and extraction for the tagged scenarios and further refine the marking.

[0073] S204. Classify the data tables of each business system;

[0074] Specifically, the information collected earlier can also include archiving time or synchronization time. Based on the time range, such as by year, each table can be categorized into the corresponding scenario, making it easier for other personnel to analyze the data as needed. Furthermore, the relevant query statements of the tables in each scenario are collected and analyzed to identify commonly used related query statements, thereby matching each table and categorizing each table into the corresponding scenario, making it easier to provide reference or reuse for other personnel.

[0075] S205. Display the cached results on the interface and provide search terms;

[0076] Specifically, the middleware automatically compiles relevant system tables based on the relationships and meanings of the tables, as well as the fields, and displays them externally. When operators need to perform data analysis, they only need to enter the corresponding search terms according to the required scenario, and the big data platform can display detailed information such as tables, mapping relationships, and fields for the corresponding scenario.

[0077] This embodiment uses middleware to collect and analyze data from business systems and big data platforms, categorizing and caching the analysis results on a cloud platform. The middleware then displays these cached results and provides relevant search terms, allowing data operators to easily access information and understand the context without needing to consult developers across different business lines. This saves developers' time and improves the efficiency of operators.

[0078] Corresponding to the embodiments of the aforementioned methods, this application also provides embodiments of a data table management device and a terminal for its application:

[0079] like Figure 3 As shown, Figure 3 This is a block diagram of a data table management device provided in an embodiment of this application. The device is applied to a big data platform and includes:

[0080] The determination module 31 is used to obtain the target table synchronized by the business system and determine the mapping table corresponding to the target table.

[0081] The statistics module 32 is used to collect information from the target table, mark the table name of the mapping table based on the collection results, and count the association relationship between the target table and each associated table through key fields; wherein, the marked table name is used to characterize the correspondence between the mapping table and the target table; the associated table is other data table that is associated with the target table;

[0082] The caching module 33 is used to cache the collection results and statistical results to the cloud platform.

[0083] In some embodiments, the above-mentioned collection results include at least one of the following:

[0084] The target table's system ID, table structure comments, and filename.

[0085] In some embodiments, the key fields mentioned above include primary keys and foreign keys; the statistics module 32 includes a statistics submodule, which is used for:

[0086] If the primary key ID of the target table is a foreign key of the associated table, a master-slave relationship is determined to exist between the target table and the associated table, and the master-slave relationship is used as a statistical result.

[0087] In some embodiments, the above-described statistics submodule is further configured to:

[0088] Based on the business scenario corresponding to the target table, the target table and the associated table are tagged, and the tagging results are used as statistical results; the tagging results are used to mark the behavioral operations targeting the target table and the associated table.

[0089] In some embodiments, the above-described apparatus further includes:

[0090] The classification module is used to classify the target table according to preset attributes.

[0091] In some embodiments, the preset attributes mentioned above include any one of the following:

[0092] Archive time, synchronization time, query statement.

[0093] In some embodiments, the above-described apparatus further includes:

[0094] The display module is used to display the collected and statistical results on an interface and provide search terms; the search terms are extracted based on the collected and statistical results.

[0095] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0096] This application also provides an electronic device, please refer to [link to application]. Figure 4 , Figure 4This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device may include a processor 410, a communication interface 420, a memory 430, and at least one communication bus 440. The communication bus 440 is used to enable direct communication between these components. In this embodiment, the communication interface 420 of the electronic device is used for signaling or data communication with other node devices. The processor 410 may be an integrated circuit chip with signal processing capabilities.

[0097] The processor 410 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor, or the processor 410 can be any conventional processor.

[0098] The memory 430 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The memory 430 stores computer-readable instructions. When these computer-readable instructions are executed by the processor 410, the electronic device can perform the aforementioned operations. Figure 1 or Figure 2 The various steps involved in the method implementation examples.

[0099] Alternatively, the electronic device may also include a storage controller and an input / output unit.

[0100] The memory 430, storage controller, processor 410, peripheral interface, and input / output unit are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses 440. The processor 410 is used to execute executable modules stored in the memory 430, such as software function modules or computer programs included in electronic devices.

[0101] The input / output unit is used to provide users with the ability to create tasks and to set optional start periods or preset execution times for those tasks, thereby enabling user-server interaction. The input / output unit may be, but is not limited to, a mouse and keyboard.

[0102] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0103] This application also provides a storage medium storing instructions. When the instructions are run on a computer, the computer program is executed by a processor to implement the method described in the method embodiment. To avoid repetition, the method will not be described again here.

[0104] This application also provides a computer program product that, when run on a computer, causes the computer to perform the method described in the method embodiment.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0106] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0107] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0109] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0110] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A data table management method, characterized in that, Applications in big data platforms include: Obtain the target table synchronized by the business system, and determine the mapping table corresponding to the target table; Information about the target table is collected, and the table names of the mapping table are marked based on the collection results. The association relationships between the target table and each associated table are statistically analyzed using key fields. The marked table names are used to characterize the correspondence between the mapping table and the target table. The associated tables are other data tables that are related to the target table. The collected and statistical results are cached on the cloud platform; The step of marking the table name of the mapping table includes: for tables with the same data, defining them in the form of key-value pairs, where the key is a custom enumerated meaning value, 1 represents a business table, 2 represents a configuration table, and the value is the table name of the target table of the business system combined with the table name of the mapping table of the big data platform.

2. The method according to claim 1, characterized in that, The collected results include at least one of the following: The target table's system ID, table structure comments, and filename.

3. The method according to claim 1, characterized in that, The key fields include primary keys and foreign keys; The process of calculating the relationships between the target table and each associated table using key fields includes: If the primary key ID of the target table is a foreign key of the associated table, a master-slave relationship is determined to exist between the target table and the associated table, and the master-slave relationship is used as a statistical result.

4. The method according to claim 3, characterized in that, The step of calculating the relationships between the target table and each associated table using key fields also includes: Based on the business scenario corresponding to the target table, the target table and the associated table are tagged, and the tagging results are used as statistical results; the tagging results are used to mark the behavioral operations targeting the target table and the associated table.

5. The method according to claim 1, characterized in that, The method further includes: The target table is categorized according to preset attributes.

6. The method according to claim 5, characterized in that, The preset attribute includes any one of the following: Archive time, synchronization time, query statement.

7. The method according to claim 1, characterized in that, The method further includes: The collected and statistical results are displayed on an interface, and search terms are provided; the search terms are extracted based on the collected and statistical results.

8. A data table management device, characterized in that, Applications in big data platforms include: The determination module is used to obtain the target table synchronized by the business system and determine the mapping table corresponding to the target table. The statistics module is used to collect information from the target table, mark the table names of the mapping table based on the collection results, and count the relationships between the target table and each associated table through key fields; wherein, the marked table names are used to characterize the correspondence between the mapping table and the target table; the associated tables are other data tables that are related to the target table; The caching module is used to cache the collected and statistical results to the cloud platform; The step of marking the table name of the mapping table includes: for tables with the same data, defining them in the form of key-value pairs, where the key is a custom enumerated meaning value, 1 represents a business table, 2 represents a configuration table, and the value is the table name of the target table of the business system combined with the table name of the mapping table of the big data platform.

9. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 7.

Images