Table data processing method and device, equipment, medium and product

Abstract

The invention discloses a table data processing method and device, equipment, a medium and a product, relates to the technical field of big data and the technical field of data processing, and can be applied to the field of financial science and technology. The method comprises the steps of obtaining metadata information corresponding to adjacent detection batches from a metadata database according to a target use table name of a target data lake table; and determining table change contents according to the metadata information of the adjacent detection batches. Through the technical scheme, the change condition of the table in the database can be found in time.

Description

Technical Field

[0001] The present invention relates to the fields of big data technology and data processing technology, and can be applied to the field of financial technology. In particular, it relates to a table data processing method, device, equipment, medium and product. Background Art

[0002] With the popularization of big data technology, various banking business systems are increasingly dependent on data lake data. However, the number of tables stored in the data lake is huge and the fields are complex. Maintaining the table results is a huge project. Once the table structure in the data lake changes and the downstream users of the table cannot be notified in time, it may cause production problems and cause immeasurable losses to the business.

[0003] To deal with changes in the data lake table structure, the current common practice is: 1. Data lake table producers and users jointly agree on a platform to maintain and view the data lake table structure information on the platform. 2. When the data lake table structure needs to change, the producer updates the table structure information on the platform. 3. Data lake table users regularly check the data lake table structure information registered on the platform and compare it with the current table structure. If any differences are found, the landing table structure is adjusted according to the information registered on the platform.

[0004] However, the existing technology has the following disadvantages: 1. The data lake table structure information on the platform is manually maintained by the data producer, and there are often cases of missed maintenance and delayed maintenance, which has a serious impact on the data lake table users. 2. The data lake table structure comparison work on the platform is manually performed by the data lake table users, which is labor-intensive, inefficient and prone to errors. 3. The data lake table structure changes cannot be notified to the data lake table users in a timely manner. When the production is approaching, the table structure adjustment by the data lake table users will cause huge risks to the production operation. Summary of the invention

[0005] The present invention provides a table data processing method, device, equipment, medium and product to quickly detect changes in table data.

[0006] According to one aspect of the present invention, a table data processing method is provided, the method comprising:

[0007] According to the target usage table name of the target data lake table, obtain the metadata information corresponding to the adjacent detection batches from the metadata database;

[0008] The table change content is determined according to the metadata information of the adjacent detection batches.

[0009] According to another aspect of the present invention, there is provided a table data processing device, the device comprising:

[0010] The metadata acquisition module is used to obtain metadata information corresponding to adjacent detection batches from the metadata database according to the target usage table name of the target data lake table;

[0011] The table change content determination module is used to determine the table change content according to the metadata information of the adjacent detection batches.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the table data processing method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the table data processing method described in any embodiment of the present invention when executed.

[0017] According to another aspect of the present invention, a computer program product is provided. The computer program product comprises a computer program. When the computer program is executed by a processor, the table data processing method according to any embodiment of the present invention is implemented.

[0018] The technical solution of the embodiment of the present invention obtains metadata information corresponding to adjacent detection batches from the metadata database according to the target use table name of the target data lake table, and then determines the table change content according to the metadata information of the adjacent detection batches. The above technical solution can quickly locate the table change situation and facilitate the use and maintenance of the table in the data lake.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a table data processing method provided according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a table data processing method provided according to an embodiment of the present invention;

[0023] Figure 3 is a structural schematic diagram of a table data processing device provided according to an embodiment of the present invention;

[0024] Figure 4 It is a structural schematic diagram of an electronic device for implementing the table data processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In addition, it should be noted that the collection, storage, use, processing, transmission, provision and disclosure of relevant information and data of data users and other parties involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals. The relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0028] Figure 11 is a flowchart of a table data processing method provided according to an embodiment of the present invention. This embodiment is applicable to the case of how to perform table data change processing. The method can be executed by a table data processing device, which can be implemented in the form of hardware and / or software. The device can be configured in an electronic device that carries the table data processing function, such as a server. Figure 1 As shown, the method includes:

[0029] S110. Obtain metadata information corresponding to adjacent detection batches from a metadata database according to the target usage table name of the target data lake table.

[0030] In this embodiment, the target data lake table refers to a data table in the data lake, and a data lake table registered in the data lake by a data user. The data user refers to a downstream user of the data lake, that is, a user of a lake table in the data lake.

[0031] It should be noted that the data lake has sufficient data storage and storage capacity to save all the data in an enterprise or organization, and can store a large amount of data of any type, including structured, semi-structured and unstructured data; the data in the data lake is raw data, which is a complete copy of the business data, that is, it maintains its original sample in the business system; the data lake needs to have perfect data management capabilities (perfect metadata), which can manage various data-related elements, including data sources, data formats, connection information, data schema, permission management, etc.; the data lake has a variety of analytical capabilities, including but not limited to batch processing, streaming computing, interactive analysis and machine learning; at the same time, it also provides certain task scheduling and management capabilities. The data lake has perfect data lifecycle management capabilities, which not only needs to store raw data, but also needs to be able to save the intermediate results of various analytical processing, and fully record the data analysis and processing process, which can help users fully and detailedly trace the generation process of any data; the data lake has perfect data acquisition and data publishing capabilities; the data lake can support a variety of data sources, and can obtain full / incremental data from related data sources; and then standardize storage; the data lake can push the results of data analysis and processing to the appropriate storage engine to meet different application access requirements. Support for big data includes ultra-large-scale storage and scalable large-scale data processing capabilities.

[0032] The combination of Hadoop and data lake in this embodiment can provide data users with an efficient, flexible and scalable data processing platform. Data users can use Hadoop's computing power to perform various complex calculations and analyses on the data in the data lake. Among them, Hadoop: is an open source distributed computing platform that allows users to perform calculations on large-scale data sets. The core components of Hadoop include a distributed file system (Hadoop Distributed File System, HDFS) and a computing framework MapReduce. HDFS provides high-fault-tolerant and high-throughput storage for data, while MapReduce provides parallel computing capabilities for data. Among them, HIVE is a data warehouse tool based on Hadoop, which is used for data extraction, transformation, and loading. This is a mechanism that can store, query and analyze large-scale data stored in Hadoop. HIVE uses metadata to understand the structure of data and how to access data; among them, metadata refers to the schema metadata of the table in HIVE.

[0033] The so-called metadata database refers to a database used to store metadata. The so-called detection batch refers to a batch of regular detection of data lake tables in the data lake to detect whether there are changes in the table data in the target data lake table.

[0034] Specifically, according to the target usage table name of the target data lake table, the metadata information corresponding to the adjacent detection batches is obtained from the metadata database using the target usage table name as an index.

[0035] S120. Determine table change content according to metadata information of adjacent detection batches.

[0036] In this embodiment, metadata information refers to the specific data of the target data lake table in the data lake. The so-called table change content refers to the content that has changed in the target data lake table, including but not limited to field addition, field deletion, field length expansion, field length reduction, field name change, etc.

[0037] An optional method is to compare the metadata information corresponding to adjacent detection batches one by one according to the fields to obtain the table change content.

[0038] The technical solution provided by the embodiment of the present invention obtains metadata information corresponding to adjacent detection batches from the metadata database according to the target use table name of the target data lake table, and then determines the table change content according to the metadata information of the adjacent detection batches. The above technical solution can quickly locate the table change situation and facilitate the use and maintenance of the table in the data lake.

[0039] On the basis of the above embodiment, as an optional method of the present invention, after determining the table change content according to the metadata information of adjacent detection batches, it also includes: obtaining the contact information of the data user corresponding to the table change content from the first database, and sending change notification information to the data user; the change notification information includes the target usage table name and the change content.

[0040] The first database is used to store relevant information of the data user, including but not limited to the table name of the data lake table, the user's contact information and the frequency of data use, and can be any type of relational database. The user's contact information includes but is not limited to email, SMS and internal corporate communication methods.

[0041] Specifically, after determining the table change content, the user contact information corresponding to the target usage table name is obtained from the first database, the change notification information is added to the message queue, and the change notification information is sent to the data user.

[0042] It is understandable that timely notification to data users after changes occur in the target data lake table improves the timeliness and accuracy of change information synchronization, thereby reducing the problem of production operation interruptions due to inconsistent data structures.

[0043] Based on the above embodiment, as an optional method of the present invention, after determining the table change content according to the metadata information of adjacent detection batches, it also includes: based on the data cleaning method, cleaning the metadata files corresponding to the historical detection batches using the target table name in the metadata database.

[0044] Specifically, based on the data cleaning method, the metadata files corresponding to the historical detection batches of the target usage table name in the metadata database can be cleaned up. For example, the metadata files corresponding to the two most recent detection batches of the target usage table name can be retained. As another example, the metadata files corresponding to the target usage table name can be cleaned up regularly according to the frequency of data use. For example, the metadata files corresponding to the daily frequency data lake table can be cleaned up for metadata files that are more than 3 days old, and other frequencies are similar. The specific cleaning cycle can also be determined according to its actual situation. For another example, you can set conditions to trigger file cleaning. For example, after the message is pushed successfully, the metadata files corresponding to the historical detection batches are cleaned up, and only the metadata files corresponding to the current detection batch are retained.

[0045] It is understandable that cleaning up the metadata files corresponding to the historical detection batches using the target table name can save system resources.

[0046] Based on the above embodiment, as an optional method of the present invention, it also includes: determining the user information of the data user; the user information includes the target usage table name, target usage frequency and user contact information; storing the user information in the first database; the first database is a relational database.

[0047] Specifically, the user information of the data user may be obtained through an interactive interface, and then the user information is stored in the first data.

[0048] It can be understood that by storing the data user information in relational data, namely the first database, it is convenient to query the data user information.

[0049] Figure 2 is a flow chart of a table data processing method provided according to an embodiment of the present invention. Based on the above embodiment, this embodiment further optimizes "determining table change content according to metadata information of adjacent detection batches" and provides an optional solution. Figure 2 As shown, the method includes:

[0050] S210. Obtain metadata information corresponding to adjacent detection batches from a metadata database according to the target usage table name of the target data lake table.

[0051] S220 , writing metadata information corresponding to adjacent detection batches into files respectively, to obtain a current metadata file and a previous metadata file.

[0052] Specifically, for each detection batch, the metadata information corresponding to the detection batch is written into a file to obtain the metadata file corresponding to the detection batch; for example, for adjacent detection batches, the current metadata file and the previous metadata file are obtained.

[0053] S230: Calculate the file verification code of the current metadata file and the previous metadata file respectively to obtain the current file verification code and the previous file verification code.

[0054] The file verification code is a verification code used to uniquely represent the metadata file, and may be an md5 verification code.

[0055] Specifically, for each detection batch, the file verification code of the metadata file corresponding to the detection batch is calculated. For adjacent detection batches, the current file verification code and the previous file verification code are obtained.

[0056] S240: Determine table change content according to the current file check code, the previous file check code, the current metadata file and the previous metadata file.

[0057] An optional method is to compare the current file checksum with the previous file checksum; if there is a difference, the current metadata file and the previous metadata file are compared to obtain the table changed content.

[0058] Specifically, compare the current file checksum with the previous file checksum. If the checksums are the same, it means that the target data lake table has not changed. If the checksums are inconsistent, it means that the target data lake table has changed. Then further compare the specific contents of the current metadata file and the previous metadata file to obtain the table changes.

[0059] It is understandable that, compared with directly comparing the metadata file, by first comparing the file verification code to determine whether the metadata file has changed, and then comparing the metadata file to determine the table change content, the table change content can be quickly determined.

[0060] The technical solution provided by the embodiment of the present invention obtains metadata information corresponding to adjacent detection batches from the metadata database by using the table name according to the target of the target data lake table, and then writes the metadata information corresponding to the adjacent detection batches into files respectively to obtain the current metadata file and the previous metadata file, and calculates the file verification code of the current metadata file and the previous metadata file respectively to obtain the current file verification code and the previous file verification code, and then determines the table change content according to the current file verification code, the previous file verification code, the current metadata file and the previous metadata file. The above technical solution significantly reduces the degree of manual participation, reduces the problems of missing registration, late registration, registration errors, etc. of variable structure information caused by human factors, saves labor costs, improves the timeliness and accuracy of change information synchronization, and thus reduces the problem of production operation interruption caused by inconsistent data structure.

[0061] Figure 3 1 is a schematic diagram of the structure of a table data processing device provided according to an embodiment of the present invention. This embodiment can be applied to the situation of how to perform table data change processing. The device can be implemented in the form of hardware and / or software. The device can be configured in an electronic device that carries the table data processing function, such as a server. Figure 3 As shown, the device comprises:

[0062] The metadata acquisition module 310 is used to obtain metadata information corresponding to adjacent detection batches from the metadata database according to the target usage table name of the target data lake table;

[0063] The table change content determination module 320 is used to determine the table change content according to the metadata information of the adjacent detection batches.

[0064] The technical solution provided by the embodiment of the present invention obtains metadata information corresponding to adjacent detection batches from the metadata database according to the target use table name of the target data lake table, and then determines the table change content according to the metadata information of the adjacent detection batches. The above technical solution can quickly locate the table change situation and facilitate the use and maintenance of the table in the data lake.

[0065] Optionally, the table change content determination module 320 includes:

[0066] A metadata file determination unit, used to write metadata information corresponding to adjacent detection batches into files respectively, to obtain a current metadata file and a previous metadata file;

[0067] A file verification code determination unit, used to calculate the file verification codes of the current metadata file and the previous metadata file respectively, to obtain the current file verification code and the previous file verification code;

[0068] The table change content determination unit is used to determine the table change content according to the current file check code, the previous file check code, the current metadata file and the previous metadata file.

[0069] Optionally, the table change content determination unit is specifically used for:

[0070] Compare the current file checksum with the previous file checksum;

[0071] If there are any differences, the current metadata file and the previous metadata file are compared to obtain the table changes.

[0072] Optionally, the device further includes a change notification module, which is used to:

[0073] After determining the table change content based on the metadata information of adjacent detection batches, the contact information of the data user corresponding to the table change content is obtained from the first database, and change notification information is sent to the data user; the change notification information includes the target table name and the change content.

[0074] Optionally, the device further includes a data cleaning module, which is used to:

[0075] After determining the table changes based on the metadata information of adjacent detection batches, the metadata files corresponding to the historical detection batches using the target table name in the metadata database are cleaned up based on the data cleaning method.

[0076] Optionally, the device further includes a user information storage module, which is used to:

[0077] Determine the user information of the data user; the user information includes the target usage table name, target usage frequency and user contact information;

[0078] The user information is stored in a first database; the first database is a relational database.

[0079] The table data processing device provided in the embodiment of the present invention can execute the table data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0080] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0081] Figure 4 It is a structural schematic diagram of an electronic device for implementing the table data processing method according to an embodiment of the present invention. Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0082] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0083] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0084] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a table data processing method.

[0085] In some embodiments, the table data processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the table data processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the table data processing method in any other appropriate manner (e.g., by means of firmware).

[0086] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0087] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0088] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0089] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0090] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0091] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0092] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0093] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A table data processing method, characterized in that: include: According to the target usage table name of the target data lake table, obtain the metadata information corresponding to the adjacent detection batches from the metadata database; The table change content is determined according to the metadata information of the adjacent detection batches.

2. The method according to claim 1, characterized in that: Determining table changes based on metadata information of adjacent test batches includes: The metadata information corresponding to the adjacent detection batches is written into files respectively to obtain a current metadata file and a previous metadata file; Calculating the file verification code of the current metadata file and the previous metadata file respectively to obtain the current file verification code and the previous file verification code; The table change content is determined according to the current file check code, the previous file check code, the current metadata file and the previous metadata file.

3. The method according to claim 2, characterized in that Determining table change content according to the current file check code, the previous file check code, the current metadata file, and the previous metadata file includes: Comparing the current file verification code with the previous file verification code; If there is a difference, the content of the current metadata file is compared with that of the previous metadata file to obtain the table changed content.

4. The method according to claim 1, characterized in that: After determining the table change content according to the metadata information of the adjacent detection batches, the method further includes: The contact information of the data user corresponding to the table change content is obtained from the first database, and change notification information is sent to the data user; the change notification information includes the target usage table name and the change content.

5. The method according to any one of claims 2 to 3, characterized in that: After determining the table change content according to the metadata information of the adjacent detection batches, the method further includes: Based on the data cleaning method, metadata files corresponding to historical detection batches of the target usage table name in the metadata database are cleaned.

6. The method according to claim 4, characterized in that Also includes: Determine the user information of the data user; the user information includes the target usage table name, target usage frequency and user contact information; The user information is stored in the first database; the first database is a relational database.

7. A table data processing device, characterized in that: include: The metadata acquisition module is used to obtain metadata information corresponding to adjacent detection batches from the metadata database according to the target usage table name of the target data lake table; The table change content determination module is used to determine the table change content according to the metadata information of the adjacent detection batches.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the table data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the table data processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the table data processing method according to any one of claims 1 to 6.

Images