Data processing method, device, electronic device and storage medium

Abstract

The present disclosure provides a data processing method, apparatus, electronic device, and storage medium, relating to the field of computer technology, particularly to the field of big data processing technology. A specific implementation scheme comprises: generating at least one metadata corresponding to each data item in a database based on at least one metadata type; and determining, when determining that a data processing system calls data, determining the data called by the data processing system in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system. Using embodiments of the present disclosure, the efficiency of data processing system calls is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the field of big data technology, and specifically to a data processing method, device, electronic device, and storage medium. Background Art

[0002] Metadata is data that describes data. Traditional data processing systems, such as Hive, Spark, Presto, and Impala, generally use metadata to describe and manage data. The metadata schemas used by these different types of data processing systems vary. Therefore, their metadata is independent of each other. Summary of the Invention

[0003] The present disclosure provides a data processing method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of the present disclosure, there is provided a data processing method, comprising:

[0005] Based on at least one metadata type, generating at least one corresponding metadata for each data in the database;

[0006] In the case of determining the data processing system call data, the data processing system call data is determined in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system.

[0007] According to another aspect of the present disclosure, there is provided a data processing apparatus, comprising:

[0008] A metadata generation module, configured to generate at least one metadata corresponding to each data in the database based on at least one metadata type;

[0009] The data calling module is used to determine the data called by the data processing system in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system when it is determined that the data processing system calls the data.

[0010] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method in any embodiment of the present disclosure.

[0014] According to another aspect of the present disclosure, a data processing system is provided, including the electronic device according to any embodiment of the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method in any embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the method in any embodiment of the present disclosure when executed by a processor.

[0017] According to the disclosed technology, metadata corresponding to at least one metadata type is provided for each data item in a database. When determining which data a data processing system is to call, the data to be called by the data processing system is determined in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system. This allows the data processing system to accurately call data, improving data calling efficiency.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0020] Figure 1 is a flow chart of the data processing method according to the first embodiment of the present disclosure;

[0021] Figure 2 is a structural block diagram of a data processing system according to a second embodiment of the present disclosure;

[0022] Figure 3 is a timing diagram of metadata update in the third embodiment of the present disclosure;

[0023] Figure 4 is a structural block diagram of a data processing device according to a fourth embodiment of the present disclosure;

[0024] Figure 5 is a structural block diagram of a data processing device according to a fifth embodiment of the present disclosure;

[0025] Figure 6It is a block diagram of an electronic device that implements the data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] Common big data processing systems, such as Hive, Spark, Presto, and Impala, typically use a single metadata schema or metadata type to describe data organization and characteristics. Consequently, metadata and data in these systems are independent, making them difficult to access.

[0028] For different types of data within the same system, such as structured, semi-structured, and unstructured data, some systems use only a single metadata type to describe these diverse data types, making it difficult to fully describe these data types. Furthermore, because different types of data require different metadata, using only a single metadata schema to describe the data cannot accurately capture all of the data's information. Furthermore, data processing systems use different types of data in different ways, and these different uses require different metadata descriptions. Therefore, it's difficult to obtain all the information needed to access data from the same abstract metadata description. Therefore, using only a single abstract metadata type to describe data can lead to inaccurate and incomplete metadata descriptions, impacting the data processing system's ability to access the data.

[0029] For data from external systems, such as external data processing systems, the data is often organized differently across different systems, making direct data sharing impossible. Consequently, some data processing systems, unable to directly access data from external systems, choose to import the data and perform some predefined processing before using it. However, this can affect the efficiency of the data processing system's access to data. Furthermore, some data processing systems use connectors to connect to external systems, accessing data and metadata from these systems. However, this approach has the following drawbacks: First, it lacks flexibility in accessing the required data or information, resulting in low efficiency; second, access to metadata is limited by the external system, making it difficult to obtain all the metadata information currently required.

[0030] To this end, an embodiment of the present disclosure provides a data processing solution, and a data processing system can accurately call data and improve the efficiency of calling data.

[0031] Figure 1 FIG. 1 is a flow chart showing a data processing method according to the first embodiment of the present disclosure. Figure 1 As shown, the method can be applied to a data processing system, a module or unit in a data processing system, or a system or electronic device shared by multiple data processing systems. The method includes the following steps:

[0032] S110, generating at least one corresponding metadata for each data in the database based on at least one metadata type;

[0033] S120 , when determining that the data processing system calls data, determine the data called by the data processing system in a database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system.

[0034] Illustratively, metadata is an abstract description of data. This abstract description can be expressed in a variety of ways, i.e., there can be multiple metadata types. The at least one metadata type can be determined based on the metadata type applicable to each of the at least one data processing system. Each metadata type can be applicable to at least one data processing system. Each data processing system can be applicable to at least one metadata type.

[0035] For example, a database may include multiple data, each of which may correspond to multiple metadata. These multiple metadata collectively describe the same data, differing only in the description methods.

[0036] For example, the metadata corresponding to data A includes {a1, a2, a3, a4}, where metadata a1 is recognizable by data processing system b1, metadata a2 is recognizable by data processing system b2, metadata a3 is recognizable by data processing systems b3 and b4, and metadata a4 is recognizable by data processing systems b5 and b6. The data processing system parses the metadata based on its recognizable metadata to obtain information, and then retrieves data from the database based on the parsed information.

[0037] In some embodiments, the data processing system can call data in a database within the system or call data in a database of an external system.

[0038] Exemplarily, when determining the data called by the data processing system, target metadata is determined from metadata provided by the data processing system based on the metadata type applicable to the data processing system, and the data called by the data processing system is determined in the database based on the target metadata. The metadata provided by the data processing system is determined based on the identifier of the data called by the data processing system.

[0039] Exemplarily, when it is determined that the first data processing system calls data from the second data processing system, the first data processing system sends a data call request to the second data processing system, the data call request including a data identifier. The second data processing system determines corresponding metadata based on the data identifier in the received data call request and returns the determined metadata to the first data processing system. The first data processing system receives the metadata returned by the second data processing system and, based on the metadata type applicable to the first data processing system, determines target metadata within the returned metadata. The first data processing system then determines the data to be called by the first data processing system from a database of the second data processing system based on the target metadata.

[0040] Exemplarily, when it is determined that the first data processing system calls data from the second data processing system, the first data processing system sends a data call request to the second data processing system, where the data call request includes a data identifier and a metadata type applicable to the first data processing system. The second data processing system determines corresponding metadata based on the data identifier in the received data call request, and determines target metadata from the determined metadata based on the metadata type in the received data call request. The first data processing system then determines the data to be called by the first data processing system from a database of the second data processing system based on the target metadata.

[0041] In some embodiments, multiple data processing systems may share a database and call data based on different metadata.

[0042] In some embodiments, when a piece of data is updated, one of the metadata of the data is updated accordingly, or updated based on a user's request.

[0043] Exemplarily, in response to an update request for the i-th data, the i-th data is updated, and based on the updated content of the i-th data, the first metadata of the i-th data is updated, where i is a positive integer.

[0044] In some embodiments, the metadata may be updated in response to a metadata update request from a user.

[0045] Exemplarily, in response to an update request for first metadata corresponding to the i-th data, the first metadata is updated based on the update content in the update request. The update content may or may not be related to the i-th data. For example, the description in the first metadata may be modified, but the data described by the first metadata may not be updated.

[0046] In some embodiments, in order to synchronize the update of metadata, the metadata of each data may be monitored. When it is monitored that one metadata of a certain data has been updated, the other metadata of the data may be updated.

[0047] Exemplarily, when it is monitored that the first metadata of the i-th data in the database has been updated, the second metadata of the i-th data is updated based on the updated content of the first metadata.

[0048] In this example, when one metadata is updated, other metadata that describe the same data can also be updated simultaneously, thereby improving metadata update efficiency and ensuring that all metadata corresponding to the same data are updated synchronously.

[0049] In the embodiment of the present disclosure, the first metadata and the second metadata are metadata of the i-th data, and are not represented as the first metadata and the second metadata.

[0050] In some embodiments, when it is monitored that one metadata of certain data has been updated, other metadata of the data are not necessarily updated.

[0051] Exemplarily, the updating of the second metadata of the i-th data based on the updated content of the first metadata includes:

[0052] In a case where the updated content of the first metadata includes the first updated content, the second metadata of the i-th data is updated based on the first updated content; wherein the first updated content is related to the updated content of the i-th data.

[0053] In this example, only when the update of the first metadata is related to the update of the i-th data, the second metadata is modified based on the updated content of the first metadata, so as to ensure the efficiency and accuracy of the update.

[0054] For example, if the updated content of the first metadata does not include the first updated content, the second metadata of the i-th data is not updated. For example, the updated content of the first metadata is related to the update of its own description method, for example, from description method A1 to description method A2.

[0055] However, in this case, although the second metadata of the i-th data is not updated, metadata of the same metadata type as the first metadata in the database will be updated to ensure that metadata of the same type in the database can be updated synchronously.

[0056] Exemplarily, when the updated content of the first metadata includes the second updated content, metadata of the same metadata type as the first metadata in the database is updated based on the second updated content; wherein the second updated content is related to the metadata type of the first metadata.

[0057] In this example, when the update of the first metadata is related to its metadata type, metadata of the same metadata type as the first metadata in the database is updated based on its updated content to ensure update efficiency and accuracy.

[0058] In some embodiments, a metadata generation service registration service may be provided to users. The registration service may be provided in the form of an interface or a registrar. Users may send a metadata generation service registration request to the data processing system through the registration service to register a corresponding metadata generation service for a specified metadata type.

[0059] Exemplarily, in response to a registration request for a metadata generation service, corresponding metadata generation logic is determined based on a first metadata type in the registration request; and based on the determined metadata generation logic, a metadata generation service corresponding to the first metadata type is registered; wherein the registered metadata generation service can be used to generate metadata corresponding to the first metadata type for data in a database.

[0060] In this exemplary embodiment, corresponding metadata generation services may be registered for new metadata types to further improve the richness of metadata.

[0061] In some embodiments, metadata generation services corresponding to one or more metadata types may also be deleted.

[0062] Exemplarily, in response to the first deletion request, the corresponding metadata generation service is deleted based on the metadata type in the first deletion request.

[0063] In some embodiments, during the process of generating metadata, a corresponding metadata generation service is first determined, and then the service is called to generate the corresponding metadata.

[0064] For example, in the above step S110, generating at least one metadata corresponding to each data in the database based on at least one metadata type may include:

[0065] Determining, based on at least one metadata type, a metadata generation service corresponding to each metadata type in the at least one metadata type;

[0066] Based on the metadata generation service corresponding to each metadata type, corresponding metadata is generated for each data in the database.

[0067] In this example, the corresponding metadata generation service can be determined based on the specified metadata type, and then the determined generation service can be called to generate corresponding metadata for each data in the database. In this way, metadata can be generated flexibly.

[0068] In some embodiments, a metadata addition service may be provided to users in the form of an interface or screen. Through the metadata addition service, users may send metadata generation requests to the data processing system to add one or more corresponding metadata for the specified data.

[0069] For example, in step S110, generating at least one metadata corresponding to each data in the database based on at least one metadata type may include:

[0070] In response to the metadata generation request, determining corresponding target data in a database based on the data identifier in the metadata generation request;

[0071] Generates corresponding metadata for the target data based on the metadata type in the metadata generation request.

[0072] In this example, corresponding metadata can be flexibly generated based on the metadata generation request, so that the generated metadata better meets the needs of the user.

[0073] In some embodiments, a corresponding metadata generation service may be determined based on the metadata type in the metadata generation request, and corresponding metadata may be generated for the target data based on the determined metadata generation service.

[0074] In some embodiments, one or more metadata of the data may also be deleted.

[0075] Exemplarily, in response to a second deletion request, based on the data identifier in the second deletion request, the corresponding target data and the metadata corresponding to the target data are determined in the database; and based on the metadata type in the second deletion request, metadata of the same type as the metadata in the second deletion request is deleted in the metadata corresponding to the target data.

[0076] In this example, the metadata can be accurately deleted.

[0077] In the embodiment of the present disclosure, by providing multiple metadata of different metadata types for data, the efficiency and accuracy of calling data in the data processing system can be improved.

[0078] In the embodiment of the present disclosure, a monitoring service is provided for metadata of data. When it is monitored that one metadata of the data has been updated, the other metadata are also updated at the same time, thereby achieving synchronous update of the metadata.

[0079] In the embodiment of the present disclosure, a registration service for metadata generation services and a deletion service for metadata generation services are provided for specified metadata types, which can flexibly provide generation services corresponding to various metadata types, thereby making the metadata description of data more diverse and comprehensive.

[0080] In the embodiment of the present disclosure, metadata addition and deletion services are provided for specified data, which can flexibly set the metadata of the data and make the metadata's description of the data more diverse and comprehensive.

[0081] Figure 2 A structural block diagram of a data processing system according to a second embodiment of the present disclosure is shown.

[0082] The metadata service interface 201 is used to add one or more metadata to the specified data; and to delete one or more metadata from the metadata corresponding to the specified data.

[0083] The metadata generation service registrar 202 is used to provide metadata generation service registration and deletion services. The registration service is used to register a metadata generation service corresponding to a specified metadata type. A registered metadata generation service can be used to generate metadata for the specified data. The deletion service is used to delete the metadata generation service corresponding to the specified metadata type.

[0084] The metadata generation service 204 generates data for the data based on the metadata generation logic corresponding to the corresponding metadata type. Each metadata type corresponds to a metadata generation logic.

[0085] The metadata listener 203 is used to monitor all metadata. When a metadata of a certain data is updated, the listener will monitor that the metadata has been updated and update other metadata in the corresponding data based on the updated content of the metadata.

[0086] Each type of data 205 can generate different metadata based on different metadata generation services 204 .

[0087] Figure 3 A timing diagram showing metadata update according to the third embodiment of the present disclosure is shown.

[0088] like Figure 3 As shown, an external user can call an interface to change metadata 1; wherein the data described by metadata 1 is data A;

[0089] The metadata service interface 301 can receive metadata update requests from external users and forward the requests to the metadata generation service 1;

[0090] Metadata generation service 1 updates metadata 1 of data A;

[0091] When metadata generation service 1 updates metadata 1, metadata listener 303 broadcasts the changes to other metadata generation services.

[0092] The metadata generation service 2 receives the updated content of the broadcasted metadata 1 and updates the metadata 2 of the data A based on the updated content of the metadata 1 .

[0093] in, Figure 3 The metadata service interface 301 and the metadata listener 303 have the same functions as the metadata service interface 201 and the metadata listener 203 in the above embodiment, and are not described in detail here.

[0094] External users can call the interface for querying metadata 2 to query new metadata 2.

[0095] Figure 4 A structural block diagram of a data processing device according to a fourth embodiment of the present disclosure is shown.

[0096] like Figure 4 As shown, the data processing device may include:

[0097] The metadata generation module 410 is configured to generate at least one metadata corresponding to each data in the database based on at least one metadata type;

[0098] The data calling module 420 is used to determine the data called by the data processing system in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system when determining that the data processing system calls the data.

[0099] Figure 5 FIG. 5 shows a structural block diagram of a data processing device according to a fifth embodiment of the present disclosure. Figure 5 As shown, the data processing device includes a metadata generation module 510, a data calling module 520, a metadata update module 530, a generation logic determination module 540, and a generation service registration module 550. The metadata generation module 510 and the data calling module 520 have the same functions as the metadata generation module 410 and the data calling module 420 in the above embodiment and are not described in detail here.

[0100] In some embodiments, as Figure 5 As shown, the device may also include:

[0101] The metadata updating module 530 is configured to update the second metadata of the i-th data based on the updated content of the first metadata when it is detected that the first metadata of the i-th data in the database has been updated, where i is a positive integer.

[0102] In some embodiments, the metadata update module 530 is configured to:

[0103] In a case where the updated content of the first metadata includes first updated content, the second metadata of the i-th data is updated based on the first updated content; wherein the first updated content is related to the updated content of the i-th data.

[0104] In some embodiments, further comprising:

[0105] A generation logic determination module 540 is configured to respond to a registration request for a metadata generation service and determine a corresponding metadata generation logic based on a first metadata type in the registration request;

[0106] The generation service registration module 550 is used to register the metadata generation service corresponding to the first metadata type based on the metadata generation logic; wherein the metadata generation service corresponding to the first metadata type is used to generate metadata corresponding to the first metadata type for data in the database.

[0107] In some embodiments, the metadata generation module 510 includes:

[0108] A generation service determining unit 511 is configured to determine, based on at least one metadata type, a metadata generation service corresponding to each metadata type in the at least one metadata type;

[0109] The metadata generating unit 512 is configured to generate corresponding metadata for each data in the database based on the metadata generating service corresponding to each metadata type.

[0110] In some embodiments, the metadata generation module 510 includes:

[0111] a target data determining unit 513, configured to determine, in response to a metadata generation request, corresponding target data in the database based on a data identifier in the metadata generation request;

[0112] The metadata adding unit 514 is configured to generate corresponding metadata for the target data based on the metadata type in the metadata generation request.

[0113] The functions of each unit, module or sub-module in each device of the embodiments of the present disclosure can be found in the corresponding description in the above method embodiments, and will not be repeated here.

[0114] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0115] According to an embodiment of the present disclosure, the present disclosure further provides a data processing system, which may include the electronic device implemented in the present disclosure.

[0116] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0117] like Figure 6 As shown, electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of electronic device 600 can also be stored in RAM 603. Computing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0118] Multiple components in the electronic device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0119] The computing unit 601 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the data processing method. For example, in some embodiments, the data processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via the ROM 102 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the data processing method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0120] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] Program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable lane change control device, such that, when executed by the processor or controller, the program code implements the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0125] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0126] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0127] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A data processing method, comprising: generating at least one metadata corresponding to each data in the database based on at least one metadata type; wherein the at least one metadata type is determined based on the metadata type applicable to each data processing system in the at least one data processing system; In the case of determining the data called by the data processing system, determining the data called by the data processing system in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system; In response to a registration request for a metadata generation service, determining corresponding metadata generation logic based on a first metadata type in the registration request; Based on the metadata generation logic, a metadata generation service corresponding to the first metadata type is registered; wherein the metadata generation service corresponding to the first metadata type is used to generate metadata corresponding to the first metadata type for the data in the database.

2. The method according to claim 1, wherein include: When it is monitored that the first metadata of the i-th data in the database has been updated, the second metadata of the i-th data is updated based on the updated content of the first metadata, where i is a positive integer.

3. The method according to claim 2, wherein: The updating of the second metadata of the i-th data based on the updated content of the first metadata includes: In a case where the updated content of the first metadata includes first updated content, the second metadata of the i-th data is updated based on the first updated content; wherein the first updated content is related to the updated content of the i-th data.

4. The method according to any one of claims 1 to 3, wherein: The step of generating at least one metadata corresponding to each data in the database based on at least one metadata type includes: Based on at least one metadata type, determining a metadata generation service corresponding to each metadata type in the at least one metadata type; Based on the metadata generation service corresponding to each metadata type, corresponding metadata is generated for each data in the database.

5. The method according to any one of claims 1 to 4, wherein: The step of generating at least one metadata corresponding to each data in the database based on at least one metadata type includes: In response to the metadata generation request, determining corresponding target data in the database based on the data identifier in the metadata generation request; Generate corresponding metadata for the target data based on the metadata type in the metadata generation request.

6. A data processing device comprising: a metadata generation module, configured to generate at least one metadata corresponding to each data in the database based on at least one metadata type; wherein the at least one metadata type is determined based on a metadata type applicable to each data processing system in at least one data processing system; a data calling module for, when determining that the data processing system calls data, determining the data called by the data processing system in the database based on the metadata type applicable to the data processing system and the metadata provided by the data processing system; a generation logic determination module, configured to respond to a registration request for a metadata generation service and determine a corresponding metadata generation logic based on a first metadata type in the registration request; A generation service registration module is used to register a metadata generation service corresponding to the first metadata type based on the metadata generation logic; wherein the metadata generation service corresponding to the first metadata type is used to generate metadata corresponding to the first metadata type for the data in the database.

7. The device according to claim 6, wherein include: The metadata updating module is configured to update the second metadata of the i-th data based on the updated content of the first metadata when it is detected that the first metadata of the i-th data in the database has been updated, where i is a positive integer.

8. The device according to claim 7, wherein The metadata updating module is used to: In a case where the updated content of the first metadata includes first updated content, the second metadata of the i-th data is updated based on the first updated content; wherein the first updated content is related to the updated content of the i-th data.

9. The device according to any one of claims 6 to 8, wherein: The metadata generation module includes: a generation service determining unit, configured to determine, based on at least one metadata type, a metadata generation service corresponding to each metadata type in the at least one metadata type; The metadata generation unit is configured to generate corresponding metadata for each data in the database based on the metadata generation service corresponding to each metadata type.

10. The device according to any one of claims 6 to 9, wherein: The metadata generation module includes: a target data determining unit, configured to determine, in response to a metadata generation request, corresponding target data in the database based on a data identifier in the metadata generation request; The metadata adding unit is used to generate corresponding metadata for the target data based on the metadata type in the metadata generation request.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A data processing system comprising the electronic device according to claim 11.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 5.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images