Data sharing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a data sharing method and device, equipment and a storage medium, and relates to the technical field of computers. The data sharing method is applied to a data sharing platform, and comprises the following steps: in response to a data acquisition request for a source data source, analyzing the source data source, and generating a data standard model corresponding to the source data source; based on the data standard model corresponding to the source data source, establishing a standard database table corresponding to the source data source in a standard database, and collecting data of the source data source to the standard database table; and in response to a sharing request for the target library table data in the standard database, encapsulating the target library table data into a target data source for sharing.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data sharing method, device, equipment and storage medium. Background Art

[0002] In order to promote data openness and cooperation, data sharing technology has gradually developed. In related technologies, a data sharing platform is provided, through which the database table data in the database can be shared. However, in addition to the database table data, there are also heterogeneous data such as interfaces, files, and messages that need to be shared. How to share these heterogeneous data has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a data sharing method, apparatus, device and storage medium for sharing heterogeneous data.

[0004] In a first aspect, an embodiment of the present application provides a data sharing method, which is applied to a data sharing platform, and the method includes: In response to a data collection request for a source data source, the source data source is parsed to generate a data standard model corresponding to the source data source; Based on the data standard model corresponding to the source data source, a standard database table corresponding to the source data source is established in a standard database, and the data of the source data source is collected into the standard database table; In response to a sharing request for target library table data in the standard database, the target library table data is encapsulated as a target data source for sharing.

[0005] In a second aspect, an embodiment of the present application provides a data sharing device, which is applied to a data sharing platform, and the device includes: A standard model generation module, configured to analyze the source data source in response to a data collection request for the source data source, and generate a data standard model corresponding to the source data source; A data collection module, used to establish a standard database table corresponding to the source data source in a standard database based on a data standard model corresponding to the source data source, and collect the data of the source data source into the standard database table; The data sharing module is used to respond to a sharing request for the target library table data in the standard database and encapsulate the target library table data as a target data source for sharing.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer-readable program to implement the data sharing method provided in the first aspect of the embodiment of the present application.

[0007] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the data sharing method as described in the first aspect of the embodiment of the present application.

[0008] In an embodiment of the present application, by responding to a data collection request for a source data source, the source data source is parsed to generate a data standard model corresponding to the source data source; based on the data standard model corresponding to the source data source, a standard database table corresponding to the source data source is established in a standard database, and the data of the source data source is collected into the standard database table; thereafter, in response to a sharing request for target library table data in the standard database, the target library table data is encapsulated as a target data source for sharing, thereby enabling sharing of heterogeneous and heterogeneous data sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0010] Figure 1 A schematic diagram of an application scenario of a data sharing method provided in an embodiment of the present application; Figure 2 A flowchart of a data sharing method provided in an embodiment of the present application; Figure 3 An example of a source library table and its corresponding data standard model and standard database table provided in an embodiment of the present application; Figure 4 An example of a source file and its corresponding data standard model and standard database table provided in an embodiment of the present application; Figure 5 An example of a source interface and its corresponding data standard model and standard database table provided in an embodiment of the present application; Figure 6 An example of a source message and its corresponding data standard model and standard database table provided in an embodiment of the present application; Figure 7 A schematic diagram of a data sharing device provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] Embodiments of the present application provide a data sharing method, apparatus, device, and storage medium.

[0012] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0013] Figure 1 is a schematic diagram of an application scenario of a data sharing method provided in an embodiment of the present application, such as Figure 1 As shown, the application scenario includes a data sharing platform 101 and multiple data source platforms 102, and each data source platform 102 is communicatively connected to the data sharing platform 101. The data sharing method provided in the embodiment of the present application can be applied to the data sharing platform 101. Based on the data sharing method provided in the embodiment of the present application, upon receiving a data collection request for a source data source in a source data source platform, the data sharing platform 101 can parse the source data source, generate a data standard model corresponding to the source data source, and establish a standard database table corresponding to the source data source in the standard database based on the data standard model, collect the data of the source data source into the standard database table, so as to persistently store the data of the source data source in the standard database. Upon receiving a sharing request for target library data in the standard database, the data sharing platform 101 can encapsulate the target library table data as a target data source to share it with the target data source platform.

[0014] The source data source platform and the target data source platform are designated data source platforms among the multiple data source platforms 102. Based on the data sharing method provided in the embodiment of the present application, it is possible to realize the sharing of heterogeneous and heterogeneous data sources, the data source types of the source data source and the target data source can be different, and there is no need to connect the source data source platform and the target data source platform.

[0015] See also Figure 2 , is a flow chart of a data sharing method provided in an embodiment of the present application. Figure 2 As shown, the data sharing method provided in the embodiment of the present application includes the following steps: S202, in response to a data collection request for a source data source, parsing the source data source, and generating a data standard model corresponding to the source data source.

[0016] In an embodiment of the present application, a data collection request for a source data source is generated based on a data collection operation performed by a user on a data sharing platform, and carries collection information of the source data source. The collection information may include information such as the location of the source data source and the type of the data source. The location information of the source data source is used to indicate the storage location of the source data source, which may be the URL information of the source data source on the source data source platform. The data source type of the source data source is used to indicate the data source type of the source data source, which may be, for example, a library table, a file, an interface, or a message.

[0017] In the embodiment of the present application, the data standard model corresponding to the source data source is a data model for uniformly and standardizing the data description of the source data source, which can be a logical data model of the source data source. The model describes the source data source as a logical entity, which can include: the name of the entity attribute in the source data source logical entity, the data type, whether it is a primary key, and other information.

[0018] The name of the entity attribute is used to indicate the meaning of the entity attribute, which is convenient for understanding and maintenance.

[0019] The data type of entity attributes is used to indicate the data type and format of entity attributes. The data types of entity attributes can include: string, decimal, integer, date, binary, and big text. Among them, the string type is used to store text information, which consists of zero or more characters. Common string types include ASCII string, UTF-8 string, etc.; the decimal type is used to store fixed-point values, which can represent integers and decimal parts; the integer type is used to store values ​​without decimal parts. Common integer types include signed integers and unsigned integers, as well as integers of different digits; the date type is used to store date and time information. Common date types include YYYY-MM-DD format, UNIX timestamp, etc.; the binary type is used to store raw binary data, usually used to represent images, audio, video and other files; big text: used to store text information with longer lengths, usually without length restrictions.

[0020] Whether the entity attribute is a primary key is used to indicate whether the entity attribute can uniquely identify the entity.

[0021] In the embodiment of the present application, the data standard model corresponding to the source data source can be generated based on the field metadata of the source data source, and the field metadata of the source data source can be obtained by parsing the source data source. The source data source is allowed to be any type of data source such as a library table, a file, an interface, a message, etc. In other words, the source data source can be a source library table, a source file, a source interface, or a source message. For each source data source, a corresponding parsing unit can be used to parse it and generate its corresponding data standard model.

[0022] In one implementation, when the source data source is a source library table, a library table parsing unit may be called to parse the source library table to generate a data standard model corresponding to the source library table.

[0023] Specifically, parsing the source library table to generate a data standard model corresponding to the source library table may include the following steps: Step 1: Perform metadata exploration on the source library table to obtain the field metadata of the source library table.

[0024] The field metadata of the source library table is used to describe the fields in the source library table, which may include: the name of the field in the source library table, the data type, whether it is a primary key, and other information.

[0025] In a specific implementation, a database connection with the source library table can be established based on the location information of the source library table carried in the data collection request, and metadata exploration can be performed on the source library table to obtain the table structure information of the source library table. The table structure information can be reversed to obtain field metadata such as the name, data type, and whether it is the primary key of the field in the source library table.

[0026] Among them, according to the database type of the source library table, you can select a suitable metadata exploration method to perform metadata exploration on the source to obtain the table structure information of the source data source of the library table type. For example, if the database type of the source library table is a relational database, you can use SQL query statements to obtain the table structure information of the source library table. If the database type of the source library table is a non-relational database, you can use query commands or APIs to obtain the table structure information of the source library table.

[0027] Step 2: Generate a data standard model corresponding to the source library table based on the field metadata of the source library table.

[0028] Among them, the source library table can be mapped to a logical entity, a field in the source library table can be mapped to an entity attribute in the data standard model of the logical entity, the name of the field can be mapped to the name of the entity attribute, the data type of the field can be mapped to the data type of the entity attribute, and whether the field is the primary key can be mapped to whether the entity attribute is the primary key, thereby generating a data standard model corresponding to the source library table.

[0029] Assume that the source database table is as follows Figure 3 The database table shown in (a) above has a data standard model corresponding to the source database table, such as Figure 3 As shown in (b).

[0030] In one implementation, when the source data source is a source file, a file parsing unit may be called to parse the source file and generate a data standard model corresponding to the source file.

[0031] Specifically, parsing the source file to generate a data standard model corresponding to the source file may include the following steps: Step 1: parse the source file based on the file type of the source file to obtain the field metadata of the source file.

[0032] The field metadata of the source file is used to describe the fields in the source file, which may include information such as the name of the field in the source file, data type, whether it is a primary key, etc.

[0033] The source file is a formatted file, and its file type can be excel, csv or text. For different types of source files, you can parse them based on the corresponding methods to obtain their field metadata.

[0034] In one implementation, when the source file is an Excel file, each column of the Excel file can be mapped to a field, and the field name of each field can be determined based on the column header of each column, the data type of each field can be determined based on the length or type of sample data in each column, and whether each field is a primary key can be determined based on the column header name of each column.

[0035] For example, for each column in an Excel file, the column header in the column can be mapped to the field name; the first data in the column can be determined as the sample data of the column, and it is determined whether the length of the sample data is greater than 2000. If so, it is determined that the data type of the field corresponding to the column is large text. If not, it is determined whether the data type of the sample data is "string". If so, it is determined that the data type of the field corresponding to the column is string. If not, it is determined whether the data type of the sample data is an integer. If so, it is determined that the data type of the field corresponding to the column is an integer. If not, it is determined whether the data type of the sample data is a date. If so, it is determined that the data type of the field corresponding to the column is a date. If not, it is determined that the data type of the field corresponding to the column is a decimal. It can be determined whether the field corresponding to the column is the primary key based on whether the column header in the column is "id".

[0036] In one implementation, when the source file is a csv file or a text file, the header row in the csv file or the text file can be read, each header separated by a delimiter in the header row can be mapped to a field, the field name of each field can be determined based on each header name, the data type of each field can be determined based on the length or type of the sample data under each header, and whether each field is a primary key can be determined based on the title name of each title. The delimiter in the header row is, for example, a comma or a semicolon, and the string between every two adjacent delimiters is a header.

[0037] For example, for a csv file or each title in a csv file, the title name of the title can be mapped to a field name; the first data under the title is determined as the sample data of the title, and it is determined whether the length of the sample data is greater than 2000. If so, it is determined that the data type of the field corresponding to the title is large text; if not, it is determined whether the data type of the sample data is "string"; if so, it is determined that the data type of the field corresponding to the title is a string; if not, it is determined whether the data type of the sample data is an integer; if so, it is determined that the data type of the field corresponding to the title is an integer; if not, it is determined whether the data type of the sample data is a date; if so, it is determined that the data type of the field corresponding to the title is a date; if not, it is determined that the data type of the field corresponding to the title is a decimal; whether the field corresponding to the title is the primary key can be determined based on whether the title name of the title is "id".

[0038] Step 2: Generate a data standard model corresponding to the source file based on the field metadata of the source file.

[0039] Specifically, the source file can be mapped to a logical entity, each field in the source file can be mapped to an entity attribute in the data standard model of the logical entity, and the name, data type, whether the field is the primary key and other information of each field can be mapped to the name, data type, whether the field is the primary key and other information of the entity attribute, thereby generating a data standard model corresponding to the source file.

[0040] Assume the source file is as follows Figure 4 For example, in the Excel file shown in (a), the data standard model corresponding to the source file is as follows: Figure 4 As shown in (b).

[0041] In one implementation, when the source data source is a source interface, an interface parsing unit may be called to parse the source interface and generate a data standard model corresponding to the source interface.

[0042] Specifically, the source interface is parsed to generate a data standard model corresponding to the source interface, which specifically includes the following steps: Step 1: parse the interface data of the source interface to obtain the field metadata of the interface data.

[0043] The interface data may be JSON data returned by the source interface, and the source interface may be called through an API to obtain the JSON data returned by the source interface.

[0044] The field metadata of the interface data is used to describe the fields in the interface data, which may include: the name of the field, the data type, whether the field is a primary key, and other information.

[0045] In specific implementation, each key of all key-value pairs in the JSON data can be mapped to a field, and the key name of each key can be mapped to the field name of the field corresponding to each key. Based on the length or type of the value corresponding to each key, the data type of the field corresponding to each key can be determined. Based on the key name of each key, it can be determined whether each field is the primary key.

[0046] For example, for each key in the JSON data, the key name of the key can be mapped to the field name. It can be determined whether the data length of the value corresponding to the key is greater than 2000. If so, the data type of the field corresponding to the key is determined to be large text. If not, it is determined whether the data type of the value corresponding to the key is "string". If so, the data type of the field corresponding to the key is determined to be a string. If not, it is determined whether the data type of the value corresponding to the key is an integer. If so, the data type of the field corresponding to the key is determined to be an integer. If not, it is determined whether the data type of the value corresponding to the key is a date. If so, the data type of the field corresponding to the key is determined to be a date. If not, the data type of the field corresponding to the key is determined to be a decimal. It can be determined whether the field corresponding to the key is a primary key based on whether the key name of the key is "id".

[0047] Step 2: Generate a data standard model corresponding to the interface data based on the field metadata of the interface data.

[0048] Specifically, the source interface can be mapped to a logical entity, each field in the interface data can be mapped to an entity attribute in the data standard model of the logical entity, and the field name, data type, whether it is the primary key and other information of each field can be mapped to the name, data type, whether it is the primary key and other information of the entity attribute, thereby generating a data standard model corresponding to the source file.

[0049] Assume the source interface is as follows Figure 5 For the interface shown in (a), the data standard model corresponding to the source interface is, for example, Figure 5 As shown in (b).

[0050] In an embodiment of the present application, when the source data source is a source message, a message parsing unit can be called to parse the source message and generate a data standard model corresponding to the source message.

[0051] Specifically, parsing the source message and generating a data standard model corresponding to the source message includes the following steps: Step 1: parse the message body of the source message to obtain the field metadata of the message body.

[0052] The field metadata of the source message is used to describe the fields in the source message, which may include: the name of the field, data type, whether it is a primary key, and other information.

[0053] Similar to obtaining the field metadata of the interface data based on the key-value information in the interface data mentioned above, each key in all key-value pairs of the message body of the source message can be mapped to a field, and the key name of each key can be mapped to the field name of the field corresponding to each key. Based on the length or type of the value corresponding to each key, the data type of the field corresponding to each key can be determined, and based on the column header name of each column, it can be determined whether each field is the primary key.

[0054] Step 2: Generate a data standard model corresponding to the source message based on the field metadata of the message body.

[0055] Specifically, the source message can be mapped to a logical entity, each field in the message body of the source message can be mapped to an entity attribute in the data standard model of the logical entity, and the name, data type, whether it is the primary key and other information of each field can be mapped to the name, data type, whether it is the primary key and other information of the entity attribute, thereby generating a data standard model corresponding to the source message.

[0056] Assume the source message is Figure 6 For the message shown in (a), the data standard model corresponding to the source message is, for example, Figure 6 As shown in (b).

[0057] It is worth mentioning that in the embodiments of the present application, for each source data source, the method of parsing the source data source to generate a data standard model is encapsulated into a corresponding parsing unit. In specific implementation, the parsing unit corresponding to each source data source is called to parse each source data source to generate a data standard model for each source data source. In some embodiments, the method of parsing each source data source and generating a data standard model corresponding to each source data source may not be encapsulated. For each source data source, the corresponding method mentioned above may be directly used to parse it to obtain its corresponding data standard model.

[0058] S204, based on the data standard model corresponding to the source data source, establish a standard database table corresponding to the source data source in the standard database, and collect the data of the source data source into the standard database table.

[0059] In the embodiment of the present application, the standard database is used to persistently store the data in the source data source. A table creation statement can be generated based on the data standard model corresponding to the source data source and the database type of the standard database; the table creation statement is executed to create a standard database table corresponding to the source data source in the standard database. For example, Figure 3 The source database table shown in (a) corresponds to the standard database table, for example Figure 3 As shown in (c); Figure 4The standard database table corresponding to the source file shown in (a) is as follows Figure 4 As shown in (c); Figure 5 The standard database table corresponding to the source interface shown in (a) is as follows Figure 5 As shown in (c); Figure 6 The standard database table corresponding to the source message shown in (a) is as follows Figure 6 As shown in (c).

[0060] In the embodiment of the present application, for each source data source, the data in the source data source can be collected into the standard database table corresponding to the source data source based on the mapping relationship between the fields in the source data source and the fields in the standard database table corresponding to the source data source. The fields in the source data source and the fields in the standard database table corresponding to the source data source are mapped through the entity attributes of the data standard model corresponding to the source data source.

[0061] For example, when the source data source is a source database table, data can be imported from the source database table into the standard database table corresponding to the source database table based on the mapping relationship between the fields of the source data source database table and the standard database table.

[0062] When the source data source is a source file, the data in the source file can be imported into the standard database table corresponding to the file type source data source through the mapping relationship between the fields in the source file and the fields in the standard library table.

[0063] When the source data source is a source interface, the interface data of the source interface can be imported into the standard database table through the mapping relationship between the fields in the interface data of the source interface and the fields in the standard database table.

[0064] When the source data source is a source message, the interface data of the source message can be imported into the standard database table through the mapping relationship between the fields in the message body of the source message and the fields in the standard database table.

[0065] It is worth mentioning that in the embodiment of the present application, for each data collection request to the source data source, the above steps S202 to S204 are executed, so that the data from different source data sources are persistently stored in the same standard database.

[0066] S206 , in response to a request for sharing target library table data in the standard database, encapsulate the target library table data as a target data source for sharing.

[0067] In the embodiment of the present application, a target library table data sharing request is generated based on a sharing operation performed by a user on a data sharing platform, and carries the target library table data demand information. The target library table data is specifically determined according to the demand information carried in the sharing request.

[0068] In the embodiment of the present application, standard database tables corresponding to different source data sources are stored in the standard database. The target database table data allows any standard database table data in the standard database, allowing a single source data source to be shared separately, and also allowing multiple heterogeneous source data sources to be integrated and shared. The target database table data may specifically include: part of the database table data in one or more standard database tables in the standard database, or all of the database table data in one or more standard database tables in the standard database.

[0069] For example, the target library table data can be all the library table data in the standard database table 1 corresponding to the source data source 1, or part of the library table data in the standard database table 1 corresponding to the source data source 1, or the collection data of all the library table data in the standard database table 1 corresponding to the source data source 1 and all the library table data in the standard database table 2 corresponding to the source data source 2, or the collection data of part of the library table data in the standard database table 1 corresponding to the source data source 1 and part of the library table data in the standard database table 2 corresponding to the source data source 2.

[0070] In the embodiment of the present application, the data source type of the target data source can be any type such as an interface, a message, a file, or a library table, that is, the target data source can be a target interface, a target message, a target file, or a target library table. The specific type is determined according to the demand information carried by the sharing request. The demand information carried by the sharing request includes the demand type of the target data source. The data source type of the target data source can be different from that of the source data source.

[0071] In one implementation, when the requirement type of the target data source is a library table, the target library table data may be encapsulated into a target library table, and the target library table may be sent to the target database for sharing.

[0072] The target database may be a database of a target data source platform, which is specified by a user and may be determined according to the demand information carried in the sharing request. The database type of the target database table is the database type of the target database.

[0073] In one implementation, when the required type of the target data source is a file, the target library table data may be encapsulated as a target file, and the target file may be sent to a target file server for sharing.

[0074] The target file may be an excel file, a csv file or a text file, which is determined according to the demand information. The target file may be uploaded to the target resource directory in the target file server by FTP. The target file server may be a server in the target data source platform, and the target file server and the target resource directory may be determined according to the demand information carried in the sharing request.

[0075] In one implementation, when the demand type of the target data source is a message, the target library table data is encapsulated as a target message, and the target message can be pushed to the target topic for sharing.

[0076] The target topic is the message topic of the target data source platform, which can be determined based on the demand information carried in the sharing request. The message consumer can obtain the target message from the target topic for message consumption.

[0077] In one implementation, when the requirement type of the target data source is an interface, the target library table data is encapsulated as a target interface, and the call information of the target interface is output to share the target interface.

[0078] The target interface may be an interface of the target data source platform, and the call information of the target interface may be displayed on the user interaction interface of the data sharing platform, so that the user can call the target interface based on the call information to access the target library table data in the standard database. The call information may include the interface information of the target interface and the related information of the interface authentication, etc., which are used for calling the target interface.

[0079] In one implementation mode, further, in order to ensure the consistency and accuracy of shared data and guarantee the smooth progress of data sharing, the amount of data collected from the source data source can be compared with the amount of data in the standard database table corresponding to the source data source to determine whether the data collection process is abnormal. If so, data collection abnormality information is output; the amount of target library table data can be compared with the amount of data in the target data source to determine whether the data sharing process is abnormal. If so, data sharing abnormality information is output.

[0080] For example, during the data collection process, the amount of data collected from the source data source can be compared with the amount of data in the standard database table corresponding to the source data source after the data is collected. When the two amounts are inconsistent, it is determined that the data collection process is abnormal, and the data collection abnormality information is displayed on the screen of the data sharing platform. During the data sharing process, the amount of data in the target library table data can be compared with the amount of data in the target data source after the target library table data is encapsulated as the target data source. When the two amounts are inconsistent, it is determined that the data sharing process is abnormal, and the data sharing abnormality information is displayed on the screen of the data sharing platform.

[0081] The data collection abnormality information may include: prompt information of data collection abnormality and the reason of data collection abnormality, etc. The data sharing abnormality information may include: prompt information of data sharing abnormality and the reason of data sharing abnormality, etc.

[0082] Furthermore, when data collection or data sharing is abnormal, the data can be corrected based on the primary key field.

[0083] It is worth mentioning that in the embodiment of the present application, the data collection process and the data sharing process are independent of each other, and the two processes can be separated by a long period of time. After the data of the source data source is collected into the standard database, the standard database can store the data persistently, and then, when the data of the source data source is needed, the data can be encapsulated as a target data source for sharing, without the need for online sharing.

[0084] The data sharing method provided in the embodiment of the present application, by responding to a data collection request for a source data source, parsing the source data source, generating a data standard model corresponding to the source data source; based on the data standard model corresponding to the source data source, establishing a standard database table corresponding to the source data source in a standard database, and importing the data of the source data source into the standard database table; then, in response to a sharing request for target library table data in the standard database, encapsulating the target library table data as a target data source for sharing, can realize heterogeneous data sharing. Based on the data sharing method provided in the embodiment of the present application, the platforms of the two parties of data sharing only need to communicate with the data sharing platform, and there is no need for the platforms of the two parties of data sharing to communicate with each other.

[0085] The above is a data sharing method provided in an embodiment of the present application. Based on the same idea, an embodiment of the present application also provides a data sharing device, which can be applied to a data sharing platform. Figure 7 Schematic diagram of a data sharing device provided in an embodiment of the present application. Figure 7 As shown, the data sharing device includes the following modules: The standard model generation module 702 is used to analyze the source data source in response to a data collection request for the source data source, and generate a data standard model corresponding to the source data source; A data collection module 704 is used to establish a standard database table corresponding to the source data source in a standard database based on a data standard model corresponding to the source data source, and collect the data of the source data source into the standard database table; The data sharing module 706 is used to respond to a sharing request for the target library table data in the standard database and encapsulate the target library table data into a target data source for sharing.

[0086] In one implementation, the source data source includes: a source library table, a source file, a source interface or a source message, and the standard model generation module 702 is specifically used to: In the case where the source data source is a source library table, calling a library table parsing unit to parse the source library table and generate a data standard model corresponding to the source library table; In the case where the source data source is a source file, calling a file parsing unit to parse the source file and generate a data standard model corresponding to the source file; In the case where the source data source is a source interface, calling an interface parsing unit to parse the source interface and generate a data standard model corresponding to the source interface; In the case where the source data source is a source message, the message parsing unit is called to parse the source message to generate a data standard model corresponding to the source message.

[0087] In one implementation, the standard model generation module 702 is specifically used to: In the case where the source data source is a source library table, metadata exploration is performed on the source library table to obtain field metadata of the source library table; based on the field metadata of the source library table, a data standard model corresponding to the source library table is generated; In the case where the source data source is a source file, the source file is parsed based on the file type of the source file to obtain field metadata of the source file; based on the field metadata of the source file, a data standard model corresponding to the source file is generated; In the case where the source data source is a source interface, parsing the interface data of the source interface to obtain field metadata of the interface data; and generating a data standard model corresponding to the source interface based on the field metadata of the interface data; In the case where the source data source is a source message, the message body of the source message is parsed to obtain the field metadata of the message body; based on the field metadata of the message body, a data standard model corresponding to the source message is generated.

[0088] In one implementation manner, the target library table data includes: part of the library table data in one or more standard database tables specified in the standard database, or all of the library table data in one or more standard database tables specified in the standard database.

[0089] In one implementation manner, the target data source includes: a target library table, a target file, a target interface or a target message, and encapsulating the target library table data into a target data source for sharing includes: When the target data source requirement type is a library table, encapsulate the target library table data into a target library table, and send the target library table to the target database for sharing; When the target data source requirement type is a file, the target library table data is encapsulated as a target file, and the target file is sent to a target file server for sharing; When the target data source's demand type is a message, the target library table data is encapsulated as a target message, and the target message is pushed to the target topic for sharing; When the requirement type of the target data source is an interface, the target library table data is encapsulated as a target interface, and the call information of the target interface is output for sharing.

[0090] In one implementation, the source data source and the target data source are of different data source types.

[0091] In one implementation, the data sharing device further includes: The collection anomaly detection module is used to compare the amount of data collected from the source data source with the amount of data in the standard database table corresponding to the source data source to determine whether the data collection process is abnormal. If it is abnormal, output data collection anomaly information; The sharing anomaly detection module is used to compare the amount of data in the target library table with the amount of data in the target data source to determine whether the data sharing process is abnormal. If it is abnormal, it outputs data sharing anomaly information.

[0092] The data sharing device provided in the embodiment of the present application analyzes the source data source in response to a data collection request for the source data source, generates a data standard model corresponding to the source data source; based on the data standard model corresponding to the source data source, establishes a standard database table corresponding to the source data source in the standard database, and imports the data of the source data source into the standard database table; then, in response to a sharing request for target library table data in the standard database, encapsulates the target library table data as a target data source for sharing, and can realize heterogeneous data sharing. Based on the data sharing device provided in the embodiment of the present application, the platforms of the two parties of data sharing only need to communicate with the data sharing platform, and there is no need for the platforms of the two parties of data sharing to communicate with each other.

[0093] The above methods and devices provided in the embodiments of the present application can be applied to electronic devices of a data sharing platform. Figure 8 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device includes a processor 801 and a memory 802. The memory 802 stores a computer program that can be executed on the processor 801. When the computer program is executed by the processor 801, the various processes of the above-mentioned data sharing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0094] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned data sharing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0096] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0097] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0098] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0099] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0100] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0101] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0102] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0103] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data sharing method, characterized in that: Applied to a data sharing platform, the method comprises: In response to a data collection request for a source data source, the source data source is parsed to generate a data standard model corresponding to the source data source; Based on the data standard model corresponding to the source data source, a standard database table corresponding to the source data source is established in a standard database, and the data of the source data source is collected into the standard database table; In response to a sharing request for target library table data in the standard database, the target library table data is encapsulated as a target data source for sharing.

2. The data sharing method according to claim 1, characterized in that: The source data source includes: a source library table, a source file, a source interface or a source message. The source data source is parsed to obtain a data standard model corresponding to the source data source, including: In the case where the source data source is a source library table, calling a library table parsing unit to parse the source library table and generate a data standard model corresponding to the source library table; In the case where the source data source is a source file, calling a file parsing unit to parse the source file and generate a data standard model corresponding to the source file; In the case where the source data source is a source interface, calling an interface parsing unit to parse the source interface and generate a data standard model corresponding to the source interface; In the case where the source data source is a source message, the message parsing unit is called to parse the source message to generate a data standard model corresponding to the source message.

3. The data sharing method according to claim 1 or 2, characterized in that: In the case where the source data source is a source library table, the source library table is parsed to generate a data standard model corresponding to the source library table, including: Performing metadata exploration on the source library table to obtain field metadata of the source library table; generating a data standard model corresponding to the source library table based on the field metadata of the source library table; When the source data source is a source file, the source file is parsed to generate a data standard model corresponding to the source file, including: Parsing the source file based on the file type of the source file to obtain field metadata of the source file; generating a data standard model corresponding to the source file based on the field metadata of the source file; When the source data source is a source interface, the source interface is parsed to generate a data standard model corresponding to the source interface, including: Parsing the interface data of the source interface to obtain field metadata of the interface data; generating a data standard model corresponding to the source interface based on the field metadata of the interface data; When the source data source is a source message, the source message is parsed to generate a data standard model corresponding to the source message, including: The message body of the source message is parsed to obtain field metadata of the message body; based on the field metadata of the message body, a data standard model corresponding to the source message is generated.

4. The data sharing method according to claim 1, characterized in that: The target library table data includes: part of the library table data in one or more standard database tables in the standard database, or all of the library table data in one or more standard database tables in the standard database.

5. The data sharing method according to claim 1, characterized in that: The target data source includes: a target library table, a target file, a target interface or a target message, and the target library table data is encapsulated as a target data source for sharing, including: When the target data source requirement type is a library table, encapsulate the target library table data into a target library table, and send the target library table to the target database for sharing; When the target data source requirement type is a file, the target library table data is encapsulated into a target file, and the target file is sent to a target file server for sharing; When the target data source's demand type is a message, the target library table data is encapsulated as a target message, and the target message is pushed to the target topic for sharing; When the requirement type of the target data source is an interface, the target library table data is encapsulated as a target interface, and the call information of the target interface is output for sharing.

6. The data sharing method according to claim 1, characterized in that: The source data source and the target data source are of different data source types.

7. The data sharing method according to claim 1, characterized in that: The method further comprises: Compare the amount of data collected from the source data source with the amount of data in the standard database table corresponding to the source data source to determine whether the data collection process is abnormal, and if abnormal, output data collection abnormality information; Compare the amount of data in the target library table with the amount of data in the target data source to determine whether the data sharing process is abnormal. If abnormal, output data sharing abnormal information.

8. A data sharing device, characterized in that: Applied to a data sharing platform, the device comprises: A standard model generation module, configured to analyze the source data source in response to a data collection request for the source data source, and generate a data standard model corresponding to the source data source; A data collection module, used to establish a standard database table corresponding to the source data source in a standard database based on a data standard model corresponding to the source data source, and collect the data of the source data source into the standard database table; The data sharing module is used to respond to a sharing request for the target library table data in the standard database and encapsulate the target library table data as a target data source for sharing.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer-readable program to implement the data sharing method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the data sharing method according to any one of claims 1 to 7.

Images