Data processing method creation method and device, data information processing method and device

Abstract

The present application discloses a method and device for creating a data processing method, a method and device for processing data information, a computer storage medium and an electronic device. The creation method includes: obtaining label information, the label information is used to describe the attribute information of an entity or the attribute information of an entity relationship; outputting the label information to a configuration interface; receiving data processing conditions based on the configuration interface with the label information; saving the data processing conditions and the label information as call information during data processing; thereby enabling data mining and data analysis to be performed quickly and conveniently based on the existing label data, and generating target data.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and more particularly to a method and device for creating a data processing method. The present application also relates to a method and device for processing data information. Background Art

[0002] With the continuous development of the Internet, the era of big data has been formed, that is, data is presented in a massive and diversified form. In order to better analyze and understand data information, the existing technology can be used to label the data, thereby realizing data analysis and facilitating users to read the required data information.

[0003] The tag can be an attribute of an entity or an attribute of a relationship between entities. By labeling and modeling the user's data, the user's data can be presented in the form of tags, which can provide a view that is easier for users to understand and use, while providing functions such as fast tag data viewing, processing, statistical measurement, and service-oriented publishing.

[0004] However, after using labels to label user data for modeling, there are the following problems:

[0005] 1. The data of the original labels does not meet the application requirements of the new business scenarios, resulting in a reduction in the effectiveness of the original labels.

[0006] 2. The data of the original labels and the new labels generated by processing are not on the same data source, resulting in inaccurate data information acquisition. Summary of the invention

[0007] The present application provides a method for creating a data processing method to solve the problem in the prior art that the original labels cannot meet the needs of data analysis and use.

[0008] This application provides a method for creating a data processing method, including:

[0009] Acquire tag information, where the tag information is used to describe attribute information of an entity or attribute information of an entity relationship;

[0010] Outputting the tag information to a configuration interface;

[0011] Based on the configuration interface of the existing tag information, receiving data processing conditions;

[0012] The data processing condition and the tag information are saved as call information during data processing.

[0013] In some embodiments, obtaining tag information includes:

[0014] Select the data source to be processed from the available selectable data sources;

[0015] The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data source.

[0016] In some embodiments, it also includes:

[0017] Determine whether the data source has a data partition;

[0018] If yes, set the partition value of the data partition;

[0019] The acquiring the label information according to at least one of the entity information and the entity relationship information determined in the data source includes:

[0020] The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data partition corresponding to the partition value.

[0021] In some embodiments, it also includes:

[0022] A target data source is selected according to the tag information and the data processing condition.

[0023] In some embodiments, selecting a target data source according to the tag information and the data processing condition includes:

[0024] Preselecting the tag information desired to be obtained;

[0025] detecting the pre-selected tag information according to the data processing condition;

[0026] If the detection result is normal, the pre-selected label information is selected as the target data source.

[0027] In some embodiments, the step of saving the data processing condition and the tag information as call information during data processing includes:

[0028] Based on the selected target data source, determine whether to configure the target entity and target label;

[0029] If yes, configuring the target entity and target tag for the target data source;

[0030] The configured target data source and the data processing conditions are saved as call information for data processing.

[0031] In some embodiments, the configuration interface based on the existing tag information, receiving the data processing condition, includes:

[0032] According to the requirements of the target tag to be generated, determining the screening conditions based on the existing tag information on the configuration interface;

[0033] The screening condition is determined as the received data processing condition.

[0034] In some embodiments, the step of determining, on the configuration interface, a screening condition based on the existing tag information according to the requirement of generating the target tag includes:

[0035] According to the requirement of the target tag to be generated, a screening condition based on the existing tag information is selected or set.

[0036] In some embodiments, outputting the tag information to a configuration interface includes:

[0037] By moving the tag information, the tag information is output to the configuration interface.

[0038] In some embodiments, it also includes:

[0039] Converting the data processing condition and the tag information into a tag query statement;

[0040] The step of saving the data processing condition and the tag information as call information during data processing includes:

[0041] The data processing condition and the label query statement corresponding to the label information are saved as call information during data processing.

[0042] The present application also provides a device for creating a data processing method, including:

[0043] An acquisition unit, used to acquire label information, where the label information is used to describe attribute information of an entity or attribute information of an entity relationship;

[0044] An output unit, used to output the tag information to a configuration interface;

[0045] A receiving unit, configured to receive data processing conditions based on a configuration interface of the existing tag information;

[0046] The storage unit is used to store the data processing conditions and the label information as call information during data processing.

[0047] The present application also provides a method for processing data information, including:

[0048] Creating call information for data processing based on tag information and data processing conditions;

[0049] Generate a target table for storing processed data according to the parameters recorded in the submitted call information;

[0050] The calling information is called, data processing is performed on the tag data to be processed, and the processed data is stored in the target table.

[0051] In some embodiments, the creation of the call information for performing data processing based on the tag information and the data processing condition includes:

[0052] Acquire tag information, where the tag information is used to describe attribute information of an entity or attribute information of an entity relationship;

[0053] Outputting the tag information to a configuration interface;

[0054] Based on the configuration interface of the existing tag information, receiving data processing conditions;

[0055] The data processing condition and the tag information are saved as call information during data processing.

[0056] In some embodiments, obtaining tag information includes:

[0057] Select the data source to be processed from the available selectable data sources;

[0058] The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data source.

[0059] In some embodiments, it also includes:

[0060] Determine whether the data source has a data partition;

[0061] If yes, set the partition value of the data partition;

[0062] The acquiring the label information according to at least one of the entity information and the entity relationship information determined in the data source includes:

[0063] The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data partition corresponding to the partition value.

[0064] In some embodiments, it also includes:

[0065] A target data source is selected according to the tag information and the data processing condition.

[0066] In some embodiments, selecting a target data source according to the tag information and the data processing condition includes:

[0067] Preselecting the tag information desired to be obtained;

[0068] detecting the pre-selected tag information according to the data processing condition;

[0069] If the detection result is normal, the pre-selected label information is selected as the target data source.

[0070] In some embodiments, the step of saving the data processing condition and the tag information as call information during data processing includes:

[0071] Based on the selected target data source, determine whether to configure the target entity and target label;

[0072] If yes, configuring the target entity and target tag for the target data source;

[0073] The configured target data source and the data processing conditions are saved as call information for data processing.

[0074] In some embodiments, the configuration interface based on the existing tag information, receiving the data processing condition, includes:

[0075] According to the requirements of the target tag to be generated, determining the screening conditions based on the existing tag information on the configuration interface;

[0076] The screening condition is determined as the data processing condition.

[0077] In some embodiments, the step of determining, on the configuration interface, a screening condition based on the existing tag information according to the requirement of generating the target tag includes:

[0078] According to the requirement of the target tag to be generated, a screening condition based on the existing tag information is selected or set.

[0079] In some embodiments, outputting the tag information to a configuration interface includes:

[0080] By moving the tag information, the tag information is output to the configuration interface.

[0081] In some embodiments, it also includes:

[0082] Converting the data processing condition and the tag information into a tag query statement;

[0083] The step of saving the data processing condition and the tag information as call information during data processing includes:

[0084] The data processing condition and the label query statement corresponding to the label information are saved as call information during data processing.

[0085] In some embodiments, generating a target table for storing processed data according to the parameters recorded in the submitted call information further includes:

[0086] Determine whether the connection path of the data source and the connection path of the target data source in the call information are normal;

[0087] If normal, a target table for storing processed data is generated according to the parameters recorded in the calling information.

[0088] In some embodiments, generating a target table for storing processed data according to the parameters recorded in the submitted call information further includes:

[0089] Determining whether the target table is bound to the processed data;

[0090] If so, the target table is bound to at least one of the processed target entity information and target entity relationship information; and the table column of the target table is bound to the processed target label.

[0091] In some embodiments, the calling of the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table further includes:

[0092] When processing the tag data to be processed, determining whether a data processing cycle is configured;

[0093] If so, data processing is performed on the tag data to be processed according to the configured data processing cycle, and the processed data is stored in the target table.

[0094] In some embodiments, the calling of the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table further includes:

[0095] Obtain at least one of entity data and entity relationship data to be processed, and label data;

[0096] Importing at least one of the entity data and entity relationship data, and the label data into a target data source;

[0097] According to calling the calling information, data processing is performed on at least one of the entity data and entity relationship data imported from the target data source, and the label data, and the processed data is stored in the target table.

[0098] In some embodiments, it also includes:

[0099] Detecting a connection path of at least one of the acquired entity data to be processed and the entity relationship data, and a common data source of the label data;

[0100] If the detection is normal, the calling information is called, data processing is performed on the tag data to be processed, and the processed data is stored in the target table.

[0101] In some embodiments, the calling of the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table further includes:

[0102] According to the binding relationship determined between the target table and the target tag, the target table is output on the configuration interface or the target tag in the target table is output.

[0103] The present application also provides a data information processing device, including:

[0104] A creation unit, used for creating call information for calling when performing data processing based on tag information and data processing conditions;

[0105] A generating unit, used for generating a target table for storing processed data according to the parameters recorded in the submitted calling information;

[0106] The processing unit is used to call the calling information, perform data processing on the tag data to be processed, and store the processed data in the target table.

[0107] The present application also provides a computer storage medium for storing data generated by a network platform, and a program for processing the data generated by the network platform;

[0108] When the program is read and executed, the program executes the steps of the method for creating the data processing mode as described above; or executes the steps of the method for processing the data information as described above.

[0109] The present application also provides an electronic device, comprising:

[0110] processor;

[0111] The memory is used to store a program for processing data generated by a network platform. When the program is read and executed by the processor, it executes the steps of the method for creating the data processing method as described above; or executes the steps of the method for processing data information as described above.

[0112] Compared with the prior art, this application has the following advantages:

[0113] The present application provides a method for creating a data processing method, which outputs the acquired tag information to a configuration interface, receives data processing conditions based on the configuration interface of the existing tag information, saves the data processing conditions and the tag information as call information during data processing, and calls the call information to conveniently and quickly perform data mining and data analysis based on the existing tag data, and generate target data. The target data can be at least one of target entity information, target entity relationship information, and target tag information, or only generate a physical table of the target data.

[0114] In addition, the present application also provides a method for processing data information, by creating call information that is called when data processing is performed based on label information and data processing conditions, and generating a target table for storing processed data according to the parameters recorded in the submitted call information; calling the call information, performing data processing on the label data to be processed, and storing the processed data in the target table; thereby completing the mining and generation of target data based on the created data processing method, improving the full utilization of source data and existing label information, and being able to ensure that the generated target data comes from the same target data source, and in addition, adopting a mobile operation method to output the existing label information to the configuration interface, so that the target data generation process is simple and fast to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] Figure 1 It is a flow chart of an embodiment of a method for creating a data processing method provided by the present application;

[0116] Figure 2 It is a structural schematic diagram of an embodiment of a creation device of a data processing method provided by the present application;

[0117] Figure 3 It is a flow chart of an embodiment of a method for processing data information provided by the present application;

[0118] Figure 4 It is a structural diagram of an embodiment of a data information processing device provided by the present application. DETAILED DESCRIPTION

[0119] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0120] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The descriptions used in this application and the appended claims, such as "a", "first", and "second", are not limitations on quantity or sequence, but are used to distinguish the same type of information from each other.

[0121] Based on the description of the background technology part, in order to better understand the creation method of the data processing method provided by this application, the background of the technical solution of this application is now explained through application scenarios. For example: based on the scenario where data is generated by commodity transactions, the attributes of the relationship between the shopping platform and the customer can be extracted as labels by analyzing the transaction data generated by the shopping platform, such as: the data of the top 10 customers with the highest purchasing power on the shopping platform or the consumers who are female; commodity attributes can also be extracted as labels, such as: the commodities with the most consumption times, etc., and these data are presented to users in the form of labels. For users, they only need to understand the content displayed by the labels, and there is no need to display the underlying physical data information. It can be seen that the label is the attribute information of the entity or the attribute information of the entity relationship. By establishing a label system for data information, the analysis needs of data in different business scenarios can be met. For example: analysis of commodity data information, analysis of medical data information, analysis of traffic data information, analysis of financial data information, etc., the data that users are concerned about can be directly obtained through labels.

[0122] However, when the existing label content can no longer meet the user's needs, the target label can be generated by reprocessing the existing label (original label) content. The purpose of the method for creating a data processing method provided in the present application is to create a data information processing process that can form a target label by reprocessing the data information based on the existing labels. By calling the data information processing process, the target label can be generated based on the existing labels and entity data information to meet the user's needs. It can be seen that the method for creating a data processing method provided in the present application is to create a processing process for the target label derived from the existing labels.

[0123] The following is a detailed description of a method for creating a data processing method provided by this application. It should be noted that when describing the creation method, an example of an application scenario will be used to provide a better understanding of this solution. However, the creation method provided by this application is not limited to the application scenario in the example. Please refer to Figure 1 As shown, Figure 1 It is a flowchart of an embodiment of a method for creating a data processing method provided by the present application, and the method for creating includes:

[0124] Step S101: Acquire tag information, where the tag information is used to describe attribute information of an entity or attribute information of an entity relationship.

[0125] The purpose of step S101 is to obtain the existing tag information. In this embodiment, the tag information can be obtained through a data source, and therefore includes:

[0126] Select the data source to be processed from the available selectable data sources;

[0127] The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data source.

[0128] The so-called data source refers to a database that stores data. Different types of data or different data contents may be stored in different databases. Therefore, it is necessary to select the data source that needs to be processed from the selectable data sources. The data source that needs to be processed here can be understood as the data source that needs to be processed again.

[0129] The data source may be a database for storing data, so the data source that needs to be processed should be selected first to provide a basis for subsequent data processing. In this embodiment, the data source can store the storage and calculation data information of batch structured data, and can realize the import and output of data.

[0130] Based on the above, it can be understood that for the selected data source, there may be data partitions in various dimensions, such as data partitions divided by geographical dimensions or data partitions divided by time dimensions, etc. Data partitioning is a physical database design technology, which aims to reduce the total amount of data read and write in specific SQL operations to reduce response time. Data partitioning does not generate a new data table, but distributes the data in the data table to different hard disks, systems or different server storage media according to dimensions or needs, so as to improve the efficiency of data retrieval and reduce the frequent IO pressure value of the database.

[0131] Therefore, it also includes:

[0132] Determine whether the data source has a data partition;

[0133] If yes, set the partition value of the data partition;

[0134] The acquiring the label information according to at least one of the entity information and the entity relationship information determined in the data source includes:

[0135] The label information is obtained according to at least one of entity information, entity relationship information, and label custom function information included in the data partition corresponding to the partition value. After obtaining the label information, the data involved in the label information needs to be processed again to generate target data. The target data may include at least one of target label, target entity, and target entity relationship. Therefore, the target data source can be selected according to the label information and the data processing condition.

[0136] Based on the selected data source, entity information, entity relationship information and original label information are determined, and then a target data source is selected based on at least one of the entity information, entity relationship information and label information, and data processing conditions.

[0137] In this embodiment, the target data source may be selected as follows:

[0138] The step S101-1: determines the original entity information and the original entity relationship information according to the data source; for example: based on the commodity information, merchant information and customer information provided in the commodity trading platform as the original entity information, that is, the entity can be a merchant, a customer and a commodity; the relationship information between the merchant and the customer is used as the entity relationship information, the relationship information between the commodity and the merchant can also be used as the entity relationship information, and the relationship information between the commodity and the customer can be used as the entity relationship information.

[0139] Step S101 - 2: according to the determined entity information and entity relationship information, obtaining the tag information in a tag list describing the attributes of the entity information or the attributes of the entity relationship information;

[0140] The entity information and the relationship information between the entity information in step S101-2 can be stored in the database in the form of a data table. Based on the example in step S102-1, the label information can be based on at least one of the attributes of the product, the merchant, the customer, the attributes of the relationship between the merchant and the customer, and the attributes between the product and the customer in the data table. The label information generated based on the entity information and the entity relationship information is obtained by using the label column (also called the attribute column) in the data table.

[0141] Therefore, it can be understood that the data processing method in the creation method of the data processing method provided by the present application can be created based on a visual operation interface, and the target data source is selected through the user's operation behavior. In this embodiment, it is explained in the form of a drag operation, and other operation behaviors such as voice operation, click operation, etc. are not excluded. Of course, the original entity relationship information and the original label information can also be edited in a label statement editing manner to obtain the editing result, and the target data source is selected according to the editing result. The label statement can be TQL: tag query language, a label query statement, a statement consisting of entity relationships and labels, and the writing form is similar to SQL, wherein the entity relationship is similar to a table in SQL, and the label is similar to a field in a table in SQL. In other words, the selection of the target data source can be implemented based on a visual interface or through a label query statement.

[0142] Step S102 - 3 : Select a target data source according to at least one of the entity information and entity relationship information included in the tag information and the data processing condition.

[0143] Steps S101-1 to S101-3 are mainly used to describe the process of how to select a target data source. This is because, in order to process the data again to form a target label, it is necessary to base it on the source data information that you want to analyze or reference. Therefore, it is necessary to select the target data source on the selected data source based on at least one of the entity information and entity relationship information included in the already formed label information and data processing conditions, and then process the target label (new label) for the target data source.

[0144] In this embodiment, the acquisition of label information can also be based on the determined entity information and entity relationship information, and the label custom function can be obtained from the custom label processing function list that describes the entity information attributes or the entity relationship information attributes; that is: obtaining the UDF function in the UDF (User Defined Function) label custom function list to obtain the label information.

[0145] Therefore, the target data source may also be selected based on at least one of the entity information, entity relationship information, and UDF function information included in the tag information, and the data processing condition.

[0146] Since the selected target data source may not be the required target data source, the selected target data source needs to be tested. Therefore, the target data source is selected according to the tag information and the data processing condition, including:

[0147] Preselecting the tag information desired to be obtained;

[0148] detecting the pre-selected tag information according to the data processing condition;

[0149] If the detection result is normal, the pre-selected label information is selected as the target data source.

[0150] If the detection result is wrong, the label information is obtained again.

[0151] Of course, it is understandable that when the original entity relationship information and the original label information are edited by using the label statement editing method, obtaining the editing result may also include testing the written TQL statement. If the test result is correct, it means that there is no problem with the writing, and the target data source can be selected according to the TQL statement. If the test result is wrong, the TQL statement needs to be rewritten to ensure the accuracy of the subsequent selected target data source.

[0152] Step S102: output the tag information to the configuration interface.

[0153] The purpose of S102 is to display the acquired tag information on the configuration interface for the user to view. The configuration interface can be understood as outputting part of the parameter information of the processed data on a visual operation interface before obtaining the processing result during the data processing process, and the acquired tag information can be output on this interface. The tag information output on the configuration interface can be completed by moving the tag information, such as voice operation, click operation, etc. In this embodiment, the tag information can be dragged to the configuration interface in the form of a drag operation.

[0154] It can be understood that the target data source can also be determined by the above operation of dragging the tag information.

[0155] Step S103: receiving data processing conditions based on the configuration interface of the existing tag information.

[0156] The data processing condition in step S103 can be understood as configuration information for generating a target tag based on the configuration of the target data source. Therefore, step S103 may include:

[0157] Step S103-1: according to the requirement of the target tag to be generated, determining the screening condition based on the existing tag information on the configuration interface;

[0158] The screening condition in step S103-1 can be selected through existing screening conditions, or can be set when it is necessary to generate a derived target tag. The so-called setting is to configure the screening condition, for example: the target data source is the transaction data of the first quarter, and the demand for the derived target tag to be generated can be based on the label that the customer is a female in the transaction data of the first quarter in the target data source, and the configured screening condition can be that the consumer is a female customer.

[0159] Step S103-2: Determine the screening condition as the data processing condition.

[0160] Continuing with the example in step S103-1, the screening condition can be the condition that the consumer meets the female requirement, which can be determined as the data processing condition. In the subsequent data processing process, the screening condition needs to be met, and the data describing the attribute information in the data that meets the screening condition is the target label.

[0161] Based on the above, after the target data source is selected in step S101, step S103 may include:

[0162] Based on the selected target data source, determine whether to configure the target entity and target label;

[0163] If yes, configuring the target entity and target tag for the target data source;

[0164] The configured target data source and the data processing conditions are saved as call information for data processing.

[0165] Step S104: Save the data processing conditions and the tag information as call information for data processing.

[0166] The purpose of step S104 is to determine the data processing method when generating the target label based on the label information obtained in the above steps S101 to S103 and the received data processing conditions. That is to say, when it is necessary to generate a target label for the original label, the target label can be generated by calling the process described in the above steps S101 to S103.

[0167] Therefore, the above steps S101 to S104 are only data processing methods for creating the target label to be generated, which can also be called a processing or generation process. When generating the target label, the data processing method can be used to guide the user to select the required original label based on the target data source through operations such as dragging and dropping, and after the dragging operation, the target label can be generated based on the original label and data processing conditions, so that the target label generation process is easy to operate, and can generate more meaningful target labels that meet user needs based on the original labels.

[0168] To facilitate user operations, the above data processing method can be based on a visual interface to complete corresponding operations. During the data processing process, the call information is called based on the visual interface, so that the data can be quickly mined and analyzed.

[0169] It should be noted that in the above steps S101-S104, the creation of the data processing method can be completed based on the configuration interface (visual interface) according to the requirements of the original tag and the generated target tag, so that the user can quickly operate the original tag to generate the target data. In order to provide different operation requirements for different user groups, it can also include:

[0170] Converting the data processing condition and the tag information into a tag query language (TQL);

[0171] During the implementation of step S104, the data processing condition and the tag query statement corresponding to the tag information may be saved as call information during data processing.

[0172] The TQL statement can be understood as a statement composed of entity relationships and labels, and its writing form is similar to SQL (database query statement). The entity relationship can be regarded as a table in SQL, and the label can be regarded as a field of the table in SQL.

[0173] The above is a detailed description of an embodiment of a method for creating a data processing method provided by the present application. Corresponding to the embodiment of the method for creating a data processing method provided above, the present application also discloses an embodiment of a device for creating a data processing method. Figure 2 Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0174] like Figure 2 As shown, Figure 2 : is a structural schematic diagram of an embodiment of a device for creating a data processing method provided by the present application, the device comprising:

[0175] An acquisition unit 201 is used to acquire tag information, where the tag information is used to describe attribute information of an entity or attribute information of an entity relationship;

[0176] An output unit 202, used to output the tag information to a configuration interface;

[0177] A receiving unit 203, configured to receive data processing conditions based on a configuration interface of the existing tag information;

[0178] The saving unit 204 is used to save the data processing condition and the tag information as calling information during data processing.

[0179] The acquisition unit 201 includes:

[0180] A selection subunit is used to select a data source to be processed from the provided selectable data sources;

[0181] The acquisition subunit is used to acquire the label information according to at least one of the entity information, entity relationship information, and label custom function information included in the data source.

[0182] In an embodiment of the creation device of the data processing method provided by the present application, the invention further includes:

[0183] A partition determination unit, used to determine whether a data partition exists in the data source;

[0184] The partition value setting unit is used to set the partition value of the data partition if the partition determination unit determines that there is a data partition.

[0185] The acquisition subunit is specifically used to acquire the label information according to at least one of entity information, entity relationship information, and label user-defined function information included in the data partition corresponding to the partition value.

[0186] The output unit 202 includes:

[0187] A moving subunit, used for performing a moving operation on the tag information;

[0188] The output subunit is used to output the label information to the configuration interface according to the movement operation of the label information in the movement subunit.

[0189] The specific implementation process of the receiving unit 203 may include:

[0190] A condition determination subunit, used to determine, on the configuration interface, a screening condition based on the existing tag information according to the requirements of the target tag to be generated;

[0191] The receiving subunit is used to determine the screening condition as the received data processing condition.

[0192] The condition determination subunit is specifically used to select or set a screening condition based on the existing tag information according to the requirement of generating the target tag.

[0193] The creation device further includes:

[0194] The target source selection unit is used to select a target data source according to the tag information and the data processing condition.

[0195] The target source determination unit comprises:

[0196] A preselection subunit, used for preselecting the tag information desired to be obtained;

[0197] A detection subunit, used for detecting the pre-selected tag information according to the data processing condition;

[0198] The selection subunit is used to select the pre-selected label information as the target data source if the detection result is normal.

[0199] If the detection result is an error, the process may return to the acquisition unit 201 to reacquire the tag data.

[0200] The storage unit 204 may include:

[0201] A configuration determination subunit, used to determine whether to configure a target entity and a target tag based on a selected target data source;

[0202] a configuration subunit, configured to configure a target entity and a target tag for the target data source when the determination result of the configuration determination subunit is yes;

[0203] The saving subunit is used to save the configured target data source and the data processing conditions as call information during data processing.

[0204] Based on the above, it also includes:

[0205] A conversion unit, used for converting the data processing condition and the tag information into a tag query statement;

[0206] The storage unit is specifically used to store the data processing condition and the label query statement corresponding to the label information as call information during data processing.

[0207] The above is a description of an embodiment of a device for creating a data processing method provided in the present application. For specific contents of the device, reference may be made to the description of steps S101 to S104 in the above embodiment of a method for creating a data processing method.

[0208] The description of the above-mentioned creation method embodiment based on the data processing method and the creation device embodiment of the data processing method is to explain the provision of a processing generation method for generating target tags, which can also be understood as providing a processing generation framework or mode, in which the target tag data information can be implemented, that is, the target tag to be generated is output by calling the data processing method for user reference. Therefore, the present application also provides a data information processing method, such as Figure 3 As shown, Figure 3This is a flowchart of an embodiment of a data information processing method provided by the present application, and the processing method includes:

[0209] Step S301: creating call information for performing data processing based on tag information and data processing conditions;

[0210] The purpose of step S301 is to create a data processing method for a target tag or target relationship derived (or generated) from the original entity information, the original entity relationship information, and the original tag information. The specific data processing method has been described in the above steps S101 to S104, and will not be repeated here. For the specific implementation process of step S301, please refer to the above steps S101 to S104.

[0211] The step S301 may also be understood as providing preparation conditions for generating target tags or target relationship data information.

[0212] Step S302: Generate a target table for storing processed data according to the parameters recorded in the submitted call information.

[0213] The step S302 is to generate a target table for storing processed data based on the created data processing method. Before generating the target table for storing processed data, the data source and target data source in the created data processing method may be tested to check whether the data connection path is normal. Of course, the UDF function may also be tested.

[0214] If the above tests are all normal, a target table for storing the processed data is generated based on the parameters involved in the created data processing method, such as the selected target data source, data processing conditions, etc.

[0215] To meet the different needs of different users, for example: for technical development users, they can obtain technical-level target data information through data processing, that is, the physical table of target data information; for business analysis users, they can obtain at least one of the target realization relationship information and target label information that can be visualized through the data processing process.

[0216] Therefore, after generating the target table, you can also include:

[0217] Determining whether the target table is bound to the processed data;

[0218] If so, the target table is bound to at least one of the target entity information and the target entity relationship information involved in the target data source; and the table column of the target table is bound to the target label involved in the target data source.

[0219] Based on the binding result, at least one of corresponding target entity information, target entity relationship information, and target tag information is provided to the user.

[0220] When it is determined that the binding result of the target table and the processed data is no, a corresponding target data physical table is provided to the user.

[0221] Step S303: calling the calling information, performing data processing on the tag data to be processed, and storing the processed data into the target table.

[0222] The specific implementation process of step S303 may include:

[0223] Converting the data processing conditions in the data processing method into structured query language, i.e., SQL;

[0224] Based on the structured query language, data in the target data source is processed, and the processed data is stored in the target table.

[0225] The purpose of step S303 is to extract the data satisfying the data processing conditions from the target data source based on the execution of steps S301 and S302, store them in the target table, and output them in a visual form for user reference and analysis.

[0226] It is understandable that in step S303, connectivity detection may also be performed on the target data source. When a UDF function exists, detection of the UDF function may also be included. The data processing method created in step S301 may also be detected. There is no specific limitation on the step in which the detection occurs, and it is sufficient to ensure that the data processing method and related data information and paths are normal.

[0227] The step S303 in the data processing mode calling process may also include:

[0228] When processing the data in the target data source according to the data processing conditions in the data processing method, whether to process the data according to the configured processing cycle;

[0229] If so, data processing is performed on the data in the target data source according to the configured processing cycle.

[0230] If not, the data is processed according to the configured one-time processing method, and the processing process can generate processing SQL statements.

[0231] Finally, the processed data can be stored in a target table and output and displayed in a visual form, for example, at least one of the entity relationship information, entity information, and label information derived from the target table can be displayed.

[0232] The above is a detailed description of an embodiment of a data information processing method provided by the present application. Corresponding to the aforementioned embodiment of a data information processing method, the present application also discloses an embodiment of a data information processing device. Figure 4 Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.

[0233] like Figure 4 As shown, Figure 4 1 is a schematic diagram of a data information processing device embodiment provided by the present application. The processing device includes:

[0234] A creation unit 401, used to create call information for calling when performing data processing based on tag information and data processing conditions;

[0235] The creation unit 401 can refer to the description of the creation device for the data processing process mentioned above, which will not be repeated here. In the embodiment of the data information processing device provided in the present application, the creation of the data processing method is the same as the creation method and creation device of the above-mentioned data processing method.

[0236] A generating unit 402, configured to generate a target table for storing processed data according to the parameters recorded in the submitted calling information;

[0237] In this embodiment, the following may also be included:

[0238] A connection path determination unit, used to determine whether the connection path of the data source and the connection path of the target data source are normal;

[0239] The generating unit may be specifically configured to generate a target table for storing processed data according to the parameters recorded in the data processing method when the result determined by the connected path determining unit is normal.

[0240] In this implementation, it may also include:

[0241] A binding determination unit, used to determine whether the target table is bound to the processed data;

[0242] A binding unit is used to bind the target table with at least one of the target entity information and target entity relationship information involved in the target data source when the determination result in the binding determination unit is yes; and to bind the table column of the target table with the target label involved in the target data source.

[0243] The processing unit 403 is used to call the calling information, perform data processing on the tag data to be processed, and store the processed data into the target table.

[0244] The processing unit 403 includes:

[0245] A conversion subunit, used for converting the data processing conditions in the data processing method into a structured query language;

[0246] The processing subunit is used to process the data in the target data source based on the structured query language, and store the processed data in the target table.

[0247] In this embodiment, it also includes:

[0248] A processing cycle determining unit, configured to determine whether to perform data processing according to a configured processing cycle when performing data processing on the data in the target data source according to the data processing conditions in the data processing mode;

[0249] The processing unit is specifically configured to process the data in the target data source according to the configured processing cycle when the determination result of the processing cycle determination unit is yes.

[0250] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0251] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0252] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0253] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0254] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A method for creating a data processing method, characterized in that: include: Acquire tag information, where the tag information is used to describe attribute information of an entity or attribute information of an entity relationship; Outputting the tag information to a configuration interface; Based on the configuration interface of the existing tag information, receiving data processing conditions; Saving the data processing conditions and the tag information as call information during data processing; The obtaining of label information includes: Select the data source to be processed from the available selectable data sources; The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data source.

2. The method for creating a data processing method according to claim 1, characterized in that: Also includes: Determine whether the data source has a data partition; If yes, set the partition value of the data partition; The acquiring the label information according to at least one of the entity information and the entity relationship information determined in the data source includes: The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data partition corresponding to the partition value.

3. The method for creating a data processing method according to claim 1, characterized in that: Also includes: A target data source is selected according to the tag information and the data processing condition.

4. The method for creating a data processing method according to claim 3, characterized in that: The selecting a target data source according to the tag information and the data processing condition includes: Preselecting the tag information desired to be obtained; detecting the pre-selected tag information according to the data processing condition; If the detection result is normal, the pre-selected label information is selected as the target data source.

5. The method for creating a data processing method according to claim 4, characterized in that: The step of saving the data processing condition and the tag information as call information during data processing includes: Based on the selected target data source, determine whether to configure the target entity and target label; If yes, configuring the target entity and target tag for the target data source; The configured target data source and the data processing conditions are saved as call information for data processing.

6. The method for creating a data processing method according to claim 1, characterized in that: The configuration interface based on the existing tag information receives data processing conditions, including: According to the requirements of the target tag to be generated, determining the screening conditions based on the existing tag information on the configuration interface; The screening condition is determined as the received data processing condition.

7. The method for creating a data processing method according to claim 6, characterized in that: Determining, on the configuration interface, a screening condition based on the existing tag information according to the requirement of the target tag to be generated includes: According to the requirement of the target tag to be generated, a screening condition based on the existing tag information is selected or set.

8. The method for creating a data processing method according to claim 1, characterized in that: The step of outputting the tag information to a configuration interface includes: The tag information is output to the configuration interface by moving the tag information; the moving operation is a drag operation.

9. The method for creating a data processing method according to claim 1, characterized in that: Also includes: Converting the data processing condition and the tag information into a tag query statement; The step of saving the data processing condition and the tag information as call information during data processing includes: The data processing condition and the label query statement corresponding to the label information are saved as call information during data processing.

10. A device for creating a data processing method, characterized in that: include: An acquisition unit, used to acquire label information, where the label information is used to describe attribute information of an entity or attribute information of an entity relationship; An output unit, used to output the tag information to a configuration interface; A receiving unit, configured to receive data processing conditions based on a configuration interface of the existing tag information; A storage unit, used for storing the data processing conditions and the label information as call information during data processing; When acquiring label information, the acquisition unit is specifically used to select a data source to be processed from the provided selectable data sources; and acquire the label information based on at least one of entity information, entity relationship information, and label custom function information included in the data source.

11. A method for processing data information, characterized in that: include: Creating call information for data processing based on tag information and data processing conditions; Generate a target table for storing processed data according to the parameters recorded in the submitted call information; Calling the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table; The creation of the call information for data processing based on the tag information and the data processing condition includes: Select the data source to be processed from the available selectable data sources; According to at least one of the entity information, entity relationship information, and label self-defined function information included in the data source, label information is acquired, where the label information is used to describe attribute information of the entity or attribute information of the entity relationship; Outputting the tag information to a configuration interface; Based on the configuration interface of the existing tag information, receiving data processing conditions; The data processing condition and the tag information are saved as call information during data processing.

12. The method for processing data information according to claim 11, characterized in that: Also includes: Determine whether the data source has a data partition; If yes, set the partition value of the data partition; The acquiring the label information according to at least one of the entity information and the entity relationship information determined in the data source includes: The label information is obtained according to at least one of entity information, entity relationship information, and label user-defined function information included in the data partition corresponding to the partition value.

13. The method for processing data information according to claim 11, characterized in that: Also includes: A target data source is selected according to the tag information and the data processing condition.

14. The method for processing data information according to claim 13, characterized in that: The selecting a target data source according to the tag information and the data processing condition includes: Preselecting the tag information desired to be obtained; detecting the pre-selected tag information according to the data processing condition; If the detection result is normal, the pre-selected label information is selected as the target data source.

15. The method for processing data information according to claim 14, characterized in that: The step of saving the data processing condition and the tag information as call information during data processing includes: Based on the selected target data source, determine whether to configure the target entity and target label; If yes, configuring the target entity and target tag for the target data source; The configured target data source and the data processing conditions are saved as call information for data processing.

16. The method for processing data information according to claim 11, characterized in that: The configuration interface based on the existing tag information receives data processing conditions, including: According to the requirements of the target tag to be generated, determining the screening conditions based on the existing tag information on the configuration interface; The screening condition is determined as the data processing condition.

17. The method for processing data information according to claim 16, characterized in that: Determining, on the configuration interface, a screening condition based on the existing tag information according to the requirement of the target tag to be generated includes: According to the requirement of the target tag to be generated, a screening condition based on the existing tag information is selected or set.

18. The method for processing data information according to claim 11, characterized in that: The step of outputting the tag information to a configuration interface includes: The tag information is output to the configuration interface by moving the tag information; the moving operation is a drag operation.

19. The method for processing data information according to claim 11, characterized in that: Also includes: Converting the data processing condition and the tag information into a tag query statement; The step of saving the data processing condition and the tag information as call information during data processing includes: The data processing condition and the label query statement corresponding to the label information are saved as call information during data processing.

20. The method for processing data information according to claim 11, characterized in that: The step of generating a target table for storing processed data according to the parameters recorded in the submitted call information also includes: Determine whether the connection path of the data source and the connection path of the target data source in the call information are normal; If normal, a target table for storing processed data is generated according to the parameters recorded in the calling information.

21. The method for processing data information according to claim 11, characterized in that: The step of generating a target table for storing processed data according to the parameters recorded in the submitted call information also includes: Determining whether the target table is bound to the processed data; If so, the target table is bound to at least one of the processed target entity information and target entity relationship information; and the table column of the target table is bound to the processed target label.

22. The method for processing data information according to claim 11, characterized in that: The calling information is called, the tag data to be processed is processed, and the processed data is stored in the target table, and further includes: When processing the tag data to be processed, determining whether a data processing cycle is configured; If so, data processing is performed on the tag data to be processed according to the configured data processing cycle, and the processed data is stored in the target table.

23. The method for processing data information according to claim 11, characterized in that: The calling information is called, the tag data to be processed is processed, and the processed data is stored in the target table, and further includes: Obtain at least one of entity data and entity relationship data to be processed, and label data; Importing at least one of the entity data and entity relationship data, and the label data into a target data source; According to calling the calling information, data processing is performed on at least one of the entity data and entity relationship data imported from the target data source, and the label data, and the processed data is stored in the target table.

24. The method for processing data information according to claim 23, characterized in that: Also includes: Detecting a connection path of at least one of the acquired entity data to be processed and the entity relationship data, and a common data source of the label data; If the detection is normal, the calling information is called, data processing is performed on the tag data to be processed, and the processed data is stored in the target table.

25. The method for processing data information according to claim 11, characterized in that: The calling information is called, the tag data to be processed is processed, and the processed data is stored in the target table, and further includes: According to the binding relationship determined between the target table and the target tag, the target table is output on the configuration interface or the target tag in the target table is output.

26. A data information processing device, characterized in that: include: A creation unit, used for creating call information for calling when performing data processing based on tag information and data processing conditions; A generating unit, used for generating a target table for storing processed data according to the parameters recorded in the submitted calling information; A processing unit, used for calling the calling information, performing data processing on the tag data to be processed, and storing the processed data in the target table; The creation unit is specifically used to select a data source to be processed from the provided selectable data sources; According to at least one of the entity information, entity relationship information, and label self-defined function information included in the data source, label information is acquired, where the label information is used to describe attribute information of the entity or attribute information of the entity relationship; Outputting the tag information to a configuration interface; Based on the configuration interface of the existing tag information, receiving data processing conditions; The data processing condition and the tag information are saved as call information during data processing.

27. A computer storage medium for storing data generated by a network platform and a program for processing the data generated by the network platform; When the program is read and executed, it executes the steps of the method for creating a data processing method as described in any one of claims 1 to 9; or executes the steps of the method for processing data information as described in any one of claims 11 to 25.

28. An electronic device comprising: processor; A memory for storing a program for processing data generated by a network platform, wherein when the program is read and executed by the processor, the program executes the steps of a method for creating a data processing method as described in any one of claims 1 to 9; or executes the steps of a method for processing data information as described in any one of claims 11 to 25.

Images