Data processing method and device based on data form and electronic equipment

Abstract

The embodiment of the invention relates to a data processing method and device based on a data form and electronic equipment. The data form is a second SDK comprising a first SDK and data form configuration information, the first SDK is used for data acquisition, and the data form configuration information is a set of definition information related to data processing. The data processing method comprises the steps that at least one data form is stored in a data form library in advance, and first SDKs and / or data form configuration information of different data forms are different; receiving a selection operation of selecting a specified data form from the data form library by a user, and receiving information of a specified service system input by the user; and connecting the specified data form with a data interface of the specified service system, so that the first SDK in the specified data form collects data in the specified service system. In the embodiment of the invention, the data form is connected with the service system, so that the data in the service system can be collected, the data form configuration information is provided to provide a basis for data processing, and the development cost is reduced.

Description

[0001] This application is a divisional application of the authorized patent (patent application number 2020108448740). Technical Field

[0002] The embodiments of the present disclosure relate to the field of machine data processing technology, and more particularly to a data processing method, device, and electronic device based on data form. Background Art

[0003] The process of implementing artificial intelligence (AI) technology in real-world scenarios includes, but is not limited to, problem definition, data governance, feature engineering, machine learning model building (modeling), model deployment, feedback collection, and iterative model updates. Modeling is a crucial step in the machine learning process, while prior data governance and feature engineering are essential for obtaining valid data necessary for modeling, thereby enabling the creation of an effective model.

[0004] In practice, data collection, storage, and organization in many business systems and scenarios are governed by rules set by the business side, or even by no rules at all. Implementing AI applications in these business systems and applications requires significant time and effort for data governance. This raises the barrier to entry for AI applications and hinders their practical implementation. Summary of the Invention

[0005] In order to solve at least one problem existing in the prior art, at least one embodiment of the present disclosure provides a data processing method, apparatus, and electronic device based on data form.

[0006] In a first aspect, embodiments of the present disclosure provide a data processing method based on a data form, wherein the data form is a second SDK including a first SDK and data form configuration information, wherein the first SDK is used for data collection, and the data form configuration information is a collection of definition information related to data processing; the method includes:

[0007] Pre-storing at least one data form in a data form library, wherein different data forms have different first SDKs and / or data form configuration information;

[0008] receiving a user's selection operation of selecting a specified data form from the data form library, and receiving information of a specified business system input by the user;

[0009] The designated data format is connected to the data interface of the designated business system, so that the first SDK of the designated data format collects data in the designated business system.

[0010] In some embodiments, the method further comprises:

[0011] Receive a user's selection operation of selecting a specified application from an application library, run the specified application, so that the specified application opens the second SDK of the specified data format, and processes the data collected by the first SDK of the specified data format based on the data format configuration information of the specified data format.

[0012] In some embodiments, the second SDK further includes a third SDK, and the third SDK is used to process the data collected by the first SDK based on the data format configuration information;

[0013] The method further comprises:

[0014] The third SDK of the specified data format performs data processing on the data collected by the first SDK of the specified data format based on the data format configuration information of the specified data format.

[0015] In some embodiments, the data form configuration information includes: data form description information;

[0016] The method further comprises:

[0017] Displaying data form description information of each data form in the data form library for user reference;

[0018] Correspondingly, after receiving the user's selection operation of selecting a specified data form from the data form library, the specified data form is determined according to the selection operation.

[0019] In some embodiments, the data format description information includes at least one of the following:

[0020] Scenario definition of data form, type definition of data form and task definition associated with data form.

[0021] In some embodiments, the data form configuration information includes: a data form docking method;

[0022] Accordingly, the connecting the designated data format with the data interface of the designated business system includes:

[0023] Based on the docking method of the specified data format, the first SDK of the specified data format is docked with the data interface of the specified business system.

[0024] In some embodiments, the data form configuration information includes: an application associated with the data form;

[0025] The method further comprises:

[0026] Displaying applications associated with each data form in the data form library for user selection;

[0027] Correspondingly, after receiving the user's selection operation of selecting a designated application from the application library, the designated application is determined according to the selection operation.

[0028] In some embodiments, the data form configuration information includes: a data table schema definition for the data form;

[0029] Correspondingly, the processing of the data collected by the first SDK in the specified data format includes: performing data processing on the data collected by the first SDK in the specified data format based on a data table schema definition of the specified data format.

[0030] In some embodiments, the data form configuration information includes: a status of the data form, the status including available and unavailable;

[0031] The method further comprises:

[0032] If the data collected by the first SDK in the specified data format meets the data table schema definition of the specified data format, setting the status of the specified data format to available;

[0033] If the data collected by the first SDK in the specified data format does not meet the data table model definition of the specified data format, the status of the specified data format is set to unavailable.

[0034] In some embodiments, the method further comprises:

[0035] If the status of the specified data format is set to available, applications associated with the specified data format are displayed for user selection; if the status of the specified data format is set to unavailable, applications associated with the specified data format are not displayed.

[0036] In some embodiments, the data form configuration information includes: a storage definition of the data form;

[0037] The method further includes: storing the data collected by the first SDK in the specified data format based on a storage definition of the specified data format.

[0038] In some embodiments, the method further comprises:

[0039] Display the data collection SDKs in the data collection SDK library so that developers can select the data collection SDK for the specified business system;

[0040] receiving a selection operation of a developer selecting a specified data collection SDK from the data collection library, and / or receiving a data collection SDK developed by the developer for the specified business system;

[0041] Receive data format configuration information developed by the developer for the specified business system;

[0042] The designated data collection SDK and / or developed data collection SDK, and the data format configuration information are integrated and packaged to obtain a designated data format for the designated business system.

[0043] In some embodiments, the method further comprises:

[0044] Displaying applications in the application library so that the developer can select an application associated with the specified data format;

[0045] A selection operation of a developer selecting a designated application from the application library is received, and the designated application is associated with the designated data format.

[0046] In some embodiments, the method further comprises:

[0047] Providing an application development SDK for developers to develop specific applications for the specified data format;

[0048] Receive a designated application developed by a developer using the application development SDK;

[0049] The specified application is associated with the specified data form.

[0050] In a second aspect, embodiments of the present disclosure further provide a data processing device based on a data form, wherein the data form is a second SDK including a first SDK and data form configuration information, wherein the first SDK is used for data acquisition, and the data form configuration information is a collection of definition information related to data processing; at least one data form is pre-stored in a data form library, wherein different data forms have different first SDKs and / or data form configuration information;

[0051] The device comprises:

[0052] a first receiving unit, configured to receive a selection operation of a user selecting a specified data form from the data form library, and receive information of a specified business system input by the user;

[0053] The docking unit is used to dock the specified data format with the data interface of the specified business system, so that the first SDK of the specified data format collects data in the specified business system.

[0054] In some embodiments, the apparatus further comprises:

[0055] The running unit is used to receive a selection operation of a user selecting a specified application from an application library, run the specified application so that the specified application opens the second SDK in the specified data format, and processes the data collected by the first SDK in the specified data format based on the data format configuration information of the specified data format.

[0056] In some embodiments, the second SDK further includes a third SDK, and the third SDK is used to process the data collected by the first SDK based on the data format configuration information;

[0057] Correspondingly, the third SDK of the specified data format performs data processing on the data collected by the first SDK of the specified data format based on the data format configuration information of the specified data format.

[0058] In some embodiments, the data form configuration information includes: data form description information;

[0059] The device further comprises:

[0060] A first display unit is used to display data form description information of each data form in the data form library for user reference;

[0061] Correspondingly, the first receiving unit is further configured to receive a selection operation of a user selecting a specified data form from the data form library, and determine the specified data form according to the selection operation.

[0062] In some embodiments, the data format description information includes at least one of the following:

[0063] Scenario definition of data form, type definition of data form and task definition associated with data form.

[0064] In some embodiments, the data form configuration information includes: a data form docking method;

[0065] Correspondingly, the docking unit docking the specified data format with the data interface of the specified business system includes: docking the first SDK of the specified data format with the data interface of the specified business system based on the docking method of the specified data format.

[0066] In some embodiments, the data form configuration information includes: an application associated with the data form;

[0067] The device further comprises:

[0068] A second display unit is used to display applications associated with each data form in the data form library for user selection;

[0069] Correspondingly, the running unit is further configured to receive a selection operation from a user to select a designated application from the application library, and then determine the designated application according to the selection operation.

[0070] In some embodiments, the data form configuration information includes: a data table schema definition for the data form;

[0071] Correspondingly, the processing of the data collected by the first SDK in the specified data format includes: performing data processing on the data collected by the first SDK in the specified data format based on a data table schema definition of the specified data format.

[0072] In some embodiments, the data form configuration information includes: a status of the data form, the status including available and unavailable;

[0073] The device further comprises:

[0074] A state adjustment unit is used to set the state of the specified data form to available if the data collected by the first SDK of the specified data form meets the data table pattern definition of the specified data form; if the data collected by the first SDK of the specified data form does not meet the data table pattern definition of the specified data form, then set the state of the specified data form to unavailable.

[0075] In some embodiments, the apparatus further comprises:

[0076] The third display unit is used to display applications associated with the specified data form for user selection after the state adjustment unit sets the state of the specified data form to available; and not display applications associated with the specified data form after the state adjustment unit sets the state of the specified data form to unavailable.

[0077] In some embodiments, the data form configuration information includes: a storage definition of the data form;

[0078] The device further comprises:

[0079] The storage unit is configured to store the data collected by the first SDK in the specified data format based on the storage definition of the specified data format.

[0080] In some embodiments, the apparatus further comprises:

[0081] The fourth display unit is used to display various data collection SDKs in the data collection SDK library so that developers can select the data collection SDK for the specified business system;

[0082] The second receiving unit is configured to receive a selection operation of a developer selecting a specified data collection SDK from the data collection library, and / or receive a data collection SDK developed by the developer for the specified business system; and receive data format configuration information developed by the developer for the specified business system;

[0083] The integration and packaging unit is used to integrate and package the specified data acquisition SDK and / or the developed data acquisition SDK, and the data format configuration information to obtain the specified data format for the specified business system.

[0084] In some embodiments, the fourth display unit is further used to display various applications in the application library so that the developer can select an application associated with the specified data form;

[0085] Correspondingly, the second receiving unit is further configured to associate the designated application with the designated data format after receiving a selection operation of the developer selecting a designated application from the application library.

[0086] In some embodiments, the apparatus further comprises: a providing unit, configured to provide an application development SDK for a developer to develop a specified application for the specified data format;

[0087] The second receiving unit is further configured to receive a designated application developed by a developer using the application development SDK; and associate the designated application with the designated data format.

[0088] In a third aspect, an embodiment of the present disclosure further proposes an electronic device, comprising: a processor and a memory; the processor is used to execute the steps of the method described in any embodiment of the first aspect by calling a program or instruction stored in the memory.

[0089] In a fourth aspect, an embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium for storing a program or instruction, wherein the program or instruction enables a computer to execute the steps of the method described in any embodiment of the first aspect.

[0090] It can be seen that in at least one embodiment of the present disclosure, the data collection SDK and the configuration information related to data processing can be packaged into a new SDK (i.e., data form) and saved in a data form library, so that the user can select a specified data form suitable for a specified business system from the data form library. After the specified data form is connected to the specified business system, the data in the specified business system can be collected, and the configuration information related to data processing provides a basis for data processing, realizing data governance before feature engineering. Therefore, there is no need to strictly bind the modeling template to the data used for modeling. Third-party professional developers can develop different replaceable data forms based on their own business understanding, transferring the development difficulty from users and platforms to third-party developers, thereby reducing the development costs of users and platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0092] Figure 1 is an exemplary scene graph in the form of usage data provided by an embodiment of the present disclosure;

[0093] Figure 2 This is an exemplary scenario diagram of a development data form and application provided by an embodiment of the present disclosure;

[0094] Figure 3 is an exemplary flow chart of a data processing method based on data form provided by an embodiment of the present disclosure;

[0095] Figure 4 is an exemplary block diagram of a data processing device based on data form provided by an embodiment of the present disclosure;

[0096] Figure 5 It is an exemplary block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0097] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the present disclosure is further described in detail below with reference to the accompanying drawings and examples. It will be understood that the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. The specific embodiments described herein are merely used to explain the present disclosure, rather than to limit the present disclosure. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art are within the scope of protection of the present disclosure.

[0098] It should be noted that, in this document, relational terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0099] Because users (customers) are familiar with their own business scenarios, they can leverage the platform (the machine learning platform provided by the machine learning service provider) to develop modeling templates tailored to their business scenarios. Alternatively, the platform can develop modeling templates tailored to the user's business needs. Regardless of whether the user or the platform develops modeling templates for a specific business scenario, both face challenges in managing the data in the business scenario into the format required by the modeling template, resulting in high development costs for different business scenarios.

[0100] To solve these problems, it is difficult to make breakthroughs relying solely on the capabilities of individual machine learning products (e.g., AI applications), because the more customized an AI application is, the more professional it tends to be. Conversely, the more open an AI application is, the higher the threshold for use will be.

[0101] To this end, in the embodiments of the present disclosure, third-party professional developers can use highly open underlying SDKs (such as data collection SDKs and application development SDKs) to design and develop data formats and applications, allowing users to use AI applications with a low threshold by using data formats. In addition, AI solutions can be put online (for example, models obtained through AI applications can be put online to provide estimation services). In addition, the design of AI solutions is more open (for example, a model solution can be designed to perform model self-learning based on the model solution and online data to obtain an online model).

[0102] Figure 1 An exemplary scene graph in the form of data is provided for an embodiment of the present disclosure. Figure 1 Although not shown, those skilled in the art will appreciate that the first user interface is a user interface provided by an electronic device, which can be a mobile device such as a smartphone, laptop, or tablet, or a stationary device such as a desktop computer. The data format library and application library can be located in the electronic device or in other electronic devices, with different electronic devices communicating with each other to exchange information between the data format library and the application library. In some embodiments, the data format library and application library can be implemented as software or a combination of software and hardware.

[0103] Combine Figure 1 , describes the use of data forms from the user's perspective.

[0104] exist Figure 1In the first user interface, the user can input information of the specified business system, such as but not limited to: the name of the business system, the scenario definition of the business system, the type of the business system, etc.

[0105] exist Figure 1 In the first user interface, the user can view one or more data forms stored in the data form library through the first user interface.

[0106] Data Form can be understood as a special data acquisition SDK (Software Development Kit). Unlike ordinary data acquisition SDKs, Data Form not only possesses data acquisition functionality but also includes a collection of definition information related to data processing, which provides the basis for data processing. Data acquisition and data processing are two distinct phases: data acquisition comes first and is used to acquire data; data processing comes later and is used to process the data acquired by the data acquisition function. In other words, data acquisition can be understood as acquiring raw data without processing it, while data processing can be understood as processing the raw data to obtain new data that is different from the original data. Therefore, data acquisition is not a form of data processing, but rather a phase independent of data processing.

[0107] In some embodiments, the data format includes a first SDK and a second SDK with data format configuration information. The first SDK is used for data collection, and the data format configuration information is a collection of definition information related to data processing, providing a basis for data processing. Different data formats have different first SDKs and / or data format configuration information.

[0108] In some embodiments, the data form (ie, the second SDK) may further include a third SDK for processing the data collected by the first SDK based on the data form configuration information. In this embodiment, the data form not only has a data collection function, but also has a data processing function.

[0109] In some embodiments, to help users understand the functions and content of data forms, a description document for each data form can be added to the data form library. The description document includes information about the problems (or tasks) that the data form can solve. If a user is unfamiliar with a data form, they can first review the description document, further lowering the barrier to entry.

[0110] exist Figure 1 In the first user interface, the user can view one or more applications stored in the application library through the first user interface. The applications stored in the application library can be understood as AI applications. Different AI applications can implement different AI functions, such as estimation function.

[0111] exist Figure 1In the , users can select a specified data form from the data form library and a specified application from the application library based on the information of the specified business system, so as to clearly determine which data form and which application to use to perform the task of the specified business system (such as estimation task).

[0112] At this point, the user's operation using the data form is completed. It can be seen that the user only needs to enter the information of the specified business system, select the specified data form and select the specified application. The operation is simple and uncomplicated.

[0113] In this way, other work using the data form is done by the electronic device.

[0114] For example, the electronic device may associate a specified data form with a specified application. Since the specified data form is essentially an SDK, the user can directly open the specified data form through the associated specified application, and view the data form configuration information in the specified application in a similar way to file management (such as viewing files or folders). It can be seen that in this embodiment, the electronic device associates the data form with the application (such as an AI application), and the user only needs to select the data form and the application, and then implement a similar file management method through the application, thereby reducing the cost of the user using the data form and improving the management efficiency of the data form, thereby lowering the threshold for using AI applications.

[0115] For another example, the electronic device can connect the specified data form with the data interface of the specified business system to realize the construction of the data pipeline, and then collect the data in the specified business system through the specified data form. Since the businesses of different business systems will not be interoperable, different business systems have different data interfaces and are independent of each other, that is, the data interface is not a universal interface, but a unique interface of the business system. Different business systems have independent data interfaces to achieve data isolation and improve the security of business data. In some embodiments, the process of data connection of the electronic device can be carried out under the supervision of the user to ensure that the data form and the business system can match the data, for example, the necessary data fields required by the data form (which are part of the data form configuration information) are connected one-to-one with the data of the business system.

[0116] Furthermore, if the specified data format includes a third SDK, the third SDK of the specified data format processes the data collected by the first SDK of the specified data format based on the data format configuration information of the specified data format. This processing can be understood as data governance before feature engineering. In this embodiment, the data format not only has data collection functions, but also data processing functions.

[0117] For another example, an electronic device can run a specified application to open a second SDK in a specified data format and process the data collected by the first SDK in the specified data format based on the data format configuration information of the specified data format. This processing can be understood as data governance before feature engineering. In this embodiment, the data format only has a data collection function and does not have a data processing function (that is, the data format does not include the third SDK).

[0118] Furthermore, after a designated application processes data collected by the first SDK, or after a third SDK in a designated data format processes data collected by the first SDK, the designated application performs feature engineering on the processed data, and then uses the output of feature engineering to train an AI model. The electronic device can then deploy this AI model online to provide services, making the AI ​​solution available online.

[0119] Considering that some business systems do not have corresponding data formats or applications, third-party professional developers are needed to develop data formats or applications.

[0120] Figure 2 An exemplary scenario diagram of a development data form and application provided by an embodiment of the present disclosure. Figure 2 Although not shown, those skilled in the art will appreciate that the second user interface is a user interface provided by the electronic device. Figure 1 The first user interfaces shown in can be the same interface or different interfaces. Figure 2 In the data collection SDK library, application development SDK, data format library and application library, they can be located in the electronic device; they can also be located in other electronic devices.

[0121] Combine Figure 2 , describes the development of data forms from the developer's perspective.

[0122] exist Figure 2 In the example, for a given business system, developers can develop data form configuration information through the second user interface based on business technical know-how. In some embodiments, developers can develop different data form configuration information for the same business system, achieving rich and replaceable data form configuration information, increasing development flexibility.

[0123] exist Figure 2 In the data collection SDK library, developers can view one or more data collection SDKs stored in the data collection SDK library through the second user interface.

[0124] In some embodiments, developers can determine whether to develop a new data collection SDK. For example, if a business system requires three types of data to be collected, and none of the data collection SDKs in the data collection SDK library can collect all three types of data, then a new data collection SDK needs to be developed. Alternatively, if a data collection SDK in the library can collect one of these three types of data, then a new data collection SDK needs to be developed. Alternatively, if all three types of data can be collected, then there is no need to develop a new data collection SDK.

[0125] exist Figure 2 In the second user interface, developers can develop data collection SDK.

[0126] exist Figure 2 In the example, the developer can select a specified data collection SDK from the data collection SDK library through the second user interface. The specified data collection SDK can collect part or all of the data in the specified business system.

[0127] At this point, the developer has completed the operation of developing the data form. It can be seen that the developer only needs to develop the data form configuration information and develop a new data acquisition SDK and / or select an existing data acquisition SDK. In this way, the other work of developing the data form is completed by the electronic device.

[0128] In some embodiments, the electronic device integrates and packages the received data format configuration information, the developed data collection SDK and the designated data collection SDK into a designated data format for a designated business system, wherein the developed data collection SDK and the designated data collection SDK are integrated and packaged into a first SDK.

[0129] In some embodiments, if the developer has not developed a new data collection SDK, the electronic device integrates and packages the received data format configuration information and the designated data collection SDK into a designated data format for a designated business system, where the designated data collection SDK is the first SDK.

[0130] In some embodiments, if the developer does not specify a data collection SDK and only develops a data collection SDK, the electronic device will receive the data format configuration information and the developed data collection SDK, and integrate and package them into a specified data format for a specified business system, where the developed data collection SDK is the first SDK.

[0131] In some embodiments, the developer can trigger an integration and packaging operation. For example, the developer clicks an integration and packaging button on the second user interface. The electronic device responds to the integration and packaging operation and performs integration and packaging to obtain a specified data format for a specified business system.

[0132] Combine Figure 2 , describes application development from a developer's perspective.

[0133] exist Figure 2 In the application library, developers can view one or more applications stored in the application library through the second user interface. For a specific business system, developers can determine whether they need to develop new applications.

[0134] In some embodiments, if a new application needs to be developed, the developer can use the application development SDK in the second user interface to develop a specific application. For example, the developer can build a test environment to obtain test data formats, and then use the application development SDK to develop a specific application based on the test data formats. As can be seen, in this embodiment, the application development SDK allows developers to define their own applications, achieving a considerable degree of freedom. This allows them to develop not only AI applications, but also BI (business intelligence) applications and even other applications.

[0135] In some embodiments, if there is no need to develop a new application, the developer can select a specified application from the application library through the second user interface.

[0136] The electronic device associates the specified data form developed by the developer with the specified application developed (or selected), saves the specified data form in the data form library, and saves (or updates) the specified application in the application library. Here, updating the specified application to the database can be understood as updating the specified application in the application library, and the updated content is the information of the specified data form associated with the specified application.

[0137] In some embodiments, the developer can trigger an association operation, for example, the developer clicks an association button on the second user interface, and the electronic device responds to the association operation and associates the specified data form developed by the developer with the developed (or selected) specified application.

[0138] Figure 3 This is an exemplary flow chart of a data processing method based on data form provided in an embodiment of the present disclosure. The execution subject of the method is an electronic device. For ease of description, the following embodiments illustrate the process of the data processing method based on data form using an electronic device as the execution subject.

[0139] In this embodiment, the data form is a second SDK including a first SDK and data form configuration information, wherein the first SDK is used for data collection, and the data form configuration information is a collection of definition information related to data processing.

[0140] In some embodiments, the data form configuration information includes but is not limited to one or more of the following (1) to (5):

[0141] (1) Data format description information. In some embodiments, the data format description information may include, but is not limited to, one or more of the following (11) to (14):

[0142] (11) Name of the data format.

[0143] (12) Scenario definition in data form. In some embodiments, the scenario definition may include but is not limited to the name of the business system.

[0144] (13) Type definition of data form. In some embodiments, type definition includes but is not limited to custom types to facilitate user retrieval.

[0145] (14) Task definition associated with data form. In some embodiments, the task definition includes but is not limited to a task ID.

[0146] (2) Data-based docking methods. In some embodiments, data-based docking methods may include, but are not limited to, one or more of the following (21) and (22):

[0147] (21) Streaming docking method.

[0148] (22) Batch docking method.

[0149] (3) Application of data form association.

[0150] (4) Data table schema definition in data form. In some embodiments, the schema definition may include but is not limited to one or more of the following (41) to (43):

[0151] (41) The name of one or more data tables.

[0152] (42) The fields included in each data table.

[0153] (43)Data relationships between multiple data tables.

[0154] (5) Storage definition of data form. The storage definition is used to store the collected data of the business system in preparation for subsequent modeling. The stored data has a clear temporal characteristic. When the data form is connected to the real-time data of the business system, the data generation time has been strictly recorded. In some embodiments, the storage definition of the data form may include but is not limited to one or more of the following (51) and (52):

[0155] (51) Storage location. In some embodiments, the storage location is, for example, HDFS (Hadoop Distributed File System) or RtiDB (Real-time Feature Storage Engine).

[0156] (52) Storage path.

[0157] (6) Status of data form: In some embodiments, the status of the data form is available or unavailable.

[0158] In step 301, the electronic device pre-stores at least one data form in a data form library, wherein different data forms have different first SDKs and / or data form configuration information.

[0159] In some embodiments, the electronic device can save the data form to the data form library in response to a save operation by the user (the user is a user or a developer). It should be noted that step 301 is a pre-executed step and is not a necessary step. For example, if the data form is already saved in the data form library, step 301 is not necessary.

[0160] In step 302, the electronic device receives a selection operation of a user selecting a specified data form from a data form library, and receives information of a specified business system input by the user.

[0161] In some embodiments, the electronic device is provided with a user interface that can display data form description information for each data form in the data form library for user reference. The electronic device can receive a user selection operation for a specific data form through the user interface, and then determine the specific data form based on the selection operation. In addition, the electronic device can receive user input of information specifying a business system through the user interface, and then determine the specific business system.

[0162] In step 303, since the electronic device has determined the designated data format and the designated business system, it can connect the designated data format with the data interface of the designated business system, so that the first SDK of the designated data format collects data from the designated business system. In some embodiments, the electronic device can connect the first SDK of the designated data format with the data interface of the designated business system based on the connection method of the designated data format.

[0163] In some embodiments, the electronic device may store the data collected by the first SDK in a specified data format based on a storage definition of the specified data format.

[0164] In some embodiments, if the electronic device determines that the data collected by the first SDK in the specified data form meets the Schema definition of the specified data form, the status of the specified data form is set to available; if it determines that the data collected by the first SDK in the specified data form does not meet the Schema definition, the status of the specified data form is set to unavailable.

[0165] In some embodiments, for data in a specified business system collected by a first SDK in a specified data format, there may be two data processing (which may be understood as data governance) methods:

[0166] Method 1: The application processes the data.

[0167] In this mode, the electronic device is provided with a user interface that can display applications associated with various data formats in the data format library for user selection. It should be noted that if the status of a specified data format is unavailable, the applications associated with the specified data format will not be displayed; only when the status of the specified data format is available will the applications associated with the specified data format be displayed. The electronic device can receive a selection operation from the user through the user interface to select a specified application from the application library, then determine the specified application based on the selection operation, and run the specified application, so that the specified application opens a second SDK for the specified data format and processes data collected by the first SDK for the specified data format based on the data format configuration information of the specified data format.

[0168] Method 2: Data processing in data form.

[0169] In this manner, the second SDK in the data format also includes a third SDK, which is configured to process data collected by the first SDK based on the data format configuration information. In this embodiment, the third SDK in the specified data format processes data collected by the first SDK in the specified data format based on the data format configuration information for the specified data format.

[0170] In some embodiments, regardless of method 1 or method 2, the process of data processing (governance) is similar, as described below:

[0171] First, the data collected by the first SDK in the specified data format is processed according to the schema definition of the specified data format. Second, the processed data is converted according to the definition information of the relevant data of the specified business system to obtain the relevant data of the specified business system. The definition information of the relevant data of the specified business system includes but is not limited to: request data, exposure data, and feedback data.

[0172] Taking a credit card marketing system as an example, request data refers to the information sent to an AI application. For example, if 10,000 customers are selected as a candidate set, and the AI ​​application is combined with other information to help determine the probability that each customer will actually apply for a credit card, these 10,000 pieces of data are the request data. After the AI ​​application makes predictions, the actual marketing call will not involve calling all 10,000 customers. Instead, the business side will select only customers who are more likely to apply for a credit card, perhaps selecting only 100 of them. These 100 pieces of data are the exposure data. Finally, whether the customer actually applies for a credit card after the call is the feedback data. In addition to request data, exposure data, and feedback data, the scenario may also include auxiliary data, which is other information that may help improve the AI ​​application's prediction results, such as basic customer information, transaction records, and credit records.

[0173] In some embodiments, the electronic device may provide a user interface that may display various data collection SDKs in a data collection SDK library for developers to select a data collection SDK for a specified business system.

[0174] In some embodiments, the electronic device may receive, through a user interface, a developer's selection of a specified data collection SDK from a data collection library and / or a data collection SDK developed by the developer for a specified business system. In some embodiments, the electronic device may receive, through a user interface, data format configuration information developed by the developer for the specified business system.

[0175] In some embodiments, the electronic device may integrate and package the designated data collection SDK selected by the developer and / or the data collection SDK developed by the developer, as well as the data format configuration information developed by the developer, to obtain a designated data format for a designated business system.

[0176] In some embodiments, the electronic device may display various applications in the application library through a user interface for the developer to select an application associated with a specified data format. Accordingly, the electronic device receives the developer's selection operation of selecting a specified application from the application library and associates the specified application with the specified data format.

[0177] In some embodiments, the electronic device may provide an application development SDK for developers to develop specified applications for specified data forms. Accordingly, the electronic device receives the specified application developed by the developer using the application development SDK and associates the specified application with the specified data form. In some embodiments, the application development SDK may be provided by a platform, which may be provided in the electronic device or in other devices independent of the electronic device, such as a server. The server may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the server may be local or remote.

[0178] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art will understand that the embodiments of the present disclosure are not limited by the order of the actions described, because according to the embodiments of the present disclosure, certain steps can be performed in other orders or simultaneously. In addition, those skilled in the art will understand that the embodiments described in the specification are all optional embodiments.

[0179] Figure 4 An embodiment of the present disclosure provides a data form-based data processing device 400, which can be used in electronic devices. The data form includes a first SDK and a second SDK with data form configuration information. The first SDK is used for data acquisition, and the data form configuration information is a collection of definition information related to data processing. At least one data form is pre-stored in a data form library, and different data forms have different first SDKs and / or data form configuration information.

[0180] like Figure 4 As shown, the data processing device 400 includes:

[0181] The first receiving unit 401 is configured to receive a selection operation of a user selecting a specified data form from a data form library, and receive information of a specified business system input by the user.

[0182] The docking unit 402 is used to dock the specified data format with the data interface of the specified business system, so that the first SDK of the specified data format collects data in the specified business system.

[0183] In some embodiments, the data processing device 400 further includes:

[0184] The running unit is used to receive a selection operation of a user selecting a specified application from an application library, run the specified application, so that the specified application opens the second SDK in the specified data format, and processes the data collected by the first SDK in the specified data format based on the data format configuration information of the specified data format.

[0185] In some embodiments, the second SDK also includes a third SDK, which is used to process the data collected by the first SDK based on the data format configuration information; accordingly, the third SDK of the specified data format processes the data collected by the first SDK of the specified data format based on the data format configuration information of the specified data format.

[0186] In some embodiments, the data format configuration information includes: data format description information; the data processing device 400 further includes:

[0187] The first display unit is used to display the data form description information of each data form in the data form library for user reference;

[0188] Correspondingly, the first receiving unit 401 is further configured to receive a selection operation of a user selecting a specified data form from the data form library, and determine the specified data form according to the selection operation.

[0189] In some embodiments, the data form description information includes at least one of the following: a scenario definition of the data form, a type definition of the data form, and a task definition associated with the data form.

[0190] In some embodiments, the data form configuration information includes: the docking method of the data form; accordingly, the docking unit 402 docks the specified data form with the data interface of the specified business system, including: based on the docking method of the specified data form, docking the first SDK of the specified data form with the data interface of the specified business system.

[0191] In some embodiments, the data format configuration information includes: an application associated with the data format; accordingly, the data processing device 400 further includes:

[0192] The second display unit is used to display applications associated with each data form in the data form library for user selection; accordingly, the operation unit is also used to receive a selection operation from the user to select a specified application from the application library, and determine the specified application according to the selection operation.

[0193] In some embodiments, the data format configuration information includes: a data table schema definition of the data format; accordingly, processing the data collected by the first SDK of the specified data format includes: performing data processing on the data collected by the first SDK of the specified data format based on the data table schema definition of the specified data format.

[0194] In some embodiments, the data format configuration information includes: a data format status, where the status includes available and unavailable; the data processing device 400 further includes:

[0195] The state adjustment unit is used to set the state of the specified data form to available if the data collected by the first SDK in the specified data form meets the data table model definition of the specified data form; if the data collected by the first SDK in the specified data form does not meet the data table model definition of the specified data form, then the state of the specified data form is set to unavailable.

[0196] In some embodiments, the data processing device 400 further includes:

[0197] The third display unit is used to display applications associated with the specified data format for user selection after the state adjustment unit sets the state of the specified data format to available; and not display applications associated with the specified data format after the state adjustment unit sets the state of the specified data format to unavailable.

[0198] In some embodiments, the data format configuration information includes: a storage definition of the data format; the data processing device 400 further includes: a storage unit for storing data collected by the first SDK in the specified data format based on the storage definition of the specified data format.

[0199] In some embodiments, the data processing device 400 further includes:

[0200] The fourth display unit is used to display various data collection SDKs in the data collection SDK library, so that developers can select the data collection SDK for a specific business system;

[0201] The second receiving unit is configured to receive a selection operation of a developer selecting a specified data collection SDK from a data collection library, and / or receive a data collection SDK developed by the developer for a specified business system; and receive data format configuration information developed by the developer for the specified business system;

[0202] The integration and packaging unit is used to integrate and package the specified data collection SDK and / or developed data collection SDK, as well as the data format configuration information, to obtain the specified data format for the specified business system.

[0203] In some embodiments, the fourth display unit is further used to display each application in the application library so that the developer can select an application associated with a specified data format; accordingly, the second receiving unit is further used to associate the specified application with the specified data format after receiving the developer's selection operation of selecting a specified application from the application library.

[0204] In some embodiments, the data processing device 400 further includes: a providing unit configured to provide an application development SDK for a developer to develop a specified application for a specified data format. Accordingly, the second receiving unit is further configured to receive a specified application developed by the developer using the application development SDK; and associate the specified application with the specified data format.

[0205] In some embodiments, the division of the units in the data processing device 400 is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple units may be implemented as a single unit, or a single unit may be divided into multiple sub-units. It is understood that each unit or sub-unit can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application.

[0206] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure.

[0207] like Figure 5 As shown, the electronic device includes: at least one processor 501, at least one memory 502 and at least one communication interface 503. The various components in the electronic device are coupled together through a bus system 504. The communication interface 503 is used to transmit information with external devices. It can be understood that the bus system 504 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 504 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 504 .

[0208] It can be understood that the memory 502 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0209] In some embodiments, the memory 502 stores the following elements, executable units or data structures, or a subset or an extended set thereof: an operating system and application programs.

[0210] The operating system includes various system programs, such as the framework layer, core library layer, and driver layer, which are used to implement various basic tasks and process hardware-based tasks. The application program includes various application programs, such as a media player and a browser, which are used to implement various application tasks. The program that implements the data processing method based on the data form provided in the embodiment of the present disclosure can be included in the application program.

[0211] In the embodiment of the present disclosure, the processor 501 calls the program or instructions stored in the memory 502, specifically, the program or instructions stored in the application, and the processor 501 is used to execute the steps of each embodiment of the data processing method based on data form provided in the embodiment of the present disclosure.

[0212] The data processing method based on data form provided in the embodiment of the present disclosure can be applied to the processor 501, or implemented by the processor 501. The processor 501 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 501 or the instruction in the form of software. The above-mentioned processor 501 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0213] The steps of the data processing method based on data format provided in the embodiments of the present disclosure can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software units in the decoding processor. The software units can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 502, and processor 501 reads the information in memory 502 and combines it with hardware to complete the steps of the method.

[0214] The embodiments of the present disclosure also propose a non-transitory computer-readable storage medium, which stores programs or instructions. The programs or instructions enable a computer to execute the steps of each embodiment of the data processing method based on data form. To avoid repeated description, they are not repeated here.

[0215] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0216] Those skilled in the art will appreciate that although some embodiments described herein include certain features and not others included in other embodiments, the combination of features from different embodiments is intended to be within the scope of this disclosure and to form different embodiments.

[0217] Those skilled in the art will understand that the description of each embodiment has its own focus, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0218] Although the embodiments of the present disclosure have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data processing method based on data form, characterized in that: The data form is a second SDK including a first SDK and data form configuration information, wherein the first SDK is used for data collection, and the data form configuration information is a collection of definition information related to data processing; the method includes: Pre-storing at least one data form in a data form library, wherein different data forms have different first SDKs and / or data form configuration information; receiving a selection operation of a user selecting a specified data form from the data form library, and receiving information of a specified business system input by the user; The designated data form is connected to the data interface of the designated business system, so that the first SDK of the designated data form collects data in the designated business system.

2. The method according to claim 1, characterized in that: The method further comprises: Receive a selection operation of a user selecting a specified application from an application library, run the specified application so that the specified application opens the second SDK in the specified data form, and processes data collected by the first SDK in the specified data form based on data form configuration information of the specified data form.

3. The method according to claim 1, characterized in that The second SDK also includes a third SDK, and the third SDK is used to process the data collected by the first SDK based on the data format configuration information; The method further comprises: The third SDK of the specified data format performs data processing on the data collected by the first SDK of the specified data format based on the data format configuration information of the specified data format.

4. The method according to claim 1, characterized in that: The data form configuration information includes: data form description information; The method further comprises: Displaying data form description information of each data form in the data form library for user reference; Correspondingly, after receiving the selection operation of the user selecting a designated data form from the data form library, the designated data form is determined according to the selection operation.

5. The method according to claim 4, characterized in that The data form description information includes at least one of the following: Scenario definition of the data form, type definition of the data form, and task definition associated with the data form.

6. The method according to claim 1, characterized in that The data form configuration information includes: a data form docking method; Accordingly, the connecting the specified data form with the data interface of the specified business system includes: Based on the docking method of the specified data format, the first SDK of the specified data format is docked with the data interface of the specified business system.

7. The method according to claim 2, characterized in that The data form configuration information includes: applications associated with the data form; The method further comprises: Displaying applications associated with each data form in the data form library for selection by the user; Correspondingly, after receiving the selection operation of the user selecting a designated application from the application library, the designated application is determined according to the selection operation.

8. A data processing device based on data form, characterized in that: The data form is a second SDK including a first SDK and data form configuration information, wherein the first SDK is used for data collection, and the data form configuration information is a collection of definition information related to data processing; at least one data form is pre-stored in a data form library, wherein different data forms have different first SDKs and / or data form configuration information; The device comprises: A first receiving unit is used to receive a selection operation of a user selecting a specified data form from the data form library, and receive information of a specified business system input by the user; The docking unit is used to dock the specified data form with the data interface of the specified business system, so that the first SDK of the specified data form collects data in the specified business system.

9. An electronic device, characterized in that: include: Processor and memory; The processor is used to execute the steps of the method according to any one of claims 1 to 7 by calling the program or instruction stored in the memory.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the method according to any one of claims 1 to 7.

Images