Data processing model construction method and device, electronic equipment and storage medium

Abstract

The invention discloses a data processing model construction method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a component customization operation, displaying a component customization page, and obtaining component basic configuration information from the component customization page; creating a component function configuration template according to the component basic configuration information, and acquiring component function configuration information by using the component function configuration template; generating a component application configuration page according to the component function configuration information, acquiring component application configuration information from the component application configuration page, and storing the component application configuration information to generate a customized component; in response to the component selection operation, displaying a component selection page, and determining a target component selected from customized components and preset components displayed in the component selection page; and creating a data processing model according to the target component. According to the method, the configuration of the user-defined component application is realized, and the difficulty of customizing the component with the required function by the user is reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to a data processing model construction method, device, electronic device and storage medium. Background Art

[0002] The existing data processing model construction method only supports the use of existing components to build the model, for example, the use of components configured by professionals to build the model.

[0003] However, when non-professionals want to use data processing models for data processing, if the existing components cannot meet the functional requirements of the users, that is, the existing components have fewer functions, it will increase the difficulty of use for users and make the construction of data processing models more difficult.

[0004] Therefore, it is urgent to propose a new method to solve the above problems. Summary of the invention

[0005] The embodiments of the present invention provide a data processing model construction method, device, electronic device and storage medium, which realizes the configuration of custom component applications and reduces the difficulty for users to customize components with required functions.

[0006] According to one aspect of the present invention, a method for constructing a data processing model is provided, the method comprising:

[0007] Respond to component customization operations to display the component customization page and obtain component basic configuration information from the component customization page;

[0008] Create a component function configuration template according to the component basic configuration information, and use the component function configuration template to obtain the component function configuration information;

[0009] Generate a component application configuration page according to the component function configuration information, obtain the component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component;

[0010] In response to the component selection operation, a component selection page is displayed, and a target component is selected from the custom components and preset components displayed on the component selection page;

[0011] Create a data processing model based on the target components.

[0012] According to another aspect of the present invention, there is provided a data processing model construction device, the device comprising:

[0013] The customization module is used to display the component customization page in response to the component customization operation and obtain the basic configuration information of the component from the component customization page;

[0014] The acquisition module is used to create a component function configuration template according to the component basic configuration information, and obtain the component function configuration information using the component function configuration template;

[0015] A saving module, used to generate a component application configuration page according to the component function configuration information, obtain the component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component;

[0016] A determination module, used to display a component selection page in response to a component selection operation, and to determine a target component selected from custom components and preset components displayed on the component selection page;

[0017] Create a module to create a data processing model based on the target component.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the data processing model construction method of any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the data processing model construction method of any embodiment of the present invention when executed by a processor.

[0023] The data processing model construction method provided by the embodiment of the present invention displays a component customization page in response to a component customization operation, and obtains component basic configuration information from the component customization page; creates a component function configuration template according to the component basic configuration information, and obtains component function configuration information using the component function configuration template; generates a component application configuration page according to the component function configuration information, obtains component application configuration information from the component application configuration page, and saves the component application configuration information to generate a customized component; displays a component selection page in response to a component selection operation, and determines the target component selected from the customized component and the preset component displayed on the component selection page; creates a data processing model according to the target component. In the above technical scheme, on the one hand, firstly, by pre-filling the basic configuration information, a component function configuration template is created, so that a component configuration template containing the required functions is quickly generated according to the basic configuration. Secondly, the component function configuration information is obtained through the component function configuration template, so that the component function configuration information is quickly improved according to the generated component function configuration template, which reduces the difficulty of non-professionals in creating custom components and provides a basis for completing the required custom components later. On the other hand, a component application configuration page is generated according to the component function configuration information, and the component application configuration information is obtained from the component application configuration page, and the component application configuration information is saved to generate a customized component. Based on the visual interface, users can select the functional items required by the component and save the functional items selected by the user to generate a customized component, which realizes the configuration of the visualized custom component application and reduces the difficulty for users to customize the components with the required functions. Finally, based on the customized components and preset components, the components required for data processing are selected to obtain the target components, and a data processing model is created based on the target components, which realizes the creation of a fast and convenient data processing model.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A schematic diagram of a flow chart of a method for constructing a data processing model provided by an embodiment of the present invention;

[0027] Figure 2 An example diagram of a component customization page provided in an embodiment of the present invention;

[0028] Figure 3 An example diagram of a component function configuration template provided in an embodiment of the present invention;

[0029] Figure 4 An example diagram of a component application configuration page provided in an embodiment of the present invention;

[0030] Figure 5 An example diagram of a data processing model provided by an embodiment of the present invention;

[0031] Figure 6 Another schematic diagram of a process flow of a method for constructing a data processing model provided by an embodiment of the present invention;

[0032] Figure 7 A data processing model construction framework diagram provided by an embodiment of the present invention;

[0033] Figure 8 A schematic diagram of the structure of a data processing model building device provided by an embodiment of the present invention;

[0034] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variation of the terms "including" and "having" is intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Figure 1A flow chart of a method for constructing a data processing model provided by an embodiment of the present invention is provided. This embodiment is applicable to situations where non-professional users need to construct a data processing model. The method can be executed by a data processing model construction device, which can be implemented in the form of hardware and / or software, and the data processing model construction device can be configured in an electronic device. In this embodiment, the electronic device can be a computer or terminal used by the user. Figure 1 As shown, the method includes:

[0038] S101. In response to a component customization operation, a component customization page is displayed, and basic component configuration information is obtained from the component customization page.

[0039] The component customization operation is initiated by the user and is used to start the customization of the component containing the required functions. The component customization page is the relevant information page required for component customization displayed to the user on the display interface after responding to the component customization operation. The basic configuration information is the basic information required for customizing the component. For example, the component name, execution class name, component type, and input and output configurations.

[0040] Specifically, when the user determines that a component needs to be customized, the data processing model building device responds to the component customization operation, displays a component customization page, and obtains basic configuration information of the component required by the user from the component customization page.

[0041] For example, Figure 2 shown. Figure 2 This is an example diagram of a component customization page provided by an embodiment of the present invention. When a user needs to customize a component with a desired function, the user can click a "component customization operation" button on the operation interface of the electronic device, and the data processing model building device will respond to the component customization operation and display the component customization page on the display interface, as shown in FIG. Figure 2 The page shown. At this point, the user can fill in the basic configuration information of the component according to the component to be customized on the component customization page. For example, fill in "python custom-get the first n items" in the component name, fill in "python" in the custom component type, fill in "example" in the directory, fill in "PythonTestLimit" in the component execution class, and fill in or select other basic configuration information. After the user completes the filling, click the "Next" button, and the data processing model building device can obtain the basic configuration information of the component from the component customization page.

[0042] S102: Create a component function configuration template according to component basic configuration information, and use the component function configuration template to obtain component function configuration information.

[0043] The component function configuration template is a code template containing the required functions. The component function configuration information is the logic code information of the component of the required functions.

[0044] Specifically, after obtaining the basic component configuration information, the data processing model building device can create a component function configuration template with the currently required functions according to the basic component configuration information. The user can determine the logic code in the component function configuration template based on the component function configuration template, so that the data processing model building device can obtain the component function configuration information using the component function configuration template.

[0045] For example, Figure 3 shown. Figure 3 This is an example diagram of a component function configuration template provided by an embodiment of the present invention. Following the example of the previous step, when the user fills in the basic configuration information, the data processing model building device will create a component function configuration template based on the component basic configuration information, such as creating a "script dependency". Among them, the component name of the component function configuration template is as follows Figure 3 As shown, it is "PythonTestLimit.py". In addition, in the created component function configuration template, the template code of the current component will be directly shown. For example, the following code:

[0046] from dm_operators.Unit import Unit

[0047] from pyspark.sql import SparkSession

[0048] class PythonTestLimit(Unit):

[0049] “““

[0050] Custom component model code description:

[0051] The component class must inherit Unit and implement the run method.

[0052] Parameter acquisition:

[0053] In _init_, declare the parameters to be used, and then you can directly get the parameter values ​​in the program, such as: self._dc_xxx xxx represents the parameter name set in the parameter page

[0054] Input and Output:

[0055] data: the data table to be operated [df1, df2], type dataframe

[0056] return: Returns a data object, which must be of the type set in the component, such as dataframe, model, or an array if multiple outputs are present.

[0057] ”””

[0058] def_init_(self):

[0059] The above code is the component function configuration template (script dependency) of the component. Users can directly use the template to fill in and modify the logic of the components they need. For example, write component logic code.

[0060] After the user fills in and modifies the logic in the component function configuration template, clicks the "Next" button, and the data processing model building device will obtain the logic code updated by the user in the component function configuration template, that is, obtain the component function configuration information.

[0061] In this embodiment, firstly, a component function configuration template is created by pre-filling basic configuration information, so that a component configuration template containing required functions is quickly generated according to the basic configuration. Secondly, component function configuration information is obtained through the component function configuration template, so that the component function configuration information is quickly improved according to the generated component function configuration template, which reduces the difficulty for non-professionals in creating custom components and provides a basis for completing the required custom components later.

[0062] S103: Generate a component application configuration page according to the component function configuration information, obtain component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component.

[0063] The component application configuration page is a page for configuring the application of the required component. The component application configuration information is the functional items that the required component needs to implement and configure, for example, the type of parameter items that need to be configured. The customized component is a customized component that contains the functions required by the user.

[0064] Specifically, since the component function configuration information only includes the logic code of the component, it does not include the function items needed to implement the required configuration. Therefore, a component application configuration page can be generated based on the component function configuration information so that the user can configure the required function items on the component application configuration page. After the user determines the required function items on the component application configuration page, the data processing model building device will obtain the component application configuration information, save the component application configuration information, and generate a customized component.

[0065] For example, Figure 4 shown. Figure 4This is an example diagram of a component application configuration page provided by an embodiment of the present invention. Following the example of the previous step, after the data processing model building device determines the component function configuration information, it will generate a component application configuration page according to the component function configuration information, for example Figure 4 The page shown. In this page, multiple optional component application configuration information is shown, such as display components (text display, title text, tag, and prompt text, etc.), input components (single-line text, associative input, password, rich text editor, code editor, counter, filter component, and multi-line text, etc.) and selection components (drop-down selection, radio button group, multiple-choice button group, time range, date range, color selection, cascade selection, menu list, switch, slider, time selection, date selection, rating, and upload, etc.). Users can select component application configuration information with corresponding functions according to the functions required by the custom component. And, after the selection is completed, you can click Finish to save the component application configuration information and generate a customized component.

[0066] For example, each time a customized component is needed, the user selects the component application configuration information with the corresponding function, and the customized component can be used as follows: Figure 4 The component properties shown on the right side of the interface are filled in with the specific component property parameters of the component application configuration information selected this time. For example, the user can directly enter the parameter value required this time in the input box, select the required parameter value using the drop-down menu, or adjust the required parameter using the constraint adjustment item. Figure 4 As shown in the figure, the component application configuration information selected this time is a counter. In actual use, the specific configuration information of each parameter item in the component application configuration information will be displayed in the component property interface, such as the field name limit, header operation, number of titles, placeholder prompt, form grid, whether to occupy the entire row, label width 0, default value and minimum value, etc.

[0067] In this embodiment, a component application configuration page is generated according to the component function configuration information, and the component application configuration information is obtained from the component application configuration page, and the component application configuration information is saved to generate a customized component. The user can select the function items required by the component based on the visual interface, and save the function items selected by the user to generate a customized component, thereby realizing the configuration of a visualized customized component application and reducing the difficulty for the user in customizing the component with the required function.

[0068] S104. In response to the component selection operation, a component selection page is displayed, and a target component is selected from the custom components and preset components displayed on the component selection page.

[0069] The component selection operation is an operation initiated by the user to open the component selection page. The component selection page is a page for selecting components displayed to the user on the display interface after responding to the component selection operation. The preset component is a pre-set component; in this embodiment, the preset component can be a general component pre-edited by the developer. The target component includes a customized component and / or a preset component.

[0070] Specifically, when the user needs to select the required components, a component selection operation is initiated to the data processing model building device. The data processing model building device responds to the component selection operation and displays a component selection page on the display page. The user can select the components he needs on the component selection page. After the user selects the components he needs, the data processing model building device determines these selected components and uses them as target components.

[0071] Exemplarily, when a user needs to create a data processing model using different components, first, the user clicks the "component selection" operation button on the operation interface of the electronic device, and the data processing model construction device responds to the component selection operation and displays the component selection page on the display interface of the electronic device. The component selection page will show all previously customized custom components and pre-set preset components. The user can select the required components from the custom components and preset components according to the functions or other requirements of the data processing model to be created. Optionally, if the user selects a custom component, the function parameter items in the custom component can be set or adjusted according to the parameter items corresponding to the functions required this time. Optionally, if the current custom components and preset components do not have the components required by the user this time, the user can create the required custom components again using steps such as S101-S103.

[0072] After the user selects the desired component from the component selection page, the data processing model building device determines the target component.

[0073] S105. Create a data processing model according to the target component.

[0074] The data processing model is a model that can perform data processing. In this embodiment, the data processing model can be composed of multiple target components.

[0075] Specifically, after the target component is determined, a data processing model can be created based on the target component. In one implementation, the user numbers or sorts each target component, and the data processing model building device creates the data processing model based on the numbering sequence of each target component by the user. In another implementation, the user applies each target component in a visual interface and draws a data processing flow chart, and the data processing model building device can directly use the data processing flow chart as the data processing model.

[0076] For example, in one implementation, the target components may be numbered or sorted, and a data processing model may be created in the order of the numbers or sorting. For example, assuming the target components are A, B, and C, the user numbers A as 2, B as 1, and C as 3. Then the data processing model is BAC. In another implementation, if Figure 5 shown. Figure 5 This is an example diagram of a data processing model provided by an embodiment of the present invention. Assume that the target components are "olap_cube_col_m...", "filter column", "column name update" and "type conversion". At this time, the user can use these components in the visual interface to draw a data processing flow chart of these components. For example, after executing the "olap_cube_col_m..." component, execute the "filter column" component, then execute the "column name update" component, and finally execute the "type conversion" component. After drawing the data processing flow chart, the data processing flow chart can be used as a data processing model.

[0077] In this embodiment, based on customized components and preset components, components required for data processing are selected to obtain target components, and a data processing model is created according to the target components, thereby achieving fast and convenient creation of a data processing model.

[0078] The data processing model construction method provided by the embodiment of the present invention displays a component customization page in response to a component customization operation, and obtains component basic configuration information from the component customization page; creates a component function configuration template according to the component basic configuration information, and obtains component function configuration information using the component function configuration template; generates a component application configuration page according to the component function configuration information, obtains component application configuration information from the component application configuration page, and saves the component application configuration information to generate a customized component; displays a component selection page in response to a component selection operation, and determines the target component selected from the customized component and the preset component displayed on the component selection page; creates a data processing model according to the target component. In the above technical scheme, on the one hand, firstly, by pre-filling the basic configuration information, a component function configuration template is created, so that a component configuration template containing the required functions is quickly generated according to the basic configuration. Secondly, the component function configuration information is obtained through the component function configuration template, so that the component function configuration information is quickly improved according to the generated component function configuration template, which reduces the difficulty of non-professionals in creating custom components and provides a basis for completing the required custom components later. On the other hand, a component application configuration page is generated according to the component function configuration information, and the component application configuration information is obtained from the component application configuration page, and the component application configuration information is saved to generate a customized component. Based on the visual interface, users can select the functional items required by the component and save the functional items selected by the user to generate a customized component, which realizes the configuration of the visualized custom component application and reduces the difficulty for users to customize the components with the required functions. Finally, based on the customized components and preset components, the components required for data processing are selected to obtain the target components, and a data processing model is created based on the target components, which realizes the creation of a fast and convenient data processing model.

[0079] Figure 6 Another flowchart of the data processing model building method provided in an embodiment of the present invention, based on the above embodiments and feasible examples, this embodiment describes in detail the steps of creating a data processing model according to the target component and the steps after the data processing model is created.

[0080] like Figure 6 As shown, the method includes:

[0081] S601. In response to a component customization operation, a component customization page is displayed, and basic component configuration information is obtained from the component customization page.

[0082] Specifically, when the user determines that a component needs to be customized, the data processing model building device responds to the component customization operation, displays a component customization page, and obtains basic configuration information of the component required by the user from the component customization page.

[0083] S602: Create a component function configuration template according to component basic configuration information, and use the component function configuration template to obtain component function configuration information.

[0084] Specifically, after obtaining the basic component configuration information, the data processing model building device can create a component function configuration template with the currently required functions according to the basic component configuration information. The user can determine the logic code in the component function configuration template based on the component function configuration template, so that the data processing model building device can obtain the component function configuration information using the component function configuration template.

[0085] In this embodiment, firstly, a component function configuration template is created by pre-filling basic configuration information, so that a component configuration template containing required functions is quickly generated according to the basic configuration. Secondly, component function configuration information is obtained through the component function configuration template, so that the component function configuration information is quickly improved according to the generated component function configuration template, which reduces the difficulty for non-professionals in creating custom components and provides a basis for completing the required custom components later.

[0086] S603: Generate a component application configuration page according to the component function configuration information, obtain component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component.

[0087] Specifically, since the component function configuration information only includes the logic code of the component, it does not include the function parameter items required to implement the required configuration. Therefore, a component application configuration page can be generated based on the component function configuration information, so that the user can configure the required function items on the component application configuration page. After the user determines the required function items on the component application configuration page, the data processing model construction device will obtain the component application configuration information, save the component application configuration information, and generate a customized component.

[0088] In this embodiment, a component application configuration page is generated according to the component function configuration information, and the component application configuration information is obtained from the component application configuration page, and the component application configuration information is saved to generate a customized component. The user can select the function items required by the component based on the visual interface, and save the function items selected by the user to generate a customized component, thereby realizing the configuration of a visualized customized component application and reducing the difficulty of using the component when the user customizes the required function.

[0089] S604: Display a component selection page in response to the component selection operation, and determine a target component selected from the custom components and preset components displayed on the component selection page.

[0090] Specifically, when the user needs to select the required components, the component selection operation can be initiated to the data processing model building device. The data processing model building device responds to the component selection operation and displays a component selection page on the display page. The user can select the components he needs on the component selection page. After the user selects the components he needs, the data processing model building device will determine these selected components and use them as target components.

[0091] S605 , displaying multiple components on a visualization operation page, and determining a data processing flow according to visualization configuration operations performed on the multiple components on the visualization operation page.

[0092] Specifically, after determining the target component, the data processing model building device can display the target component selected by the user in a visual operation page. The user performs visual configuration operations on the target component on the visual operation page, so that the data processing model building device determines the data processing flow.

[0093] Exemplarily, the user can perform visual configuration operations such as dragging, numbering or sorting the target components on the visual operation page. For example, the target components are connected in a directed manner according to the execution order, and the data processing model construction device monitors the directed connection operations performed on multiple components, and determines the data processing flow according to the directed connection operations performed on multiple components. Among them, the directed connection operation is an operation of connecting the target components one by one in a certain direction.

[0094] In this embodiment, the visual operation interface is used to visually configure the target component and determine the data processing flow, thereby realizing rapid determination of the data processing flow.

[0095] S606: Create a data processing model according to the data processing flow and target components.

[0096] Specifically, the data processing model can be created according to the following method:

[0097] (1) In response to the component application configuration operation, a component parameter configuration page of the target component is displayed, and the component parameter configuration page includes application parameter configuration items.

[0098] Among them, the component application configuration information includes application parameter configuration items.

[0099] Specifically, after determining the data processing flow and target component, the user needs to define the function of the target component in detail. For example, it is necessary to fill in the specific parameter information of the function required by the target component in the application parameter configuration item of the component application configuration information. Therefore, the user initiates the component application configuration operation, the data processing model construction device responds to the component application configuration operation, and displays the component parameter configuration page of the target component in the visual interface.

[0100] Exemplarily, the user can click a "component application configuration" button on the visual operation interface, and the data processing model building device responds to the operation and displays the component parameter configuration page to the user.

[0101] (2) Obtain the parameter configuration value corresponding to the application parameter configuration item from the component parameter configuration page.

[0102] The parameter configuration value is filled in or modified by the user and is a specific parameter value configured for each application parameter configuration item.

[0103] Specifically, when the data processing model building device displays the component parameter configuration page of the target component on the visual interface, the user can configure the application parameter configuration item based on the component parameter configuration page displayed on the visual interface. After the user configuration is completed, the parameter configuration value corresponding to the application parameter configuration item can be obtained from the component parameter configuration page.

[0104] For example, the component parameter configuration page displays editable application parameter configuration items, and the user can configure the parameter value in the application parameter configuration item according to different configuration methods. For example, directly fill in the parameter value, select the parameter value from the drop-down menu, or drag the button to adjust the parameter value. After the user configuration is completed, the data processing model building device will obtain and save the specific content of the application parameter configuration value and configuration item filled in by the user for the target component this time from the component parameter configuration page.

[0105] (3) Create a data processing model based on parameter configuration values, data processing flow, and target components.

[0106] Specifically, according to the parameter configuration value, the parameter configuration in the target component can be improved, so that the functions of the target component are more detailed. According to the data processing flow, the execution flow between multiple target components can be clarified, making the logic between the target components clearer. Therefore, according to the parameter configuration value, the data processing flow and the target component, a data processing model of the function required by the user can be created.

[0107] In this embodiment, the application parameter configuration items in the component parameter configuration page are used to obtain the user's parameter configuration values ​​for the functions in the target component, and a data processing model is created based on the data processing flow and target component provided by the user, thereby achieving fast and convenient creation of data processing models.

[0108] S607: Load the target component.

[0109] Specifically, after determining the data processing model, if the user needs to run the data processing model, the target component can be retrieved from the component library that stores all customized components and preset components in the data processing model building device, and the target component can be loaded. At the same time, in the process of loading the target component, the parameter configuration value corresponding to the target component will also be loaded.

[0110] Exemplarily, when a user needs to run a data processing model, the data processing model construction device will perform an initialization operation, retrieve a target component from a component library, and execute the task of loading the component. By determining whether the parameters in the target component are parsed and whether the parsing is successful, it is determined whether the target component is loaded successfully. After determining that the parameters in the target component are parsed and the parsing is successful, the following steps can be continued. If the parameters in the target component are not parsed or the loading is not successful, the initialization operation is returned, or a request to the user to re-modify the target component is initiated.

[0111] S608: Call the loaded target component according to the data processing flow to execute the data processing task using the data processing model.

[0112] Specifically, according to the data processing flow provided by the user, the loaded target components are called in sequence, and the target components can process the data based on the parameter configuration values ​​to realize the execution of the data processing task using the data processing model.

[0113] For example, the user can click the "Run" button, and the data processing model construction device will run the loaded target components according to the execution order of the target components shown in the data processing flow and the parameter configuration values ​​corresponding to the target components. That is, the data processing model can be regarded as a visual underlying code of a data processing task. Running the target components in sequence according to the data processing flow is equivalent to executing the code of a complete data processing task.

[0114] In this embodiment, by loading the component, after the component is loaded successfully, the loaded target component is called according to the data processing flow, thereby realizing the use of the data processing model to execute the data processing task. While solving the problem of creating a relatively complex process for the data processing model, it also realizes the fast and efficient processing of data.

[0115] S609: Obtain task processing results.

[0116] The task processing result is the processing result after executing the data processing task.

[0117] Specifically, after executing the data processing task using the data processing model, the data processing model building device will save the operation process in the form of a log. Therefore, after each data processing task is executed, the user can retrieve the operation log and determine the task processing result.

[0118] S610: Optimize the data processing model according to the task processing result.

[0119] Specifically, after the user retrieves the operation log and determines the task processing result, the data processing model can be optimized accordingly according to the task processing result.

[0120] Exemplarily, the components of the data processing model and / or the data processing flow can be optimized according to the task processing results. For example, the user retrieves the operation data and determines that there may be problems with the key target components. Then, the preview data of the key target components can be retrieved for viewing, and the parameter items of the target components can be adjusted to optimize the data processing model. And, if there are problems with the data processing flow, the data processing flow can be modified.

[0121] In this embodiment, the task processing results after the data processing task is completed are saved, and the data processing model is optimized according to the task processing results, so that the data processing model is continuously optimized and improved, thereby improving the convenience and diversity of data processing model construction.

[0122] The data processing model construction method provided by the embodiment of the present invention displays a component customization page in response to a component customization operation, and obtains component basic configuration information from the component customization page; creates a component function configuration template based on the component basic configuration information, and obtains component function configuration information using the component function configuration template; generates a component application configuration page based on the component function configuration information, and obtains component application configuration information from the component application configuration page, and saves the component application configuration information to generate a customized component; displays a component selection page in response to a component selection operation, and determines a target component selected from the customized components and preset components displayed on the component selection page; displays multiple components on a visual operation page, and determines a data processing flow according to the visual configuration operation performed on multiple components on the visual operation page; creates a data processing model according to the data processing flow and the target component; loads the target component; calls the loaded target component according to the data processing flow to perform a data processing task using the data processing model; obtains the task processing result; and optimizes the data processing model according to the task processing result. In the above technical scheme, on the one hand, firstly, a component function configuration template is created by pre-filling the basic configuration information, so that a component configuration template containing the required functions is quickly generated according to the basic configuration. Secondly, the component function configuration information is obtained through the component function configuration template, which realizes the rapid improvement of the component function configuration information according to the generated component function configuration template, reduces the difficulty of non-professionals in creating custom components, and provides a basis for completing the required custom components later. On the other hand, the component application configuration page is generated according to the component function configuration information, and the component application configuration information is obtained from the component application configuration page, and the component application configuration information is saved to generate a customized component. The user can select the functional items required by the component based on the visual interface, and save the functional items selected by the user to generate a customized component, which realizes the configuration of the visualized custom component application and reduces the difficulty of using the component when customizing the required function. On the other hand, the target component is visually configured using the visual operation interface, and the data processing flow is determined, which realizes the rapid determination of the data processing flow. On the other hand, the application parameter configuration items in the component parameter configuration page are used to obtain the user's parameter configuration values ​​for the functions in the target component, and a data processing model is created based on the data processing flow and the target component provided by the user, which realizes the creation of a fast and convenient data processing model. Finally, by loading the component, after the component is loaded successfully, the loaded target component is called according to the data processing process, and the data processing task is performed using the data processing model. While solving the problem of creating a relatively complex process for the data processing model, the data is processed quickly and efficiently. The task processing results after the data processing task is completed are saved, and the data processing model is optimized according to the task processing results, which realizes the continuous optimization and improvement of the data processing model, thereby improving the convenience and diversity of the data processing model construction.

[0123] After introducing the data processing model construction method provided by the embodiment of the present invention, a specific implementation process is taken as an example to explain the method provided by the embodiment of the present invention in detail.

[0124] like Figure 7 shown. Figure 7 A data processing model construction framework diagram provided for an embodiment of the present invention. Specifically, the user visualization interface may be a display interface of an electronic device, such as a computer screen or a display interface of a terminal. The data processing model construction device may be configured in an electronic device, such as a computer or a user terminal. The data processing model construction device includes a data processing model management module, a data processing flow management module, a component library module, a custom component management module, and a task management log module. Among them, the data processing model management module is used to manage data processing models that have been constructed or run; the data processing flow management module is used to manage the data processing flow, for example, the data processing flow management module may be expressed in a drawing area for receiving the processing flow between target components drawn by the user in the visualization interface; the component library module is used to store all components, such as custom components created by the user through the custom component management module and preset components pre-set by the developer; the custom component management module is used to provide users with custom components, for example, according to the following Figure 2-4 The method shown completes the customized component; the task management log module is used to store the relevant operation data of the data processing model that has been run.

[0125] For example, when a user needs to process data, he can first determine whether all the target components required exist in the component library in the data processing model building device through the user visual interface. If they exist, the components in the component library can be used directly. If they do not exist, the components can be automatically managed by the component management module according to the following steps: Figure 2-4The customized component is completed in the manner shown. After the customized component is completed, the component library module saves the customized component. At this time, the user can interact with the component library module to select the component through the visual interface to determine the target component. After determining the target component, the user interacts with the data processing flow management module through the visual interface, for example, drags the target component to the drawing area, and determines the data processing flow in the drawing area through a directed connection. After determining the data processing flow, the user can choose to configure the component parameters of the target component, for example, the user can interact with the component library module through the visual interface, and fill in the required data in the application parameter configuration item in the component parameter configuration page of the target component. After determining the parameter configuration value, the data processing flow and the target component, the constructed data processing model can be generated in the data processing model management module. If the user clicks "Run" in the visual interface, the data processing model management module will complete the data processing task according to the steps of S607-S608, and feedback the relevant data of this data processing task to the task management log module. Finally, if the user needs to adjust the data processing model, the data processing model can be optimized by interacting with the task management log module, the data processing model management module, the component library module and other related modules through the visual interface. Optionally, the electronic device in the embodiment of the present invention can support stand-alone mode and cluster mode. In addition, the component library in the embodiment of the present invention can also upload other components through a third-party upload method.

[0126] Figure 8 Schematic diagram of the structure of the data processing model building device provided by the embodiment of the present invention. Figure 8 As shown, the device comprises:

[0127] The customization module 801 is used to display a component customization page in response to a component customization operation, and obtain component basic configuration information from the component customization page;

[0128] The acquisition module 802 is used to create a component function configuration template according to the component basic configuration information, and obtain the component function configuration information using the component function configuration template;

[0129] A saving module 803 is used to generate a component application configuration page according to the component function configuration information, obtain the component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component;

[0130] A determination module 804 is used to display a component selection page in response to the component selection operation, and determine a target component selected from the custom components and preset components displayed on the component selection page;

[0131] The creation module 805 is used to create a data processing model according to the target component.

[0132] Optionally, the target component includes multiple components, and the creation module 805 is specifically used for:

[0133] Display multiple components on a visual operation page, and determine a data processing flow according to visual configuration operations performed on the multiple components on the visual operation page;

[0134] Create a data processing model based on the data processing flow and target components.

[0135] Optionally, the data processing flow is determined according to the visual configuration operation performed on the multiple components on the visual operation page, and the creation module 805 is specifically used to:

[0136] Monitor directed connection operations on multiple components on the visual operation page;

[0137] The data processing flow is determined based on directed connection operations performed on multiple components.

[0138] Optionally, the component application configuration information includes application parameter configuration items, and a data processing model is created according to the data processing flow and the target component. The creation module 805 is specifically used for:

[0139] In response to the component application configuration operation, the component parameter configuration page of the target component is displayed, and the component parameter configuration page includes application parameter configuration items;

[0140] Get the parameter configuration value corresponding to the application parameter configuration item from the component parameter configuration page;

[0141] Create a data processing model based on parameter configuration values, data processing flow, and target components.

[0142] Optionally, the device further includes: an execution module, the execution module is further used to:

[0143] Load the target component;

[0144] The loaded target component is called according to the data processing flow to perform the data processing task using the data processing model.

[0145] Optionally, the execution module is further used to:

[0146] Get task processing results;

[0147] Optimize the data processing model based on the task processing results.

[0148] Optionally, the data processing model is optimized according to the task processing results, and the execution module is specifically used to:

[0149] Optimize components of the data processing model and / or the data processing flow according to the task processing results.

[0150] The data processing model building device provided in the embodiment of the present invention can execute the data processing model building method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0151] Fig. 9 A schematic diagram of the structure of an electronic device 10 provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0152] like Fig. 9 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0153] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0154] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a data processing model construction method.

[0155] In some embodiments, the data processing model building method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data processing model building method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the data processing model building method in any other appropriate manner (e.g., by means of firmware).

[0156] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing model building device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0158] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0160] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0161] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0162] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0163] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a data processing model, It is characterized in that include: In response to the component customization operation, a component customization page is displayed, and basic component configuration information is obtained from the component customization page; Creating a component function configuration template according to the component basic configuration information, and obtaining component function configuration information using the component function configuration template; Generate a component application configuration page according to the component function configuration information, obtain component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component; In response to the component selection operation, a component selection page is displayed, and a target component is selected from the custom components and preset components displayed on the component selection page; A data processing model is created according to the target component.

2. The data processing model construction method according to claim 1, It is characterized in that The target component includes a plurality of components, and the step of creating a data processing model according to the target component includes: Displaying the multiple components on a visual operation page, and determining a data processing flow according to a visual configuration operation performed on the multiple components on the visual operation page; The data processing model is created according to the data processing flow and the target component.

3. The data processing model construction method according to claim 2, It is characterized in that Determining the data processing flow according to the visual configuration operation performed on the plurality of components on the visual operation page includes: Monitoring, on the visualization operation page, directed connection operations performed on the plurality of components; The data processing flow is determined according to a directed connection operation performed on the multiple components.

4. The data processing model construction method according to claim 2, It is characterized in that The component application configuration information includes application parameter configuration items, and the step of creating the data processing model according to the data processing flow and the target component includes: In response to the component application configuration operation, a component parameter configuration page of the target component is displayed, wherein the component parameter configuration page includes the application parameter configuration items; Obtaining a parameter configuration value corresponding to the application parameter configuration item from the component parameter configuration page; The data processing model is created according to the parameter configuration values, the data processing flow and the target component.

5. The data processing model construction method according to any one of claims 2 to 4, It is characterized in that The method further comprises: Loading the target component; The loaded target component is called according to the data processing flow to execute the data processing task using the data processing model.

6. The data processing model construction method according to claim 5, It is characterized in that The method further comprises: Get task processing results; The data processing model is optimized according to the task processing result.

7. The data processing model construction method according to claim 6, It is characterized in that Optimizing the data processing model according to the task processing result includes: Optimize the components of the data processing model and / or the data processing flow according to the task processing results.

8. A data processing model building device, It is characterized in that Applied to modeling cloud, including: A customization module, used to display a component customization page in response to a component customization operation, and obtain component basic configuration information from the component customization page; An acquisition module, used to create a component function configuration template according to the component basic configuration information, and acquire component function configuration information using the component function configuration template; A saving module, used to generate a component application configuration page according to the component function configuration information, obtain component application configuration information from the component application configuration page, and save the component application configuration information to generate a customized component; A determination module, used for displaying a component selection page in response to a component selection operation, and determining a target component selected from the custom components and preset components displayed on the component selection page; A creation module is used to create a data processing model according to the target component.

9. An electronic device, It is characterized in that include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing model building method as described in any one of claims 1-7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing model construction method described in any one of claims 1 to 7 when executed.

Images