A method, device and electronic device for executing a model based on big data

Abstract

The present application relates to an execution method, device and electronic device of a model based on big data, wherein the model includes multiple nodes, and the method includes: obtaining a model view of the model; the model view includes nodes and node connection relationships; splitting the model view into multiple subgraphs according to the types of nodes and node connection relationships; the subgraph includes subgraph nodes and subgraph node connection relationships; splitting the subgraph into multiple subtasks according to the types of subgraph nodes and subgraph node connection relationships; combining multiple subtasks and executing the combined multiple subtasks; recording the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model. The present application reduces the consumption of storage resources and the time consumed by data storage by splitting the model and only recording and storing the data of the leaf nodes of the model and the leaf nodes of the split subgraph.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to a method, device and electronic device for executing a model based on big data. Background Art

[0002] In recent years, the trend toward big data and intelligence across the entire network has driven changes across various sectors. While providing big data services requires increasing resource investment, this requires a robust foundation, resulting in suboptimal task execution times. Improving resource utilization and the computational efficiency of big data tasks, achieving more efficient computational tasks using limited resources, and delivering a fast and unburdened user experience in the big data era are fundamental requirements.

[0003] When existing big data-based models are executed, model nodes are run one by one, and the data from these nodes is stored in the database one by one. When the target data to be screened is large, most of the time is consumed by reading and writing the result data of the screening nodes. Furthermore, when there are many items to be screened, the running data of each model node must be stored in the database, which consumes a large amount of data storage resources.

[0004] There is currently no effective solution to the problem that the operating data of each node of the model in the related art needs to be stored in the database and occupies a large amount of data storage resources. Summary of the Invention

[0005] In this embodiment, a method, device and electronic device for executing a model based on big data are provided to solve the problem in related technologies that the operating data of each node of the model needs to be stored in a database and occupies a large amount of data storage resources.

[0006] In a first aspect, this embodiment provides a method for executing a model based on big data, wherein the model includes multiple nodes, and the method includes:

[0007] Acquire a model view of the model; the model view includes nodes and node connection relationships;

[0008] Splitting the model view into a plurality of subgraphs according to the types of the nodes and the node connection relationships; the subgraphs include subgraph nodes and subgraph node connection relationships;

[0009] Splitting the subgraph into multiple subtasks according to the type of the subgraph nodes and the connection relationship between the subgraph nodes;

[0010] Combining the multiple subtasks and executing the combined multiple subtasks;

[0011] Record the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model.

[0012] In some of these embodiments, the type of the node includes the node coding language.

[0013] In some embodiments, the types of nodes include SQL nodes and non-SQL nodes.

[0014] In some embodiments, splitting the model view into multiple subgraphs according to the types of the nodes and the node connection relationships includes:

[0015] The SQL nodes in the model view are merged, and the merged result is used as the input of the non-SQL nodes, and the non-SQL nodes are used as leaf nodes of the current subgraph to obtain a subgraph.

[0016] In some embodiments, splitting the subgraph into multiple subtasks according to the types of the subgraph nodes and the connection relationships between the subgraph nodes includes:

[0017] The SQL nodes in the subgraph are merged, and the merged result is used as input of the non-SQL nodes to obtain a subtask.

[0018] In some of the embodiments, the model view is a model view generated at the business layer based on the model;

[0019] Splitting the model view into multiple sub-graphs is splitting the model view into multiple sub-graphs at a logical layer;

[0020] Splitting the subgraph into multiple subtasks is splitting the subgraph into multiple subtasks at a logical layer;

[0021] The combining of the multiple subtasks is combining the multiple subtasks at a physical layer.

[0022] In some embodiments, the node type includes an SQL class node, a Java class node, a Python class node, a Go class node, or a C / C++ class node.

[0023] In a second aspect, this embodiment provides an execution device for a big data-based model, the device comprising:

[0024] An acquisition module, configured to acquire a model view of the model; the model view includes nodes and node connection relationships;

[0025] A first splitting module is configured to split the model view into a plurality of subgraphs according to the types of the nodes and the node connection relationships; the subgraphs include subgraph nodes and subgraph node connection relationships;

[0026] A second splitting module is used to split the subgraph into multiple subtasks according to the type of the subgraph nodes and the connection relationship between the subgraph nodes;

[0027] an execution module, configured to combine the plurality of subtasks and execute the combined plurality of subtasks;

[0028] The recording module is used to record the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model.

[0029] On the third aspect, in this embodiment, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the execution method of the big data-based model described in any one of claims 1 to 7.

[0030] In a fourth aspect, a computer-readable storage medium is provided in this embodiment, on which a computer program is stored. When the computer program is executed by a processor, the steps of the execution method of the big data-based model described in any one of the first aspects are implemented.

[0031] Compared with related technologies, the execution method, device and electronic device of the big data-based model provided in this embodiment splits the model and only records and stores the data of the leaf nodes of the model and the leaf nodes of the split sub-graphs, thereby reducing the consumption of storage resources and the time consumed for data to be written to disk.

[0032] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0034] Figure 1 This is a hardware structure block diagram of a terminal that executes a method for executing a model based on big data according to this embodiment;

[0035] Figure 2 is a flow chart of a method for executing a model based on big data in this embodiment;

[0036] Figure 3 is a flow chart of a method for executing a model based on big data in this preferred embodiment;

[0037] Figure 4 It is a model view of a big data-based model in this embodiment;

[0038] Figure 5 This is a schematic diagram of model splitting in this embodiment;

[0039] Figure 6 This is another model splitting schematic diagram of this embodiment;

[0040] Figure 7 This is a schematic diagram of the physical layer node combination of the model task of this embodiment;

[0041] Figure 8 This is a structural block diagram of the execution device of the big data model in this embodiment. DETAILED DESCRIPTION

[0042] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0044] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 This is a hardware structure diagram of a terminal that executes a method for executing a model based on big data in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1Only one is shown) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0045] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the execution method of a model based on big data in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. The network may include a wireless network provided by the terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0047] In this embodiment, a method for executing a model based on big data is provided. Figure 2 This is a flow chart of an execution method of a big data-based model in this embodiment. Figure 2 As shown, the process includes the following steps:

[0048] Step S210: Acquire a model view of the model, where the model view includes nodes and node connection relationships.

[0049] Specifically, based on a pre-built model, a model view of the corresponding model is obtained. This model view includes nodes and node connections. The node type and coding language of the node can be obtained based on the node information in the model view. The node connections in the model view represent the business-layer logical processing relationships of the model. A model here refers to a formalized expression of a practical problem, objective object, or law that is abstracted. A model can be composed of one or more nodes, and the nodes of a model can be of the same or different coding languages. A model view refers to a view of the model after it has been arranged. From a user's perspective, a model view provides a specific perspective on the model data in the database. The model view is business-layer data and includes nodes, their types, their coding languages, and node connections. Exemplarily, this model can be a big data model scheduling technology used to determine target person identity information using the fastest path from massive amounts of personnel data. Each node in the model can implement an algorithm. It should be noted that the model in this embodiment is not limited to personnel identity confirmation models and can be applied to any big data model.

[0050] Step S220 : splitting the model view into multiple subgraphs according to the types of nodes and the node connection relationships; the subgraphs include subgraph nodes and subgraph node connection relationships.

[0051] Specifically, according to the type of the node and the node connection relationship of the model view, the node connection relationship is split according to the type of the node, and the model view is split into multiple subgraphs. The subgraph includes subgraph nodes and subgraph node connection relationships. The subgraph nodes in the subgraph can be nodes in the model view. The subgraph nodes include the type of subgraph nodes. The subgraph node connection relationship of the subgraph can be the node connection relationship in the model view. It should be noted that splitting the model view into multiple subgraphs here is to split the model view into multiple subgraphs at the logical layer. The logical layer is responsible for business logic separation. Since the business definition will continue to change, the logical layer is required to split the definition of the business layer into different logical units. Each logical unit is only responsible for an independent functional node. The logical layer is composed of logical units. The logical unit can be a model node or multiple model nodes. The latest node for issuing tasks to the outside. The multiple subgraphs are merged to obtain a complete model view. Exemplarily, the leaf nodes or terminal nodes of the subgraph can be nodes of the same type, or the leaf nodes or terminal nodes of the subgraph are leaf nodes or terminal nodes of the model view. Further illustratively, the splitting rule here may be to merge nodes of the same type and split the model view to obtain multiple subgraphs.

[0052] Step S230 : splitting the subgraph into multiple subtasks according to the types of the subgraph nodes and the connection relationships between the subgraph nodes.

[0053] Specifically, according to the type of the subgraph node of the subgraph and the subgraph node connection relationship, the subgraph node connection relationship is split according to the type of the subgraph node, and the subgraph is split into multiple subtasks. Each subtask can include multiple subtask nodes or one subtask node. When multiple subtasks are included, the subtask also includes a subtask node connection relationship. The subtask node of the subtask can be a node in the model view or a subgraph node in the subgraph. The subtask node connection relationship can be a node connection relationship in the model view or a subgraph node connection relationship in the subgraph. Exemplarily, the splitting rule here can be to merge nodes of the same type and split the subgraph to obtain multiple subtasks.

[0054] Step S240: combine the multiple subtasks and execute the combined subtasks.

[0055] Specifically, based on the model view and the subgraph, multiple subtasks are combined, and the combined multiple subtasks are executed. For example, based on the relationship between the model view and the subgraph and the relationship between the subgraph and the subtask, multiple subtasks are combined, the execution order and execution logic of the multiple subtasks are determined, and the combined multiple subtasks are executed.

[0056] Step S250 , recording the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model.

[0057] Specifically, according to the execution result of step S240, the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model are recorded.

[0058] When the existing big data-based models are executed, the model nodes are run one by one and the running node data are stored in the database one by one. When the target data to be screened is larger, most of the time is consumed in reading and writing the screening node result data, and when there are many items to be screened. The running data of each node of the model needs to be stored in the database, which takes up a large amount of data storage resources. In this embodiment, by splitting the model, only the data of the leaf nodes of the model and the leaf nodes of the split subgraphs are recorded and stored, which reduces the consumption of storage resources and the time consumed for data storage; at the same time, by splitting the model view, the optimal running mode of the model node data analysis model is obtained, and the model is adaptively split and reorganized, which can greatly improve the efficiency in both the model running time and storage resource consumption, while improving resource utilization and task execution efficiency.

[0059] In some embodiments, the node type includes the node coding language. For example, the node can be an SQL node or a non-SQL node, and the non-SQL node includes a Java node, a Python node, a Go node, or a C / C++ node.

[0060] In some embodiments, splitting a model view into multiple subgraphs includes: merging SQL nodes in the model view, using the merged result as input to non-SQL nodes, and using the non-SQL nodes as leaf nodes of the current subgraph to obtain a subgraph. Splitting a subgraph into multiple subtasks includes: merging SQL nodes in the subgraph, using the merged result as input to non-SQL nodes to obtain subtasks. The multiple subtasks obtained by splitting are combined and executed, and finally, the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model are recorded.

[0061] In this implementation, by merging the SQL-like nodes in the model view, splitting the model view into subgraphs, and further merging the SQL-like nodes in the subgraphs, only the data of the leaf nodes of the model and the leaf nodes of the split subgraphs are recorded and stored, which reduces the consumption of storage resources and the time consumed by data being written to the disk; at the same time, by splitting the model view, the optimal operation mode of the model node data analysis model is obtained, and the model is adaptively split and reorganized, which can greatly improve the efficiency in both model operation time and storage resource consumption, while improving resource utilization and task execution efficiency.

[0062] The present embodiment is described and illustrated below through preferred embodiments.

[0063] Figure 3 This is a flow chart of an execution method of a model based on big data in this preferred embodiment, such as Figure 3 As shown, the execution method of the model based on big data includes the following steps:

[0064] Step S310: Acquire model view data and analyze the node type of the model view data.

[0065] Specifically, model view data is obtained to determine the node type. A model can consist of one or more nodes, which can be coded in the same or different language types. Model view data is business-layer data, including node type, node coding language, and node relationships. Figure 4 This is a model view of a big data-based model in this embodiment, such as Figure 4 As shown, the model includes 11 nodes from A to K. The model includes multiple data sources and multiple types of processing nodes. The data sources include temporary data, external data and database data. Other node types include SQL nodes and non-SQL nodes. Node A is temporary data, node B is external data, node C is database data, nodes D, E, F, H, J and K are SQL nodes, and nodes G and I are non-SQL nodes.

[0066] Step S320: Analyze the combination mode of model operation based on the acquired type information of the model nodes.

[0067] Specifically, SQL nodes are merged and the result is used as the input of non-SQL nodes. Non-SQL nodes are used as leaf nodes and then perform operations with other nodes to split the model into multiple tasks. Figure 4 If all the nodes in the model are SQL nodes, they can be merged into one SQL task to run. If all the nodes are non-SQL nodes, they need to be split into multiple tasks based on the node type. The maximum split may be to split into several tasks according to the number of nodes.

[0068] The logic layer is responsible for separating business logic. Since business definitions are constantly changing, the logic layer needs to split the business layer definitions into different logical units. Each logical unit is responsible for only one independent functional node. Node types are divided into data classes, SQL classes, and non-SQL classes. Data classes and SQL can be combined into a large SQL class. The analysis process of the logic layer reorganization method is as follows: Figure 5 and Figure 6 shown.

[0069] like Figure 5 As shown, Figure 4 The model view in is split into 3 sub-views, such as Figure 5 From the first subgraph 501, the second subgraph 502 and the third subgraph 503, it can be seen that the splitting rule of the first subgraph 501, the second subgraph 502 and the third subgraph 503 is to merge the SQL nodes and use the result as the input of the non-SQL nodes. The non-SQL nodes are used as leaf nodes and then perform operations with other nodes.

[0070] like Figure 6 As shown, Figure 5 The first subgraph 501 in is split into three tasks, such as Figure 6 From the first task 601, the second task 602, and the third task 603, it can be seen that the splitting rule of the first task 601, the second task 602, and the third task 603 is to merge the SQL nodes and use the result as the input of the non-SQL nodes. The non-SQL nodes are used as leaf nodes and then perform operations with other nodes.

[0071] According to the data definition of the logical layer, the logical units are joined. The joined view can be converted into a query statement (non-SQL nodes do not need to be converted). When the leaf node is SQL, it will be converted into a CREATE TABLEAS SELET statement again. The splitting rules of the second subgraph 502 are the same as those of the first subgraph 501. The third subgraph 503 is all SQL nodes. They are regarded as a task node using the SQL merge method. The final model task physical layer node combination is as follows: Figure 7As shown, the model is split into 6 nodes, which are the first task 601 of the first subgraph 501, the second task 602 of the first subgraph 501, the third task 603 of the first subgraph 501, the fourth task 604 of the second subgraph 502, the fifth task 605 of the second subgraph 502 and the sixth task 606 of the third subgraph 503. The 6 model tasks are combined according to the structure of the model.

[0072] Step S330: After determining the combination mode of model operation, the combination result is sent for calculation in the form of a task.

[0073] In step S340 , each task only stores the result data of the leaf nodes of the subgraph and the leaf nodes of the model into the database.

[0074] In this embodiment, based on the model view, model node data is obtained, and the idea of layered design is adopted to isolate the business layer, logical layer, and physical layer for layered analysis; by analyzing the dependencies of the business layer, the logical layer splits the subgraphs and subtasks by class, and the physical layer follows up the analysis results of the logical layer to reorganize, so as to obtain a more optimal scheduling combination method and submit it to the computing resources for execution, which can save the number of execution nodes. In the above example model, this method reduces the number of running nodes from 11 to 6; the operation is represented by creating a view, and only non-SQL nodes and leaf nodes use physical tables to store data, which reduces the consumption of storage resources and the time consumed by data storage. In the above example model, this method reduces the disk node data from 11 to 3. This method of analyzing the optimal operation of the model by obtaining model node data, and adaptively splitting and reorganizing the model can greatly improve the efficiency in both model operation time and storage resource consumption, achieving both resource utilization and task execution efficiency.

[0075] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0076] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0077] In this embodiment, an execution device for a model based on big data is also provided, which is used to implement the above-mentioned embodiments and preferred implementation methods. The above-mentioned descriptions will not be repeated here. The terms "module", "unit", "sub-unit", etc. used below can be a combination of software and / or hardware that can implement the predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0078] Figure 8 This is a structural block diagram of the execution device of the big data model of this embodiment, such as Figure 8 The device comprises:

[0079] An acquisition module 810 is configured to acquire a model view of the model; the model view includes nodes and node connection relationships;

[0080] A first splitting module 820 is configured to split the model view into a plurality of subgraphs according to the types of the nodes and the node connection relationships; the subgraphs include subgraph nodes and subgraph node connection relationships;

[0081] A second splitting module 830 is configured to split the subgraph into a plurality of subtasks according to the types of the subgraph nodes and the connection relationships between the subgraph nodes;

[0082] An execution module 840 is configured to combine the plurality of subtasks and execute the combined plurality of subtasks;

[0083] The recording module 850 is used to record the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model.

[0084] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0085] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0086] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0087] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0088] S1, obtaining a model view of the model; the model view includes nodes and node connection relationships;

[0089] S2, split the model view into multiple subgraphs according to the node type and node connection relationship; the subgraph includes subgraph nodes and subgraph node connection relationships;

[0090] S3, split the subgraph into multiple subtasks according to the type of subgraph nodes and the connection relationship between subgraph nodes;

[0091] S4, combining multiple subtasks and executing the combined multiple subtasks;

[0092] S5, recording the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model.

[0093] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0094] In addition, in conjunction with the execution method of the big data model provided in the above embodiments, a storage medium may also be provided in this embodiment for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, it implements the steps of any of the execution methods of the big data model provided in the above embodiments.

[0095] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0096] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0097] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0098] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for executing a model based on big data, characterized in that: The model includes a plurality of nodes, and the method includes: Obtaining a model view of the model; the model view includes nodes and node connection relationships; the model view is generated based on the model at the business layer; splitting the model view into multiple subgraphs based on the node types and the node connection relationships; the subgraphs include subgraph nodes and subgraph node connection relationships; Splitting the subgraph into multiple subtasks according to the type of the subgraph nodes and the connection relationship between the subgraph nodes; Combining the multiple subtasks at the physical layer according to the relationship between the model view and the subgraph and the relationship between the subgraph and the subtask, and executing the combined multiple subtasks; Recording the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model; The types of nodes include data sources, SQL nodes and non-SQL nodes; Splitting the model view into multiple subgraphs according to the node type and the node connection relationship includes: Merging the SQL nodes in the model view at the logic layer, using the merged result as input of the non-SQL nodes, and using the non-SQL nodes as leaf nodes of the current subgraph to obtain a subgraph; The step of splitting the subgraph into multiple subtasks according to the types of the subgraph nodes and the connection relationships between the subgraph nodes includes: At the logic layer, the SQL nodes in the subgraph are merged, and the merged result is used as the input of the non-SQL nodes to obtain the subtask.

2. The execution method of the model based on big data according to claim 1, characterized in that: The type of the node includes the node coding language.

3. The execution method of the big data-based model according to claim 2, characterized in that: The node type includes an SQL class node, a Java class node, a Python class node, a Go class node or a C / C++ class node.

4. An execution device for a model based on big data, characterized in that: The device comprises: an acquisition module for acquiring a model view of the model; the model view includes nodes and node connection relationships; the model view is generated based on the model at the business layer; a first splitting module is used to split the model view into multiple subgraphs based on the types of the nodes and the node connection relationships; the subgraphs include subgraph nodes and subgraph node connection relationships; A second splitting module is used to split the subgraph into multiple subtasks according to the type of the subgraph nodes and the connection relationship between the subgraph nodes; An execution module, configured to combine the plurality of subtasks at the physical layer according to the relationship between the model view and the subgraph and the relationship between the subgraph and the subtask, and execute the combined plurality of subtasks; A recording module, used to record the execution result data of the leaf nodes of the subgraph and the leaf nodes of the model; The types of nodes include data sources, SQL nodes and non-SQL nodes; Splitting the model view into multiple subgraphs according to the node type and the node connection relationship includes: Merging the SQL nodes in the model view at the logic layer, using the merged result as input of the non-SQL nodes, and using the non-SQL nodes as leaf nodes of the current subgraph to obtain a subgraph; The step of splitting the subgraph into multiple subtasks according to the types of the subgraph nodes and the connection relationships between the subgraph nodes includes: At the logic layer, the SQL nodes in the subgraph are merged, and the merged result is used as the input of the non-SQL nodes to obtain the subtask.

5. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the execution method of the big data-based model according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the execution method of the big data-based model described in any one of claims 1 to 3 are implemented.

Images