Component library-based data processing model construction method and system, and storage medium

By identifying unique IDs and performing correlation analysis on component libraries, a data processing model was constructed, enabling the data processing model to be reused. This solves the problem that data can only be output once in existing technologies, and improves construction efficiency and fault tolerance.

CN116166238BActive Publication Date: 2026-03-24HANGZHOU HUAYI ZHILIAN TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-07
Publication Date
2026-03-24

AI Technical Summary

Technical Problem

The data processing models built by existing modeling systems can only be output once and cannot be reused.

Method used

By identifying component categories using unique IDs in the component library, performing correlation analysis and tracing, identifying data storage and output components, forming a chain from level one to level N, and splicing them together into a complete chain, the recycling of data is achieved.

Benefits of technology

It enables multiple outputs and recycling of the data processing model, improving model building efficiency and reducing error rate, and supports model rollback and customization of various execution strategies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116166238B_ABST
    Figure CN116166238B_ABST
Patent Text Reader

Abstract

The application discloses a kind of data processing model construction method, system and storage medium based on component library, wherein the method can identify the category of component by the exclusive ID of storage component, then the correlation analysis is carried out to data storage class component and data output class component, to determine which data storage class component is used to store raw data, which data storage class component is used to store the data generated by operation class processing;And through unmarked, first marking, secondary marking, ……… to N-1 marking, first-level link component, second-level link component, third-level link component, ……… to N-level link component are identified in turn, and then the complete link splicing and execution order output are carried out after finding out associated node, so that the data generated after operation class processing can be processed twice, three times, etc., solve the problem that the data processed by the data processing model constructed by the existing modeling system can only be output single time, and cannot be recycled.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of model building technology, and in particular to a data processing model building method, system, and storage medium based on a component library. Background Technology

[0002] A software development model is a structural framework for the entire process, activities, and tasks of software development. Software development includes phases such as requirements, design, coding, and testing, and may also include a maintenance phase. A software development model can clearly and intuitively express the entire software development process, explicitly defining the main activities and tasks to be completed, and serves as the basis for software project work.

[0003] Patent application No. 2022108899481 discloses a model processing method, system, and storage medium for data, comprising: receiving a first component configuration instruction from a user, retrieving model components from a component library, and constructing a first data analysis model; retrieving test data, running the first data analysis model, and outputting data test results; receiving a model release instruction from a user, and releasing the first data analysis model; or, receiving a second component configuration instruction from a user, retrieving model components from the component library and constructing a second data analysis model, and retesting the model, and receiving a model release instruction from a user, and releasing the second data analysis model; or, repeating the process of adding model components and testing data analysis models until receiving a model release instruction from a user, and releasing a third data analysis model, ..., the (N-1)th data analysis model, or the Nth data analysis model.

[0004] The aforementioned patent application aims to rapidly construct data analysis models by directly retrieving model components from a pre-stored component library. It then outputs test results by retrieving test data, allowing users to compare the output with a small amount of data. The data model is only released after the user confirms its reliability and accuracy. This addresses the problem that the reliability and accuracy of models built by traditional modeling systems cannot be effectively guaranteed. While the patent application does address the issue of low reliability and accuracy in traditional models to some extent, the data processed by its model can only be output once and cannot be reused.

[0005] It is evident that existing technologies still need improvement and development. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the purpose of this invention is to provide a data processing model construction method, system and storage medium based on a component library, which aims to solve the problem that the data processed by the data processing model constructed by the existing modeling system can only be output once and cannot be reused.

[0007] The technical solution of the present invention is as follows:

[0008] A method for constructing a data processing model based on a component library, comprising:

[0009] It receives user commands to drag and drop components from the component library and displays the dragged components in the visual interface; it also receives user commands to define the dependencies and execution order between components and displays the connection relationships between the components in the visual interface until the model is built.

[0010] Based on the component-specific IDs stored in the storage table, the categories of the components in the model are identified one by one. The categories include data storage components, operation components, and data output components.

[0011] Perform correlation analysis to determine whether the target address of the data output by the data output component is the same as the storage table address of one of the data storage components. If so, mark the corresponding data output component and data storage component, and record the unique ID of the marked data output component and data storage component in the data stack area.

[0012] The results of the correlation analysis are traced back to their source. Unlabeled components are defined as first-level link components, and labeled components are defined as second-level link components. A second correlation analysis is then performed on the second-level link components to determine whether the destination address of the data output component in the second-level link component is the same as the storage table address of one of the data storage components. If so, the corresponding data output component and data storage component are relabeled and redefined as third-level link components. This process continues until the destination address of the data output component in the Nth-level link component is different from the storage table addresses of all data storage components.

[0013] Identify the associated nodes of the first-level, second-level, third-level, ... and N-level links, connect them according to their dependencies, assemble them into a complete link, and output the execution order.

[0014] The advantages of the above solution are as follows: This invention can identify the category of a component by its unique ID, and then perform correlation analysis on data storage components and data output components to determine which data storage components are used to store raw data and which are used to store data generated by operation processing. Furthermore, it identifies first-level, second-level, third-level, and so on up to N-1 levels of link components through unmarked, first-marked, second-marked, ..., N-1-marked methods. After identifying the associated nodes, it splices the complete link and outputs the execution order, allowing the data generated after operation processing to undergo secondary and tertiary processing, etc. This solves the problem that existing modeling systems can only output data once and cannot reuse it.

[0015] In a further preferred embodiment, after the step of identifying the category of each component in the model according to the component-specific ID stored in the storage table, the category includes data storage components, operation components, and data output components, the method further includes: binding the upstream and downstream components of the operation component according to the identified operation component's unique ID and the left-right point relationship of the lines displayed on the visualization interface to generate a logical link.

[0016] The above solution is effective in the following ways: by recording the line data between components, the corresponding upstream and downstream components can be found according to the left and right point relationships of the lines. The downstream relationships can also be retrieved by the lines based on the component's own ID. By combining all components with the lines, a logical link can be formed, thereby realizing the connection of the process. The whole process does not require manual association of any information, which improves the efficiency of model building and reduces the error rate.

[0017] In a further optimized scheme, after the step of identifying the associated nodes of the first-level link, second-level link, third-level link, ... and N-level link, connecting them according to the dependency relationship, splicing them into a complete link, and outputting the execution order, the scheme further includes:

[0018] The logical link generation process is mapped one-to-one, and the components within the logical link are classified into single-upstream components and multiple-upstream components according to the number of upstream components.

[0019] Parallel computation is performed on the upstream components of the multiple upstream components and the results are output to the downstream components of the multiple upstream components. Serial computation is performed on the upstream components of the single upstream component and the results are output to the downstream components of the single upstream component.

[0020] The effect of the above scheme is that by optimizing the execution order, the operation can be optimized, matching the appropriate operation mode to the components with different dependencies, and thus obtaining the optimal execution scheme.

[0021] In a further preferred embodiment, the data processing model construction method based on the component library further includes: receiving a secondary model drawing instruction issued by the user and cloning a new, independent version of the model that is completely separate from the original model.

[0022] The effect of the above solution is that, through the above solution, the new version of the model is completely separated from the original model by cloning the new version model, thereby realizing the model rollback function and improving the fault tolerance rate of the model's secondary rendering.

[0023] In a further preferred embodiment, the data processing model construction method based on the component library also includes:

[0024] Receive user commands to call debug data, search for tasks, filter executable models, extract model strategies and running rules after scheduling models, and generate execution plans;

[0025] Determine whether the generated execution plan meets the execution conditions of this task. If not, search for the task again. If it does, submit it to the asynchronous queue.

[0026] The effect of the above solution is that after the initial model construction is completed, users can customize and test various execution strategies through debugging data to determine whether the generated model can schedule task execution on demand according to different execution strategies. Based on the judgment results, the model can be improved or released.

[0027] In a further preferred embodiment, after the step of determining whether the generated execution plan meets the execution conditions of the current task, and if not, searching for the task again, and if yes, delivering it to the asynchronous queue, the method further includes: collecting the running logs of each component and summarizing them to form a complete scheduling chain and data chain for the model's running status; and if the model encounters a problem, performing cross-process log capture.

[0028] The effect of the above solution is that, through the complete scheduling chain and data chain of the model's running status, as well as the summarized running logs of each component and the logs captured across processes when problems occur, users can clearly know the problems existing in the built model, and thus quickly and accurately modify and improve the model.

[0029] A system for implementing a component library-based data processing model construction method includes a memory and a processor. The memory stores a model processing program for data analysis, and the processor runs the model processing program for data analysis to implement the component library-based data processing model construction method as described above.

[0030] A storage medium storing a computer program, which, when executed by a processor, implements the steps of the component library-based data processing model construction method described above. The storage medium includes all the technical features of the aforementioned component library-based data processing model construction method and therefore also possesses all the technical effects of the aforementioned component library-based data processing model construction method, which will not be elaborated further here.

[0031] Compared with existing technologies, the data processing model construction method based on a component library provided by this invention can identify the category of components by their unique IDs. Then, it performs correlation analysis on data storage components and data output components to determine which data storage components are used to store raw data and which are used to store data generated by operation processing. Furthermore, it identifies first-level link components, second-level link components, third-level link components, and so on up to N-1 levels of marking through no marking, initial marking, secondary marking, ... up to N-1 marking, respectively. After finding the associated nodes, it splices the complete link and outputs the execution order, enabling the data generated after operation processing to undergo secondary processing, tertiary processing, etc. This solves the problem that the data processing models constructed by existing modeling systems can only output data once and cannot be reused. Attached Figure Description

[0032] Figure 1 This is a flowchart of a data processing model construction method based on a component library provided in a preferred embodiment of the present invention.

[0033] Figure 2 This is a schematic diagram of the interface layout of the visualization interface during the model construction process in a preferred embodiment of the present invention. Detailed Implementation

[0034] This invention provides a method, system, and storage medium for constructing a data processing model based on a component library. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0035] This invention provides a method for constructing a data processing model based on a component library, such as... Figure 1 As shown, it includes:

[0036] S100: Receives user commands to drag and drop components from the component library, displays the dragged components in the visual interface; and receives user commands to define dependencies and execution order between components, displays the connection relationships between components in the visual interface until the model is built.

[0037] Figure 2The left side displays a portion of the component library, while the right side is the visual interface. The upper right side shows the selected components, their dependencies, and execution order, while the lower right side is the editing interface for custom content. Users can drag and drop the required components from the component library on the left, and then define the dependencies and execution order between components by connecting them. They can also configure the required parameters in the custom content editing interface.

[0038] S200. Based on the component-specific IDs stored in the storage table, identify the categories of the components in the model one by one. The categories include data storage components, operation components, and data output components. In this invention, the component-specific ID specifically refers to the information stored in the storage table that corresponds one-to-one with the component library and has a component category (each component has a unique ID for querying; each unique ID corresponds to a record, and this record can show what category the component belongs to. That is, when classifying, first identify the unique ID of the component to be classified, after matching the unique ID, read the corresponding record, and then identify whether the record represents a data storage component, operation component, or data output component according to the custom rules). This invention categorizes components into data storage components, operation components, and data output components to enable the server to automatically group data during subsequent data collisions, allowing two groups of data to be compared and the required information to be found. Data storage components are used for data storage; operation components are used to process the raw data according to the purpose; and data output components are used to output the processed data for storage (a new data warehouse is required for storage, hence the generation of new data storage components; for ease of distinction, the component used to store the raw data is defined as the first data storage component, and the component used to store the new data generated after processing by the operation components is defined as the second data storage component).

[0039] S300. Perform correlation analysis to determine whether the target address of the data output by the data output component is the same as the storage table address of one of the data storage components. If so, mark the corresponding data output component and data storage component, and record the unique ID of the marked data output component and data storage component in the data stack area.

[0040] In this invention, the data stack area specifically refers to an information storage area used to store / temporarily store the marked data output class components, second data storage class components, and their corresponding relationship information. It can be understood that when a data output class component outputs data, it must specify a data storage address (i.e., the target address mentioned above). Then, the storage table address of the second data storage class component storing this part of the data must be consistent. Therefore, step S400 of this invention can identify which data storage class components are first data storage class components, which data storage class components are second data storage components, and which data output class component outputs data to which second data storage class component.

[0041] S400. The results of the correlation analysis are traced back to the source. Unmarked components are defined as first-level link components, and marked components are defined as second-level link components. Second-level link components are subjected to secondary correlation analysis to determine whether the destination address of the data output component in the second-level link component is the same as the storage table address of one of the data storage components. If so, the corresponding data output component and data storage component are marked again and redefined as third-level link components. This process continues until the destination address of the data output component in the Nth-level link component is different from the storage table address of all data storage components.

[0042] The components in this step do not include operational components; they only refer to data storage and data output components. While a link originally refers to a passive, point-to-point physical connection, in this invention it specifically refers to a complete route from data storage to data output (one output marks the end of a link, excluding the storage of the output data; the storage of the output data is the beginning of the next level link). Based on the above, unmarked data storage and data output components are defined as first-level link components (meaning they process raw data); data storage and data output components marked only once are defined as second-level link components (meaning they process data output from a first-level link); data storage and data output components marked twice are defined as third-level link components (meaning they process data output from a second-level link), and so on, with data storage and data output components marked N-1 times defined as N-level link components (meaning they process data output from an N-1-level link). If no marked components are found during the tracing process, it indicates that the model only contains a first-level link (which may contain multiple parallel first-level links or only a single first-level link); if no secondary marked components are found, it indicates that the model only contains a first-level link and a second-level link (there may be multiple first-level links or only one, but it must contain a serial link from the first-level link to the second-level link), and so on. If no component marked N times is found, it indicates that the model contains links from the first level to the Nth level (at least one serial link from the first level to the Nth level); the links provided by this invention can be infinitely large, that is, the output data can be repeatedly referenced and processed.

[0043] S500: Identify the associated nodes of the first-level link, second-level link, third-level link, ... and N-level link, connect them according to their dependencies, assemble them into a complete link, and output the execution order.

[0044] Only by finding the serial relationships between links can a complete link be pieced together, ensuring the smoothness of the data processing execution order. Therefore, this invention can only proceed with the complete link assembly and execution order output after the user has initially built the model, identified the categories of each component, and conducted correlation analysis and traced the results of the correlation analysis. Thus, the various technical features in steps S100 to S500 of this invention combine to form a complete technical solution, which is an inseparable part, and achieves the following technical effects: the category of the component can be identified through the unique ID of the storage component, and then the correlation of data storage components and data output components can be performed. This analysis determines which data storage components are used to store raw data and which are used to store data generated by operation processing. Furthermore, it identifies first-level, second-level, third-level, and so on up to N-1 levels of link components through unmarked, initial-marked, second-marked, ..., N-1-level marking processes. After identifying the associated nodes, it then assembles the complete link and outputs the execution order, allowing the data generated by operation processing to undergo secondary and tertiary processing, thus solving the problem that existing modeling systems can only output data once and cannot reuse it.

[0045] In a further preferred embodiment of the present invention, after S200, the method further includes: binding the upstream and downstream components of the operation-type component according to the identified unique ID of the operation-type component and the left-right point relationship of the lines displayed on the visualization interface, thereby generating a logical link. By recording the line data between components, the corresponding upstream and downstream components can be found according to the left-right point relationship of the lines. The downstream relationship can also be retrieved through the lines based on the component's own ID. Combining all components with the lines forms a logical link, thereby realizing process chaining. The entire process does not require manual association of any information, improving the efficiency of model construction and reducing the error rate. It should be noted that the logical link is not limited to containing only one operation-type component. One output of data is considered the end of a link; all previous processing, regardless of how many times it has been processed, is within the scope of that link. Each logical link output saves data once, which is observable and usable, while the data within the logical link is only observable and not usable. Therefore, to achieve data recycling, the complete links within the model must be spliced ​​together.

[0046] According to another aspect of the present invention, the data processing model construction method based on the component library further includes: receiving a secondary model rendering instruction issued by the user, and cloning a new, independent version model that is completely separate from the original model. Through the above-described scheme, the present invention achieves complete separation between the new version model and the original model by cloning the new version model, thus realizing a model rollback function and improving the fault tolerance rate of secondary model rendering.

[0047] Furthermore, the data processing model construction method based on the component library also includes: receiving user commands to call debug data, searching for tasks, filtering executable models, extracting model strategies and operating rules after scheduling the model, and generating an execution plan; determining whether the generated execution plan meets the execution conditions of the current task; if not, searching for tasks again; if yes, submitting the plan to the asynchronous queue. After the initial model construction is completed, users can customize and test various execution strategies (such as immediate execution, single execution, and periodic execution) through debug data to determine whether the generated model can schedule task execution on demand according to different execution strategies. Based on the judgment results, the model can be improved or released. During model debugging or actual runtime, after submission to the asynchronous queue, asynchronous queue consumption will be performed first, and then the execution model will be called to modify the model's running state and achieve model execution. The system will automatically modify the model state according to the strategy.

[0048] Preferably, after the step of determining whether the generated execution plan meets the execution conditions of the current task, and if not, searching for a new task; and if yes, submitting the task to the asynchronous queue, the method further includes: collecting and summarizing the running logs of each component to form a complete scheduling chain and data chain for the model's running status; and capturing cross-process logs if problems occur in the model. Through the complete scheduling chain and data chain for the model's running status, as well as the summarized running logs of each component and the cross-process logs captured when problems occur, users can clearly understand the problems existing in the constructed model, and thus quickly and accurately modify and improve the model.

[0049] The present invention also provides a system for implementing a component library-based data processing model construction method, comprising a memory and a processor. The memory is used to store a model processing program for data analysis, and the processor is used to run the model processing program for data analysis to implement the component library-based data processing model construction method as described above.

[0050] The present invention also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the component library-based data processing model construction method described above. The storage medium includes all the technical features of the component library-based data processing model construction method described above, and therefore also possesses all the technical effects of the method, which will not be elaborated further here.

[0051] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAM bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0052] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0053] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0054] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0055] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0056] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, any of the claimed embodiments can be used in any combination.

[0057] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for constructing a data processing model based on a component library, characterized in that, include: Receive user commands to drag and drop components from the component library and display the dragged components in the visual interface; It also receives operation instructions on the dependencies and execution order between user-defined components, and displays the connection relationships between the components in the visual interface until the model is built. Based on the component-specific IDs stored in the storage table, the categories of the components in the model are identified one by one. The categories include data storage components, operation components, and data output components. Perform correlation analysis to determine whether the target address of the data output by the data output component is the same as the storage table address of one of the data storage components. If so, mark the corresponding data output component and data storage component, and record the unique ID of the marked data output component and data storage component in the data stack area. The results of the correlation analysis are traced back to their source. Unlabeled components are defined as first-level link components. The data output components in the first-level link components output data that has not been processed by the operation components. The tagged components are defined as secondary link components. Secondary correlation analysis is performed on these secondary link components to determine if the destination address of the data output component in the secondary link component is the same as the storage table address of one of the data storage components. If so, the corresponding data output component and data storage component are re-tagged and redefined as tertiary link components. This process continues until the destination address of the data output component in the Nth-level link component is different from the storage table addresses of all data storage components. A link component specifically refers to the complete route from data storage to data output, including only data storage components and data output components, excluding operation components. The data output by the data output components in the secondary, tertiary, ..., and Nth-level link components is data processed by operation components. Identify the associated nodes of the first-level, second-level, third-level, ... and N-level links, connect them according to their dependencies, assemble them into a complete link, and output the execution order.

2. The data processing model construction method based on a component library as described in claim 1, characterized in that, The step of identifying the category of each component in the model based on the component-specific ID stored in the storage table, wherein the category includes data storage components, operation components and data output components, further includes: binding the upstream and downstream components of the operation component based on the identified operation component's unique ID and the left and right point relationship of the lines displayed on the visualization interface, thereby generating a logical link.

3. The data processing model construction method based on a component library as described in claim 2, characterized in that, The steps of identifying the associated nodes of the first-level, second-level, third-level, ... and N-level links, connecting them according to their dependencies, piecing them together into a complete link, and outputting the execution order further include: The logical link generation process is mapped one-to-one, and the components within the logical link are classified into single-upstream components and multiple-upstream components according to the number of upstream components. Parallel computation is performed on the upstream components of the multiple upstream components and the results are output to the downstream components of the multiple upstream components. Serial computation is performed on the upstream components of the single upstream component and the results are output to the downstream components of the single upstream component.

4. The data processing model construction method based on a component library as described in claim 1, characterized in that, The data processing model construction method based on the component library also includes: receiving the user's instructions for secondary model rendering and cloning a new, independent version of the model that is completely separate from the original model.

5. The data processing model construction method based on a component library as described in claim 1, characterized in that, The data processing model construction method based on the component library also includes: Receive user commands to call debug data, search for tasks, filter executable models, extract model strategies and running rules after scheduling models, and generate execution plans; Determine whether the generated execution plan meets the execution conditions of this task. If not, search for the task again. If it does, submit it to the asynchronous queue.

6. The data processing model construction method based on a component library as described in claim 5, characterized in that, The step of determining whether the generated execution plan meets the execution conditions of this task, and if not, searching for a new task; and if yes, delivering the task to the asynchronous queue, also includes: collecting and summarizing the running logs of each component to form a complete scheduling chain and data chain for the model's running status; and capturing cross-process logs if a problem occurs in the model.

7. A system for implementing a data processing model construction method based on a component library, characterized in that, The system includes a memory and a processor, wherein the memory is used for a model processing program for data analysis, and the processor is used to run the model processing program for data analysis to implement the data processing model construction method based on a component library as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the data processing model construction method based on a component library as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Full-link interface test method and device, electronic equipment and storage medium

    CN114579469A

  • Model processing method and system for data and storage medium

    CN115292279A