Multi-source data automatic extraction and calling method based on data flow

Abstract

The application relates to a multi-source data automatic extraction and calling method based on DataFlow, wherein the generated source data is marked with features, in the subsequent DFD design task execution process, the multi-source data with the same attribute is called from the previously generated source data according to the task attribute, and then the matched source data is automatically extracted from the multi-source data according to the task feature; the extracted source data is the same as the task attribute and the feature of the current DFD design task, and can be directly applied to the current task; the extracted source data is directly used as the source data file of the current task, the source data file of the current task is obtained through parameter configuration, the corresponding source data of the current task does not need to be separately generated, and the task execution time is saved. Although it is a cross-type calling, the multi-source data in multiple previous tasks can be found, the multi-source data called through the business attribute and the feature is consistent with the current task, can be directly used as the source data basis of the current task, and the situation that the task is stuck due to the inconsistent data does not occur.

Description

Technical Field

[0001] This disclosure relates to the field of dataflow design technology, and in particular to a method, apparatus and electronic device for automatic extraction and retrieval of multi-source data based on DataFlow. Background Technology

[0002] Dataflow is a software application used to create ordered flowcharts that describe how data flows between different operations. Dataflow can functionally model business process diagrams provided to users, transforming them into a series of "logical model" diagrams that developers can understand. In other words, it graphically depicts the flow and processing of data in the system, and these diagrams should be described using standardized DataFlow diagrams (DFDs).

[0003] The Data Flow Diagram (DFD) design process involves decomposing data and processes layer by layer to form a multi-level DFD. A DFD includes elements such as data flows, processes, data stores, and external entities. Data flows, represented by single arrows (e.g., ->), consist of a set of fixed data components, indicating the direction of data flow. The DFD describes data flows, not control flows. Except for data flowing to or from data stores (which doesn't need to be named), each data flow must have a suitable name to reflect its meaning.

[0004] When creating a business process diagram (DFD), due to different business attributes, multiple data streams are created. The data processing corresponding to the business bus and the straight line differs, which necessitates constructing source data files of different sources / types based on the specific attributes of the business to achieve data flow coordination across multiple sources in the Dataflow application. In the DFD design process, each task corresponds to one or more data streams. While cross-task data calls—such as extracting a data stream from task A's data source file for task B's data construction—can save time in setting up branch lines for task B, this cross-task data call presents the following inconveniences:

[0005] Cross-calling source data files from different sources / types for different tasks can easily cause mismatches in business attributes between tasks. This is because although the source data files being called are structurally consistent, their business types and task attributes are different, which can easily cause data processing short circuits.

[0006] Most data calls in straddle data structures are limited to one-to-one calls and cannot discover multi-source data across multiple tasks, thus limiting their usability. Summary of the Invention

[0007] To address the aforementioned issues, this application proposes a method, apparatus, and electronic device for automatic extraction and retrieval of multi-source data based on DataFlow.

[0008] This application proposes a method for automatic extraction and retrieval of multi-source data based on DataFlow, including the following steps:

[0009] Feature-marking is performed on the source data files generated by executing the DFD design task;

[0010] Based on the first business attribute of the DFD design task, a corresponding DataFlow database is established, and the source data file with feature tags is stored in the DataFlow database corresponding to the DFD design task.

[0011] Determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute;

[0012] If it exists, search and extract multi-source data information that matches the data processing task from the DataFlow database.

[0013] As an optional embodiment of this application, the source data file generated by performing the DFD design task may optionally be characterized, including:

[0014] Pre-defined feature labeling rules are configured on the DataFlow platform;

[0015] Each pre-defined DFD design task is executed sequentially, and each source data file in each DFD design task is generated.

[0016] Based on the data characteristics of the source data files, feature tags are applied to the source data files of the DFD design task to obtain several source data files with different feature labels.

[0017] As an optional implementation of this application, optionally, a corresponding DataFlow database is established based on the first business attribute of the DFD design task, and the source data files with feature tags are stored in the DataFlow database corresponding to the DFD design task, including:

[0018] Before executing the DFD design task, obtain the first business attribute of the DFD design task;

[0019] Based on the first business attribute, create a DataFlow database in the platform corresponding to the DFD design task;

[0020] When executing the DFD design task, the source data file with feature tags generated by the task is dynamically stored in the DataFlow database corresponding to the DFD design task.

[0021] As an optional implementation of this application, optionally, a second business attribute of the current DFD design task is determined, and the second business attribute is matched with the first business attribute to determine whether there is a data processing task with the same attribute, including:

[0022] Execute the current DFD design task and obtain the second business attribute of the current DFD design task;

[0023] Match the second business attribute of the current DFD design task with the first business attribute of the previously executed DFD design task, and output the matching result;

[0024] Based on the matching results, determine whether there is a data processing task with the same attributes as the current DFD design task:

[0025] If it does not exist, continue executing the current DFD design task and generate the corresponding source data file for feature labeling.

[0026] As an optional implementation of this application, optionally, if such data exists, multi-source data information matching the data processing task is searched and extracted from the DataFlow database, including:

[0027] If there exists a previously executed DFD design task that matches the second business attribute of the current DFD design task;

[0028] Then, according to the matching business attributes, the stored multi-source data files are extracted and retrieved from the DataFlow database of the previously executed DFD design task;

[0029] The extracted and retrieved multi-source data files will serve as the data foundation for generating source data in the current DFD design task.

[0030] As an optional implementation of this application, after extracting and retrieving the multi-source data files, the method may further include:

[0031] The multi-source data file is parsed to obtain several source data files with feature tags;

[0032] The task characteristics of the current DFD design task are extracted from the data processing task.

[0033] The task characteristics of the current DFD design task are matched with the feature tags of each of the source data files to obtain all the source data files that match the task characteristics, which are then used as multi-source target data.

[0034] As an optional implementation of this application, after obtaining the multi-source target data, the method may further include:

[0035] The multi-source target data is invoked to the current DFD design task, and the data flow structure of the current DFD design task is generated;

[0036] Based on the second business attribute of the current DFD design task, the data flow structure is parameterized to obtain the source data file of the current DFD design task;

[0037] The source data file of the current DFD design task is stored in the DataFlow database corresponding to the current DFD design task.

[0038] In another aspect, this application proposes an apparatus for implementing the aforementioned DataFlow-based multi-source data automatic extraction and retrieval method, comprising:

[0039] The source data feature tagging module is used to tag the source data files generated by performing DFD design tasks.

[0040] The attribute storage module is used to establish a corresponding DataFlow database based on the first business attribute of the DFD design task, and to store the source data file with feature tags into the DataFlow database corresponding to the DFD design task.

[0041] The business attribute matching module is used to determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute.

[0042] An extraction module is used to search and extract multi-source data information that matches the data processing task from the DataFlow database, if such information exists.

[0043] In another aspect, this application also proposes an electronic device comprising:

[0044] processor;

[0045] Memory used to store processor-executable instructions;

[0046] The processor is configured to implement the DataFlow-based multi-source data automatic extraction and retrieval method when executing the executable instructions.

[0047] Technical effects of the present invention:

[0048] This application involves feature-marking the source data files generated during the execution of a DFD design task; establishing a corresponding DataFlow database based on the first business attribute of the DFD design task, and storing the feature-marked source data files in the DataFlow database corresponding to the DFD design task; determining the second business attribute of the current DFD design task, matching the second business attribute with the first business attribute to determine if a data processing task with the same attribute exists; if so, searching and extracting multi-source data information matching the data processing task from the DataFlow database. By feature-marking the previously generated source data, during subsequent execution of the DFD design task, multi-source data with the same attribute is retrieved from the previously generated source data based on the task attribute; then, based on the task characteristics, matching source data is automatically extracted from the multi-source data. The extracted source data has the same task attributes and characteristics as the current DFD design task and can be directly applied to the current task. The extracted source data is directly used as the source data file for the current task. After parameter configuration, the source data file for the current task can be obtained, and the current task does not need to generate corresponding source data separately, saving task execution time. Although it is also a cross-task call, it can discover multi-source data from multiple previous tasks. After filtering by business attributes and characteristics, the multi-source data called is consistent with the current task and can be directly used as the source data basis for the current task, so as not to cause data inconsistency and cause task lag.

[0049] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0050] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0051] Figure 1 The diagram illustrates the implementation flow of the multi-source data automatic extraction and retrieval method based on DataFlow according to the present invention.

[0052] Figure 2 The diagram illustrates the execution of the DFD design task of this invention.

[0053] Figure 3 The diagram shown is a schematic diagram of the business attribute extraction for DFD design task 3 of the present invention;

[0054] Figure 4The diagram shows the extraction of business attributes for the current DFD design task n of this invention.

[0055] Figure 5 The diagram shows the application components of the electronic device of the present invention. Detailed Implementation

[0056] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0057] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0058] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0059] Example 1

[0060] like Figure 1 As shown, this application proposes a method for automatic extraction and retrieval of multi-source data based on DataFlow, including the following steps:

[0061] S1. Mark the features of the source data files generated by executing the DFD design task;

[0062] S2. Based on the first business attribute of the DFD design task, establish a corresponding DataFlow database, and store the source data file with feature tags into the DataFlow database corresponding to the DFD design task;

[0063] S3. Determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute.

[0064] S4. If it exists, search and extract multi-source data information that matches the data processing task from the DataFlow database.

[0065] This application uses feature marking on the previously generated source data. During the subsequent execution of the DFD design task, it retrieves multiple source data with the same attributes from the previously generated source data based on the task attributes. Then, based on the task characteristics, it automatically extracts matching source data from the multiple source data. The extracted source data has the same task attributes and characteristics as the current DFD design task and can be directly applied to the current task. The extracted source data is directly used as the source data file for the current task. After parameter configuration, the source data file for the current task can be obtained. The current task does not need to generate corresponding source data separately, saving task execution time.

[0066] The implementation plan for each step will be described in detail below. In this embodiment, the current DFD design task is calculated starting from at least the third DFD design task. For example... Figure 2 The task flow shown represents the current DFD design task's flow nodes, at least DFD design task 3 and subsequent tasks. DFD design tasks 1 and 2 are created by the user.

[0067] The functions of the DataFlow platform will not be elaborated in this embodiment.

[0068] S1. Mark the features of the source data files generated by executing the DFD design task;

[0069] As an optional embodiment of this application, the source data file generated by performing the DFD design task may optionally be characterized, including:

[0070] Pre-defined feature labeling rules are configured on the DataFlow platform;

[0071] Each pre-defined DFD design task is executed sequentially, and each source data file in each DFD design task is generated.

[0072] Based on the data characteristics of the source data files, feature tags are applied to the source data files of the DFD design task to obtain several source data files with different feature labels.

[0073] In this embodiment, the source data file of the data stream generated in the previous DFD design task is saved and stored in the corresponding database established on the DataFlow platform. The source data file is the text information of the data stream generated in the current task.

[0074] To facilitate source data matching in subsequent DFD design tasks, a data feature matching method is used. Therefore, feature tags are added to each source data file in each DFD design task.

[0075] The source data files generated by each DFD design task can be tagged with the data characteristics or attributes of the data flow. For example, if the current source data 2 in DFD design task 2 is a processing procedure for financial data of department A, then source data 2 will be labeled with the characteristic tag "department finance". Source data 2 with the characteristic tag "department finance" will be stored in the corresponding DataFlow database 2. If a subsequent task also has the task attributes of DFD design task 2, such as also being a financial business processing flow, then source data 2 in DataFlow database 2 can be retrieved and directly used as the basic data for that task.

[0076] Feature labeling rules involve extracting data features from the current source data file and generating corresponding feature labels. This can be done automatically based on keywords from the current DFD design task, generating feature labels and labeling them on the corresponding source data file. Keyword extraction methods can include, for example, NLP keyword extraction.

[0077] S2. Based on the first business attribute of the DFD design task, establish a corresponding DataFlow database, and store the source data file with feature tags into the DataFlow database corresponding to the DFD design task;

[0078] As an optional implementation of this application, optionally, a corresponding DataFlow database is established based on the first business attribute of the DFD design task, and the source data files with feature tags are stored in the DataFlow database corresponding to the DFD design task, including:

[0079] Before executing the DFD design task, obtain the first business attribute of the DFD design task;

[0080] Based on the first business attribute, create a DataFlow database in the platform corresponding to the DFD design task;

[0081] When executing the DFD design task, the source data file with feature tags generated by the task is dynamically stored in the DataFlow database corresponding to the DFD design task.

[0082] like Figure 2 As shown, the source data file generated by each task execution, after feature marking, has feature tags, thus obtaining the source data file with feature tags.

[0083] Each task has its own data flow information, which needs to be stored separately. In this embodiment, based on the business attributes of the DFD design task, a DataFlow database corresponding to the DFD design task is created separately in the DataFlow platform.

[0084] The first business attribute here refers to the business attributes of the preceding tasks, and the attributes of each task may be different.

[0085] The source data file (source data) with feature tags generated by each task will be stored in the DataFlow database corresponding to the DFD design task.

[0086] for example:

[0087] Based on the business attributes of DFD design task 1, a DataFlow database 1 corresponding to DFD design task 1 is created in the platform, and the source data 1 with feature tags is stored in the DataFlow database 1.

[0088] Based on the business attributes of DFD design task 2, a DataFlow database 2 corresponding to DFD design task 2 is created in the platform, and the source data 2 with feature tags is stored in the DataFlow database 2.

[0089] etc.

[0090] S3. Determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute.

[0091] As an optional implementation of this application, optionally, a second business attribute of the current DFD design task is determined, and the second business attribute is matched with the first business attribute to determine whether there is a data processing task with the same attribute, including:

[0092] Execute the current DFD design task and obtain the second business attribute of the current DFD design task;

[0093] Match the second business attribute of the current DFD design task with the first business attribute of the previously executed DFD design task, and output the matching result;

[0094] Based on the matching results, determine whether there is a data processing task with the same attributes as the current DFD design task:

[0095] If it does not exist, continue executing the current DFD design task and generate the corresponding source data file for feature labeling.

[0096] like Figure 3As shown in this embodiment, the second business attribute refers to the current task that needs to extract data from the preceding data stream. For example, if DFD design task 1 and DFD design task 2 have already been executed, when executing DFD design task 3 (the current DFD design task), it is necessary to match the first business attributes of DFD design task 1 and DFD design task 2 one by one according to the business attribute of the current DFD design task 3 (the second business attribute, which only represents the business attribute of the next task). For example, if the second business attribute of DFD design task 3 matches the first business attribute of DFD design task 1 and DFD design task 2, then the multi-source data—source data 1 and source data 2—can be extracted and used as the source data basis for DFD design task 3 with the same business attribute.

[0097] In this embodiment, if only source data 1 or source data 2 exists, a prompt message will be issued to ask the user to confirm whether to select source data 1 or source data 2 as the source data basis for DFD design task 3 (the current DFD design task).

[0098] If the business attributes (second business attributes) of the current DFD design task 3 do not match the first business attributes of both DFD design task 1 and DFD design task 2, it indicates that the business attributes are different and source data 1 or source data 2 cannot be used. In this case, a prompt message will be issued, prompting the user to create source data 3 for DFD design task 3 separately.

[0099] S4. If it exists, search and extract multi-source data information that matches the data processing task from the DataFlow database.

[0100] like Figure 4 As shown in this embodiment, the previous "executed tasks" are used as the data basis, and the task that matches the second business attribute of the current DFD design task is found from them according to the business attributes.

[0101] As an optional implementation of this application, optionally, if such data exists, multi-source data information matching the data processing task is searched and extracted from the DataFlow database, including:

[0102] If there exists a previously executed DFD design task that matches the second business attribute of the current DFD design task;

[0103] Then, according to the matching business attributes, the stored multi-source data files are extracted and retrieved from the DataFlow database of the previously executed DFD design task;

[0104] The extracted and retrieved multi-source data files will serve as the data foundation for generating source data in the current DFD design task.

[0105] If there exists a previously executed DFD design task that matches the second business attribute of the current DFD design task n, such as if the business attributes of DFD design tasks 1, 2, and 3 in the "executed tasks" match the business attributes of the current DFD design task n, then the stored source data 1, source data 2, and source data 3 can be extracted and retrieved from DataFlow database 1, DataFlow database 2, and DataFlow database 3, respectively.

[0106] Source data 1, source data 2, and source data 3 can all serve as the data foundation for the current DFD design task n. However, given the three source data sets, it's unclear which specific data set to choose as the data foundation for the data stream generated by the current DFD design task n. In this embodiment, the selection is made based on the previously labeled feature tags.

[0107] As an optional implementation of this application, after extracting and retrieving the multi-source data files, the method may further include:

[0108] The multi-source data file is parsed to obtain several source data files with feature tags;

[0109] The task characteristics of the current DFD design task are extracted from the data processing task.

[0110] The task characteristics of the current DFD design task are matched with the feature tags of each of the source data files to obtain all the source data files that match the task characteristics, which are then used as multi-source target data.

[0111] Several multi-source data files retrieved from the database are parsed (according to the file storage format of the DataFlow platform) to obtain the corresponding source data files (multi-source) with feature tags, such as source data 1, source data 2 and source data 3.

[0112] The task characteristics of the current DFD design task n, such as the current DFD design task being a financial settlement process, are extracted to obtain the task characteristic "financial settlement". From source data 1, source data 2, and source data 3, we check which feature label matches the task characteristic "financial settlement" of the current DFD design task n. For example, if source data 2 is "departmental finance", then DFD design task 2 matches the current DFD design task n, indicating that they are the same data processing task. Source data 2 can be used as the data stream generation technology for the current DFD design task n.

[0113] If there are multiple source data with the same feature labels, then one of them can be used randomly.

[0114] As an optional implementation of this application, after obtaining the multi-source target data, the method may further include:

[0115] The multi-source target data is invoked to the current DFD design task, and the data flow structure of the current DFD design task is generated;

[0116] Based on the second business attribute of the current DFD design task, the data flow structure is parameterized to obtain the source data file of the current DFD design task;

[0117] The source data file of the current DFD design task is stored in the DataFlow database corresponding to the current DFD design task.

[0118] Source data 2 is extracted from DataFlow database 2 and used as the data flow structure basis for generating source data for the current DFD design task n. After parameter configuration, the source data file for the current DFD design task n is obtained.

[0119] Following the previous method of data storage, the source data files of the current DFD design task n are characterized and stored in the DataFlow database n.

[0120] Continue with the next DFD design task n+1.

[0121] Therefore, although this technology is also a cross-task invocation, it can discover multi-source data from multiple previous tasks. After filtering by business attributes and characteristics, the multi-source data invoked is consistent with the current task and can be directly used as the source data basis for the current task, thus avoiding data inconsistency that could cause task lag.

[0122] It should be noted that although the NLP algorithm was used as an example to illustrate keyword extraction, those skilled in the art will understand that this disclosure is not limited to this. In fact, users can flexibly set the extraction algorithm according to the actual application scenario, as long as the technical functions of this application can be achieved by following the above techniques.

[0123] Obviously, those skilled in the art should understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the control embodiments described above. Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the control embodiments described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0124] Example 2

[0125] Based on the implementation principle of Embodiment 1, this application, in another aspect, proposes an apparatus for implementing the aforementioned DataFlow-based multi-source data automatic extraction and retrieval method, comprising:

[0126] The source data feature tagging module is used to tag the source data files generated by performing DFD design tasks.

[0127] The attribute storage module is used to establish a corresponding DataFlow database based on the first business attribute of the DFD design task, and to store the source data file with feature tags into the DataFlow database corresponding to the DFD design task.

[0128] The business attribute matching module is used to determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute.

[0129] An extraction module is used to search and extract multi-source data information that matches the data processing task from the DataFlow database, if such information exists.

[0130] For details on the functions and interaction principles of each of the above modules, please refer to the description in Example 1. This example will not repeat them.

[0131] The modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.

[0132] Example 3

[0133] like Figure 5 As shown, further, in another aspect, this application also proposes an electronic device, comprising:

[0134] processor;

[0135] Memory used to store processor-executable instructions;

[0136] The processor is configured to implement the DataFlow-based multi-source data automatic extraction and retrieval method when executing the executable instructions.

[0137] This disclosure discloses an electronic device including a processor and a memory for storing processor-executable instructions. The processor is configured to implement any of the preceding DataFlow-based multi-source data automatic extraction and retrieval methods when executing the executable instructions.

[0138] It should be noted here that the number of processors can be one or more. Furthermore, the electronic device in this embodiment may also include input devices and output devices. The processor, memory, input devices, and output devices can be connected via a bus or other means, without specific limitations herein.

[0139] As a computer-readable storage medium, the memory can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the DataFlow-based multi-source data automatic extraction and retrieval method in this embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0140] Input devices can be used to receive input digital numbers or signals. These signals can be key signals related to user settings and function control of the device / terminal / server. Output devices can include display devices such as screens.

[0141] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for automatic extraction and retrieval of multi-source data based on DataFlow, characterized in that, Includes the following steps: Feature-marking is performed on the source data files generated by executing the DFD design task; Based on the first business attribute of the DFD design task, a corresponding DataFlow database is established, and the source data file with feature tags is stored in the DataFlow database corresponding to the DFD design task. Determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute; If it exists, search and extract multi-source data information that matches the data processing task from the DataFlow database; including: extracting and retrieving the stored multi-source data file from the DataFlow database; After extracting and retrieving the multi-source data files, the process also includes: The multi-source data file is parsed to obtain several source data files with feature tags; The task characteristics of the current DFD design task are extracted from the data processing task. The task features of the current DFD design task are matched with the feature tags of each source data file to obtain all the source data files that match the task features, which are then used as multi-source target data. After obtaining the multi-source target data, the process also includes: The multi-source target data is invoked to the current DFD design task, and the data flow structure of the current DFD design task is generated; Based on the second business attribute of the current DFD design task, the data flow structure is parameterized to obtain the source data file of the current DFD design task; The source data file of the current DFD design task is stored in the DataFlow database corresponding to the current DFD design task.

2. The method for automatic extraction and retrieval of multi-source data based on DataFlow according to claim 1, characterized in that, The source data files generated from the DFD design task are characterized, including: Pre-defined feature labeling rules are configured on the DataFlow platform; Each pre-defined DFD design task is executed sequentially, and each source data file in each DFD design task is generated. Based on the data characteristics of the source data files, feature tags are applied to the source data files of the DFD design task to obtain several source data files with different feature labels.

3. The method for automatic extraction and retrieval of multi-source data based on DataFlow according to claim 2, characterized in that, Based on the first business attribute of the DFD design task, a corresponding DataFlow database is established, and the source data files with feature tags are stored in the DataFlow database corresponding to the DFD design task, including: Before executing the DFD design task, obtain the first business attribute of the DFD design task; Based on the first business attribute, create a DataFlow database in the platform corresponding to the DFD design task; When executing the DFD design task, the source data file with feature tags generated by the DFD design task is dynamically stored in the DataFlow database corresponding to the DFD design task.

4. The method for automatic extraction and retrieval of multi-source data based on DataFlow according to claim 3, characterized in that, Determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute, including: Execute the current DFD design task and obtain the second business attribute of the current DFD design task; Match the second business attribute of the current DFD design task with the first business attribute of the previously executed DFD design task, and output the matching result; Based on the matching results, determine whether there is a data processing task with the same attributes as the current DFD design task: If it does not exist, continue executing the current DFD design task and generate the corresponding source data file for feature labeling.

5. The method for automatic extraction and retrieval of multi-source data based on DataFlow according to claim 4, characterized in that, If it exists, then search and extract multi-source data information that matches the data processing task from the DataFlow database, including: If there exists a previously executed DFD design task that matches the second business attribute of the current DFD design task; Then, according to the matching business attributes, the stored multi-source data files are extracted and retrieved from the DataFlow database of the previously executed DFD design task; The extracted and retrieved multi-source data files will serve as the data foundation for generating source data in the current DFD design task.

6. An apparatus for implementing the automatic extraction and retrieval method for multi-source data based on DataFlow as described in any one of claims 1-5, characterized in that, include: The source data feature tagging module is used to tag the source data files generated by performing DFD design tasks. The attribute storage module is used to establish a corresponding DataFlow database based on the first business attribute of the DFD design task, and to store the source data file with feature tags into the DataFlow database corresponding to the DFD design task. The business attribute matching module is used to determine the second business attribute of the current DFD design task, match the second business attribute with the first business attribute, and determine whether there is a data processing task with the same attribute. An extraction module is used to search and extract multi-source data information that matches the data processing task from the DataFlow database, if such information exists.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the DataFlow-based automatic extraction and retrieval method according to any one of claims 1-5 when executing the executable instructions.

Images