Data processing methods, equipment and storage media

Abstract

This application discloses a data processing method, apparatus, and storage medium. The data processing method includes: determining the attribute information of each data source field, and based on the attribute information of each data source field, determining the processing operator to process the data source field; then, displaying the data source fields and processing operators on a display page, and connecting each data source field and its corresponding processing operator to obtain a logic diagram; further, parsing the logic diagram to obtain an analytical diagram, and calling the processing operators in the analytical diagram to process each data source field to obtain the target field corresponding to each data source field. Through the above method, the data processing method provided by this application can realize data processing on a visual page, and the processing logic from data source field to target field can be freely arranged on the visual page.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus and storage medium. Background Technology

[0002] With the development of technology, big data has become an important asset for all industries. Analyzing and mining useful value from massive amounts of data requires efficient and easy-to-use tools to support it, so as to fully realize the business value of big data analysis and mining. However, at present, when exploring data, it is inevitable to perform field-level data processing and transformation operations to meet business needs. Summary of the Invention

[0003] The main technical problem addressed by this application is to provide a data processing method, device, and storage medium that can provide a visual data processing approach and allow for the free arrangement of data processing logic.

[0004] To address the aforementioned technical problems, this application provides a data processing method comprising: determining the attribute information of each data source field, and based on the attribute information of each data source field, determining the processing operator for processing the data source field; then, displaying the data source field and the processing operator on a display page, and connecting each data source field and its corresponding processing operator to obtain a logic diagram; further, parsing the logic diagram to obtain an analytical diagram, and calling the processing operator in the analytical diagram to process each data source field to obtain the target field corresponding to each data source field.

[0005] The attribute information of the data source field includes at least one of the following: field type, field description, and field name.

[0006] Specifically, based on the attribute information of each data source field, the processing operator for processing the data source field is determined, including: based on the attribute information of each data source field, traversing all processing operators in the operator library to determine the attribute information of the input end and the attribute information of the output end of each processing operator in the operator library; comparing the attribute information of each data source field with the attribute information of the input end and the attribute information of the output end of each processing operator in the operator library to determine the processing operator corresponding to the data source field.

[0007] The process involves displaying data source fields and processing operators on the display page, and connecting each data source field and its corresponding processing operator to obtain a logic diagram. This includes: responding to a user's selection command by displaying the selected data source field and processing operator on the display page; and responding to a user's connection command by connecting the data source fields and processing operators on the display page to obtain a logic diagram.

[0008] The logical diagram contains at least one data source field and at least one processing operator.

[0009] In the logic diagram, the data source fields and processing operators are connected in series, and the processing operators in the logic diagram are connected in series or in parallel.

[0010] The process of parsing the logic diagram, before obtaining the parsed diagram, includes: obtaining the configuration parameters of all processing operators in the logic diagram, as well as the connection structure between the data source fields and all processing operators; obtaining the first information of the prediction target field based on the configuration parameters of all processing operators in the logic diagram, as well as the connection structure between the data source fields and all processing operators; obtaining the second information of the real target field based on each data source field; and verifying the validity of the logic diagram based on the comparison results of the first and second information.

[0011] The process of obtaining the configuration parameters of all processing operators in the logic diagram includes: traversing all processing operators contained in the logic diagram; and responding to the user's configuration instructions based on the display page to obtain the configuration parameters of each processing operator.

[0012] The verification of the validity of the logic diagram based on the comparison result of the first information and the second information includes: in response to the comparison result of the first information and the second information showing that the first information and the second information are consistent, determining that the configuration parameters of all processing operators in the logic diagram and the connection structure contained in the logic diagram are valid.

[0013] The process involves parsing the logic diagram to obtain an analytical diagram, including: identifying the end-processing operators contained in the logic diagram, where the end-processing operators are those whose outputs are not connected to other processing operators or data source fields; identifying the front-end data source fields in the logic diagram corresponding to the end-processing operators, where the front-end data source fields are those whose inputs are not connected to other processing operators or data source fields; identifying all unidirectional connection logic diagrams for each front-end data source field based on the end-processing operators and their corresponding front-end data source fields; and parsing all unidirectional connection logic diagrams for each front-end data source field to obtain the analytical diagram.

[0014] Specifically, the process involves parsing all unidirectional connection logic graphs for each front-end data source field to obtain an analytical graph. This includes: determining the connection structure of all unidirectional connection logic graphs for each front-end data source field; parsing the connection structure of all unidirectional connection logic graphs for each front-end data source field to obtain the logical operation method corresponding to each unidirectional connection logic graph; and determining the analytical graph for each front-end data source field based on the real-time performance parameters corresponding to the logical operation method.

[0015] Real-time performance parameters include one or more of response time and running speed.

[0016] Specifically, based on the real-time performance parameters corresponding to each logical operation method, the analytical graph of each front-end data source field is determined, including: determining the unidirectional connection logical graph corresponding to the logical operation method with the shortest response time and / or the fastest running speed as the analytical graph corresponding to each front-end data source field.

[0017] Specifically, the process involves calling the processing operators within the analytical graph to process the fields of each data source, thereby obtaining the target fields corresponding to each data source field. This includes: the configuration parameters of the processing operators based on the analytical graph and the configuration parameters of the data source fields, as well as the connection structure between the processing operators and the data source fields, determining the logical operation method corresponding to each analytical graph; and using the logical operation method, calling the processing operators within the analytical graph to process the data of each data source field, thereby obtaining the target fields corresponding to each data source field.

[0018] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a data processing device, which includes a memory and a processor. The memory is used to store program data, and the processor is used to execute the program data to implement the data processing method described above.

[0019] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium that stores program data, which, when executed by a processor, is used to implement the data processing method described above.

[0020] The beneficial effects of this application are as follows: Unlike existing technologies, the data processing method provided in this application determines the attribute information of each data source field and, based on this attribute information, determines the processing operators for processing the data source fields. Next, the data source fields and processing operators are displayed on a display page, and each data source field and its corresponding processing operator are connected to obtain a logic diagram. Furthermore, the logic diagram is parsed to obtain an analytical diagram, and the processing operators within the analytical diagram are called to process each data source field, resulting in the target field corresponding to each data source field. Through this method, the process of obtaining the target field from the data source field can be implemented on a visual page, and the processing logic between the data source field and the target field can be freely arranged based on the visual page. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0022] Figure 1 This is a flowchart illustrating the first embodiment of the data processing method provided in this application;

[0023] Figure 2 This is a schematic diagram of an embodiment of the user display page provided in this application;

[0024] Figure 3 This is a flowchart illustrating step 14 provided in this application;

[0025] Figure 4 This is a flowchart illustrating the second embodiment of the data processing method provided in this application;

[0026] Figure 5 This is a schematic diagram of the structure of an embodiment of the data processing device provided in this application;

[0027] Figure 6 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0029] See Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data processing method provided in this application. The method includes:

[0030] Step 11: Determine the attribute information of each data source field.

[0031] Specifically, the attribute information of the data source field includes at least one of the following: field type, field description information, and field name.

[0032] Step 12: Based on the attribute information of each data source field, determine the processing operators for data processing of the data source fields.

[0033] In some embodiments, the type of processing operator that can be connected to the output of a data source field can be determined based on the attribute information of the data source field.

[0034] In some embodiments, a processing operator is a unified abstraction and encapsulation of different data processing logics.

[0035] Optionally, the processing operators include string processing operators, arithmetic operation operators, and date and time processing operators.

[0036] Among them, string processing operators can be used to replace the content of data source fields with input characters or strings; they can also extract strings at specified positions; they can concatenate multiple data source fields; they can supplement the content of connected data source fields; and they can validate the content of connected data source fields, etc.

[0037] For example, string processing operators can be used to convert the content of the connected data source field from text to string; string processing operators can also be used to segment and extract the content of the connected data source field to obtain a specific segmented data source field.

[0038] Among them, arithmetic operators can perform addition, subtraction, multiplication, and division calculations on the contents of multiple connected data source fields to generate a new target field that is different from the original data source field; they can also retain decimal places on the data of the connected data source fields; and they can perform arithmetic operations on the contents of the connected data source fields and the input values, etc.

[0039] For example, using arithmetic operators to perform addition operations on at least two data source fields connected to it, a new data source field is obtained, which is formed by connecting at least two data source fields.

[0040] In some embodiments, to perform addition on at least two data source fields, the data source fields can first be converted into strings using string processing operators, then the strings can be converted into numbers for multiplication, and finally the resulting numbers can be converted back into strings. The resulting string is the result of addition using arithmetic operators.

[0041] It is worth noting that the data source fields used for arithmetic operations are of the same type.

[0042] Among them, the date and time processing operators can perform format conversion on the date and time of the connected data source fields; they can also extract the year, quarter, month, day, hour, minute, and second of the date and time of the connected data source fields; and they can also perform difference calculations on the date and time of at least two connected data source fields.

[0043] In some embodiments, the data source field is first converted into a string using string processing operators, and then the date and time format of the string is converted using date and time processing operators, such as converting "April, 1, 2022" into "4 / 1 / 2022".

[0044] It is worth noting that in some embodiments, the processing operator also includes a custom operator, which supports custom processing operators. Users develop operators in a pre-defined format of custom functions to implement specific computational logic.

[0045] In some embodiments, step 12 above may include the following steps (not shown):

[0046] Step 121: Based on the attribute information of each data source field, traverse all processing operators in the operator library to determine the attribute information of the input end and the attribute information of the output end of each processing operator in the operator library.

[0047] Specifically, the input and output of each processing operator are related to its attribute information. When the attribute information determines that the processing operator is of a certain type, the data / strings that the input and output of the processing operator can receive are fixed. For example, the output of a string processing operator can only be a string; the input and output of a date and time processing operator can only be data related to numbers, such as time / date.

[0048] Step 122: Compare the attribute information of each data source field with the attribute information of the input end and the attribute information of each processing operator in the operator library to determine the processing operator corresponding to the data source field.

[0049] In some embodiments, by comparing the attribute information at the input end of a processing operator with the attribute information at the output end of a data source field, it can be determined which processing operator can be connected to the input end of a data source field.

[0050] Step 13: Display the data source fields and processing operators on the display page, and connect each data source field and its corresponding processing operator to obtain a logic diagram.

[0051] In some embodiments, step 13 above may include the following steps (not shown):

[0052] Step 131: In response to the user's selection command, display the data source field and processing operator selected by the selection command on the display page.

[0053] Step 132: In response to the user's connection command, connect the data source fields and processing operators on the display page to obtain a logical graph, wherein the logical graph contains at least one data source field and at least one processing operator.

[0054] In some embodiments, the display page includes data source fields, processing operators and an orchestration canvas, operator configuration bars, etc. Figure 2As shown, the user display page is applied to the target platform (such as laptops, mobile phones, desktop computers, etc.). The user display page includes a list of data source fields, a list of processing operators, an arrangement canvas, and a configuration panel. The left side of the display page shows the list of data source fields and the list of processing operators. The data source field list contains several encapsulated data source fields, and the processing operator list contains several encapsulated processing operators. The middle of the display page is the arrangement canvas, where users can drag and drop data source fields and processing operators onto the canvas. Users can also connect data source fields and processing operators, and processing operators to each other, with connections being unidirectional. The right side of the display page is the configuration panel, which allows users to view the attribute information of the data source fields and set the attribute information of the processing operators.

[0055] For example, if the processing operator connected to the data source field is a string processing operator, you can select the processing operator on the layout canvas in the display page. The configuration page of the processing operator will then be displayed in the configuration bar on the right. You can configure the attribute information of the processing operator based on the configuration page. If you need to use the string processing operator to "truncate" the data source field, you can set the start and end positions to "2-4". The result of the data source field processed by the processing operator will be the string from the 2nd to the 4th position of the data source field.

[0056] In some embodiments, the arrangement canvas is equipped with "View Results" and "View Log" buttons. Users can use the "View Results" button to check at any time whether the data source field obtained after processing by the processing operator is the target field, and users can use the "View Log" button to check the data processing process at any time.

[0057] In some embodiments, the data source fields and processing operators in the logic diagram are connected in series, and the processing operators can be connected in series or in parallel.

[0058] In some embodiments, a data source field can be connected to multiple processing operators.

[0059] Step 14: Analyze the logic diagram to obtain the analytical diagram.

[0060] In some embodiments, the method of obtaining an analytical graph by parsing a logic graph is as follows: Figure 3 As shown, the specific steps include:

[0061] Step 31: Based on the logic diagram, determine the end processing operators contained in the logic diagram, where the end processing operator is the processing operator whose output end is not connected to other processing operators or data source fields.

[0062] Step 32: Based on the end processing operator, determine the front-end data source field in the logic diagram that corresponds to the end processing operator. The front-end data source field is the data source field in the logic diagram whose input end is not connected to other processing operators or data source fields.

[0063] Step 33: Based on the end-processing operator and the corresponding front-end data source field, determine all the unidirectional connection logic diagrams for each front-end data source field.

[0064] Specifically, each unidirectional connection logic graph contains a data source field and at least one processing operator, wherein the data source field serves as the front-end data source field of the unidirectional connection logic graph, and at least one of the processing operators has an output end without a connected data source field / processing operator, serving as the end processing operator of the unidirectional connection logic graph.

[0065] It is worth noting that the connection structure of each unidirectional connection logic diagram can only be a series connection; there are no parallel or ring structures.

[0066] In some embodiments, the unidirectional link logic graph is a directed acyclic graph.

[0067] Step 34: Analyze all unidirectional connection logic diagrams for each front-end data source field to obtain the analytical diagram.

[0068] In some embodiments, step 34 above may include the following steps (not shown):

[0069] Step 341: Determine the connection structure of all unidirectional connection logic diagrams for each front-end data source field.

[0070] In some embodiments, the connection structure between the data source field and the processing operator, and the connection structure between processing operators, are unidirectional.

[0071] Step 342: Analyze the connection structure of all unidirectional connection logic diagrams for each front-end data source field to obtain the logical operation method corresponding to each unidirectional connection logic diagram.

[0072] In some embodiments, by parsing the one-way connection structure, the logical operation method corresponding to the one-way connection structure can be obtained. The logical operation method can be an operation formula, such as "data source field → data processing using string processing operators → multiplication operation using arithmetic operation operators...".

[0073] In some embodiments, the logical operation method corresponding to the unidirectional connection logic diagram can be obtained by parsing the unidirectional connection structure and performing step pruning and predicate pushdown.

[0074] In some embodiments, by pruning the steps of the unidirectional connection structure, redundant steps can be eliminated to reduce acquisition time.

[0075] In some embodiments, predicate pushdown is performed on the unidirectional connection structure, and the corresponding logical operation method is obtained using the corresponding algorithm.

[0076] Step 343: Based on the real-time performance parameters corresponding to the logical operation methods, determine the parsing diagram of each front-end data source field.

[0077] In some embodiments, real-time performance parameters include one or more of response time and running speed.

[0078] In some embodiments, the unidirectional connection logic diagram corresponding to the logic operation method with the shortest response time and / or the fastest running speed is determined as the parsing diagram corresponding to each front-end data source field.

[0079] In some embodiments, the time required to obtain logical operations is shortened by comparing the time required to obtain them using a step-by-step pruning method with the normal method.

[0080] In some embodiments, the time and response speed of obtaining logical operations through predicate pushdown are shorter and faster compared to the normal acquisition method.

[0081] Step 15: Based on the analytical graph, call the processing operators to process each data source field to obtain the target field corresponding to each data source field.

[0082] In some embodiments, step 15 above may include the following steps (not shown):

[0083] Step 151: Based on the configuration parameters of the processing operators and the configuration parameters of the data source fields in the analytical graph, as well as the connection structure between the processing operators and the data source fields, determine the logical operation method corresponding to each analytical graph.

[0084] Understandably, the configuration parameters of different processing operators are related to the attribute information of that processing operator.

[0085] Step 152: Using logical operations, based on the analytical graph, call the processing operators to process the data of each data source field to obtain the target field corresponding to each data source field.

[0086] In some embodiments, users can drag and drop data source fields and processing operators on the display page to generate a node in the arrangement canvas. Clicking the output of the selected upstream node (e.g., clicking the output of the selected data source field) and connecting it to the input of the downstream node (e.g., clicking the input of the selected string processing operator) allows users to continue these steps, arranging multiple nodes (which can be data source fields or processing operators) to form a directed acyclic graph. Right-clicking and double-clicking each node displays the corresponding configuration page in the configuration panel on the right. Different processing operators have different functions, and the displayed configuration pages will also differ accordingly. Users can input different parameter values ​​based on their business type. After configuration, clicking the "Execute" button requests a connection to the server interface. The server verifies and parses the received logical graph and further processes the data to obtain the target field corresponding to the data source field.

[0087] In some embodiments, the logical operation method and the target platform's environment information are submitted to the execution engine for conversion to obtain a specific physical operation method, and the physical operation method is executed to realize the layer-by-layer processing of the data source fields, and finally obtain the target field.

[0088] The execution engine refers to the scheduling and computation service for executing operator logic, specifically the distributed computing engine Spark, which is used to perform computations on orchestrated logic.

[0089] Unlike existing technologies, the data processing method provided in this application can realize the process of obtaining the target field from the data source field on a visualization page, and the processing logic between the data source field and the target field can be freely arranged based on the visualization page.

[0090] See Figure 4 , Figure 4 This is a flowchart illustrating a second embodiment of the data processing method provided in this application. The method includes:

[0091] Step 41: Determine the attribute information of each data source field.

[0092] Step 42: Based on the attribute information of each data source field, determine the processing operators for data processing of the data source fields.

[0093] In some embodiments, the processing operators include string processing operators, arithmetic operation operators, date and time processing operators, and custom processing operators.

[0094] Step 43: Display the data source fields and processing operators on the display page, and connect each data source field and its corresponding processing operator to obtain a logic diagram.

[0095] In some embodiments, custom processing operators are used as plugins on the display page.

[0096] In some embodiments, using a custom processing operator requires downloading a custom processing operator template, then developing the operator in a local environment (application platform environment) according to the agreed format, compiling and packaging it, then uploading the custom processing operator plugin package, and registering the custom processing operator in the system so that the custom processing operator can be used on the orchestration canvas.

[0097] Steps 41 to 43 can have the same or similar technical solutions as the above embodiments, and will not be repeated here.

[0098] Step 44: Obtain the configuration parameters of all processing operators in the logical diagram, as well as the connection structure between the data source fields and all processing operators.

[0099] In some embodiments, obtaining the configuration parameters of all processing operators in the logic diagram specifically includes the following steps (not shown in the figure):

[0100] S1: Traverse all processing operators contained in the logic diagram.

[0101] S2: In response to the user's configuration instructions based on the display page, obtain the configuration parameters of each processing operator.

[0102] In some embodiments, configuration instructions are used to configure a processing operator to obtain the attribute information of the processing operator.

[0103] Step 45: Based on the configuration parameters of all processing operators in the logical diagram, as well as the connection structure between the data source field and all processing operators, obtain the first information of the prediction target field.

[0104] In some embodiments, the first information for predicting the target field is attribute information.

[0105] Step 46: Obtain the second information of the actual target field based on the fields of each data source.

[0106] In some embodiments, the second information of the actual target field is attribute information.

[0107] Step 47: Based on the comparison results of the first and second information, verify whether the logic diagram is valid.

[0108] In some embodiments, it is necessary to verify whether the logical graph is a directed acyclic graph and whether the attribute information of the data source fields and processing operators contained in the logical graph is valid.

[0109] For example, if the logic graph is not a directed acyclic graph, the verification result will show that the logic graph is invalid.

[0110] For example, if the attribute information of the data source field is invalid and the attribute information of the processing operator is invalid, the verification result will show that the logic diagram is invalid.

[0111] For example, if the logical graph is not a directed acyclic graph, or if the attribute information of the data source fields and the attribute information of the processing operators are invalid, the verification result will show that the logical graph is invalid.

[0112] In some embodiments, in response to the comparison result of the first information and the second information showing that the first information and the second information are consistent, it is determined that the configuration parameters of all processing operators in the logic diagram and the connection structure contained in the logic diagram are valid.

[0113] If valid, proceed to step 48. If inconsistent, no further steps are required.

[0114] Step 48: Analyze the logic diagram to obtain the analytical diagram.

[0115] Step 49: Based on the analytical graph, call the processing operators to process each data source field to obtain the target field corresponding to each data source field.

[0116] Steps 48 and 49 can have the same or similar technical solutions as the above embodiments, and will not be repeated here.

[0117] Unlike existing technologies, the data processing method provided in this application can process data from data source fields based on the display page to obtain target fields. Furthermore, the way from data source fields to target fields can be obtained through free arrangement, which is highly flexible and supports data processing from multiple data source fields, resulting in high efficiency.

[0118] See Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the data processing device provided in this application. The data processing device 50 includes a memory 501 and a processor 502. The memory 501 is used to store program data, and the processor 502 is used to execute the program data to implement the data processing method as described above, which will not be repeated here.

[0119] See Figure 6 , Figure 6 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this application. The computer-readable storage medium 60 stores program data 601. When the program data is executed by the processor, it is used to implement the data processing method as described above, which will not be repeated here.

[0120] The processor involved in this application may be referred to as a CPU (Central Processing Unit), which may be an integrated circuit chip, or a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0121] The storage media used in this application include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), or optical discs.

[0122] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A data processing method, characterized by, The method includes: Determine the attribute information of each data source field; Based on the attribute information of each of the data source fields, determine the processing operators for processing the data source fields; The data source fields and the processing operators are displayed on the display page, and each data source field and the corresponding processing operator are connected to obtain a logic diagram; Based on the logic diagram, the end processing operators contained in the logic diagram are determined, wherein the end processing operators are processing operators whose output ends are not connected to other processing operators or data source fields; Based on the end processing operator, a front-end data source field corresponding to the end processing operator in the logic diagram is determined, wherein the front-end data source field is a data source field in the logic diagram whose input end is not connected to other processing operators or data source fields; Based on the end-processing operator and the corresponding front-end data source field, determine all unidirectional connection logic graphs for each front-end data source field; Determine the connection structure of all the unidirectional connection logic graphs for each of the aforementioned front-end data source fields; The connection structure of all the unidirectional connection logic diagrams of each of the aforementioned front-end data source fields is parsed to obtain the logical operation method corresponding to each of the unidirectional connection logic diagrams; Based on the real-time performance parameters corresponding to the logical operation method, the parsing diagram of each of the front-end data source fields is determined; wherein, the real-time performance parameters include one or more of response time and running speed; Based on the analytical graph, the processing operators therein are invoked to process each of the data source fields to obtain the target fields corresponding to each of the data source fields.

2. The method of claim 1, wherein, The attribute information of the data source field includes at least one of the following: field type, field description information, and field name; The step of determining the processing operator for data processing of the data source fields based on the attribute information of each of the data source fields includes: Based on the attribute information of each data source field, all processing operators in the operator library are traversed to determine the attribute information of the input end and the attribute information of the output end of each processing operator in the operator library. The attribute information of each data source field is compared with the attribute information of the input end and the attribute information of each processing operator in the operator library to determine the processing operator corresponding to the data source field.

3. The method according to claim 1, characterized in that, The step of displaying the data source fields and the processing operators on the display page, and connecting each data source field and its corresponding processing operator to obtain a logic diagram, includes: In response to the user's selection command, the data source field and processing operator selected by the selection command are displayed on the display page; In response to a user's connection command, the data source fields and processing operators on the display page are connected to obtain a logic graph, wherein the logic graph contains at least one data source field and at least one processing operator.

4. The method of claim 3, wherein, The data source fields and processing operators in the logic diagram are connected in series, and the processing operators in the logic diagram are connected in series or in parallel.

5. The method according to claim 1, characterized in that, Before parsing the logic diagram to obtain the parsed diagram, the method further includes: Obtain the configuration parameters of all processing operators in the logical diagram, as well as the connection structure between the data source fields and all processing operators; Based on the configuration parameters of all processing operators in the logical diagram, and the connection structure between the data source field and all processing operators, the first information of the prediction target field is obtained; Obtain the second information of the real target field based on each of the aforementioned data source fields; Based on the comparison results of the first information and the second information, the validity of the logic diagram is verified.

6. The method according to claim 5, characterized in that, The step of obtaining the configuration parameters of all processing operators in the logic graph includes: Traverse all processing operators contained in the logic graph; In response to the user's configuration instructions based on the display page, the configuration parameters of each of the processing operators are obtained; The verification of the validity of the logic diagram based on the comparison results of the first information and the second information includes: If the comparison result of the first information and the second information shows that the first information and the second information are consistent, it is determined that the configuration parameters of all processing operators in the logic diagram and the connection structure contained in the logic diagram are valid.

7. The method according to claim 1, characterized in that, The step of determining the parsing graph of each front-end data source field based on the real-time performance parameters corresponding to each of the aforementioned logical operation methods includes: The unidirectional connection logic diagram corresponding to the logic operation method with the shortest response time and / or the fastest running speed is determined as the parsing diagram corresponding to each of the aforementioned front-end data source fields.

8. The method according to claim 1, characterized in that, The process of calling the processing operators in the analytical graph to process each of the data source fields to obtain the target field corresponding to each of the data source fields includes: Based on the configuration parameters of the processing operators and the configuration parameters of the data source fields in the analytical graph, as well as the connection structure between the processing operators and the data source fields, the logical operation method corresponding to each analytical graph is determined. Using the aforementioned logical operation method, the processing operators within the analytical graph are invoked to process the data of each of the data source fields, thereby obtaining the target fields corresponding to each of the data source fields.

9. A data processing device, characterized in that, The data processing device includes a memory and a processor, the memory storing program data, and the processor executing the program data to implement the data processing method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, which, when executed by a processor, is used to implement the data processing method as described in any one of claims 1-8.

Images