Data processing system and data processing method

Abstract

The invention provides a data processing system and a data processing method. The data processing system includes a processor and a storage device. The storage device is used for storing the data processing module, the feature processing module and the classification conversion module. The processor is electrically connected with the storage device and is used for executing the data processing module, the feature processing module and the classification conversion module. The data processing module generates summarized data according to the multiple pieces of form data. The data processing module performs feature index calculation on the summarized data to generate feature index data. The data processing module preprocesses the feature index data to generate a sample tag data set. The feature processing module generates a training data set according to the sample tag data set. The classification conversion module constructs a classification model according to the training data set, and constructs a structured semantic knowledge base according to the classification model. According to the data processing system and the data processing method, the structured semantic knowledge base can be effectively constructed.

Description

Technical Field

[0001] The present invention relates to a digital system, in particular to a data processing system and a data processing method. Background Art

[0002] Generally speaking, when suppliers are fulfilling their contracts, corporate customers need to track materials in appropriate scenarios to ensure that materials can be delivered on time and in sufficient quantities. However, in complex production processes, it is no longer possible to accurately and efficiently confirm material tracking by human judgment. Although digital business systems are currently available to automate business processes, traditional digital business systems require complex logic design and consume a large amount of data storage space and computing resources to achieve the prediction function of material tracking during the production process. Summary of the invention

[0003] The present invention is directed to a data processing system and a data processing method, which can effectively and automatically construct a structured semantic knowledge base.

[0004] According to an embodiment of the present invention, the data processing system of the present invention includes a storage device and a processor. The storage device is used to store a data processing module, a feature processing module and a classification conversion module. The processor is electrically connected to the storage device and is used to execute the data processing module, the feature processing module and the classification conversion module. The data processing module generates summary data based on multiple form data. The data processing module calculates feature indicators on the summary data to generate feature indicator data, and the data processing module pre-processes the feature indicator data to generate a sample label data set. The feature processing module generates a training data set based on the sample label data set. The classification conversion module constructs a classification model based on the training data set, and constructs a structured semantic knowledge base based on the classification model.

[0005] According to an embodiment of the present invention, the data processing method of the present invention includes the following steps: executing a data processing module through a processor to generate summary data based on multiple form data; executing a data processing module through a processor to perform feature index calculation on the summary data to generate feature index data; executing a data processing module through a processor to pre-process the feature index data to generate a sample label data set; executing a feature processing module through a processor to generate a training data set based on the sample label data set; executing a classification conversion module through a processor to build a classification model based on the training data set; and executing a classification conversion module through a processor to build a structured semantic knowledge base based on the classification model.

[0006] Based on the above, the data processing system and data processing method of the present invention can perform data processing, feature processing and classification conversion on form data to build a structured semantic knowledge base, and the structured semantic knowledge base can be used by the system to track materials based on real-time business data generated by the workflow.

[0007] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a schematic diagram of a data processing system according to an embodiment of the present invention;

[0009] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;

[0010] Figure 3 is a flow chart of a prediction method according to an embodiment of the present invention;

[0011] Figure 4 is a schematic diagram of a structured rule according to an embodiment of the present invention.

[0012] Description of Reference Numerals

[0013] 100: Data processing system;

[0014] 110: processor;

[0015] 120: storage device;

[0016] 121: data processing module;

[0017] 122: feature processing module;

[0018] 123: classification conversion module;

[0019] 200: Business system;

[0020] 400: Structural rules;

[0021] S210, S220, S230, S240, S250, S260, S310, S320, S330: steps. DETAILED DESCRIPTION

[0022] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0023] Figure 1 FIG. 1 is a schematic diagram of a data processing system according to an embodiment of the present invention. Figure 1 , the data processing system 100 includes a processor 110 and a storage device 120. The processor 110 is electrically connected to the storage device 120. In this embodiment, the data processing system 100 can communicate with the business system 200. The processor 110 can obtain relevant business data from the business system 200. The storage device 120 can store the algorithms, programs and / or software related to the data processing module 121, the feature processing module 122 and the classification conversion module 123, and can store or temporarily store the data generated during the data processing process.

[0024] In the present embodiment, the processor 110 may be, for example, a system on a chip (SOC), or may include, for example, a central processing unit (CPU) or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), programmable logic device (PLD), other similar processing devices or a combination of these devices. In the present embodiment, the storage device 120 may be, for example, a dynamic random access memory (DRAM), a flash memory or a non-volatile random access memory (NVRAM).

[0025] In this embodiment, the processor 110 can obtain multiple historical form data from an external database, and can construct a structured semantic knowledge base based on the multiple historical form data, wherein the structured semantic knowledge base can be a knowledge graph data. In addition, during the business process of the business system 200, the processor 110 can automatically perform business forecasts based on the knowledge graph data, and automatically generate corresponding forecast information. During the business process, the data processing system 100 can generate (restore) corresponding condition data through the knowledge graph data, so as to be used for comparison with real-time business data and realize the corresponding forecast function. In this regard, the data processing system 100 does not need to store complex and large amounts of original judgment logic data, but can effectively save data storage space and computing resources. In this embodiment, the business system 200 can be an enterprise resource planning (ERP) system.

[0026] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 as well as Figure 2 , the data processing system 100 may execute the following steps S210 to S260. The data processing system 100 may read the historical database or the business system 200, for example, to obtain relevant historical order information and supplier information, but the present invention is not limited thereto. In step S210, the processor 110 may execute the data processing module 121 to generate summary data based on multiple form data. In the present embodiment, the data processing module 121 may generate summary data based on the association relationship between the multiple form data, wherein the multiple form data may include purchase order data, receipt data, and supplier data. In step S220, the processor 110 may execute the data processing module 121 to perform feature index calculation on the summary data to generate feature index data. In the present embodiment, the feature index calculation is an ETL (Extract, Transform, Load) engineering calculation. For example, the processor 110 may calculate relevant features such as delay duration, delivery time, the number of orders from the supplier in the past month, the number of rejections in the past month, the average delivery time in the past month, the number of orders in the past three months, the number of rejections in the past three months and / or the average delivery time in the past three months based on the summary data and relevant business logic to generate the characteristic indicator data.

[0027] In step S230, the processor 110 may execute the data processing module 121 to pre-process the feature indicator data to generate a sample label data set. In this embodiment, the data processing module 121 may remove outliers and fill missing values ​​on the feature indicator data. In addition, the data processing module 121 further fills the average value of the corresponding feature attribute value and performs labeling processing to generate the sample label data set. For example, if the quantity delivered by the supplier meets the quantity to be purchased, the label is set to 1, otherwise the label is set to 0.

[0028] In step S2410, the processor 110 may execute the feature processing module 122 to generate a training data set based on the sample label data set. In this embodiment, the feature processing module 122 may perform analysis of variance (ANOVA) and feature binarization on the sample label data set to generate the training data set. Specifically, the feature processing module 122 may select features with greater correlation with the category by means of variance analysis. The feature processing module 122 may sort all the values ​​of the feature from small to large, and calculate the average value between each two values ​​in turn as all candidate partition points. The feature processing module 122 may use the candidate partition point with the largest information gain of the data set as the final partition point of this feature, and use the feature values ​​less than or equal to this partition point as one category, and the feature values ​​greater than this partition point as another category. In other words, the feature processing module 122 may implement feature binarization and generate a training data set.

[0029] In step S250, the processor 110 may execute the classification conversion module 123 to construct a classification model according to the training data set. The classification conversion module includes a Bayesian classifier, and the Bayesian classifier constructs the classification model according to the training data set. In this embodiment, the classification conversion module 123 may include a Bayesian classifier (Naive Bayes classifier), and the Bayesian classifier can be used to construct a Bayesian classification model according to the training data set. For example, when the label is 0 or 1, the processor 110 can calculate the probability of each feature in each interval distribution. Assuming that the category variable of the data set D is set to c, the partition point of feature A is a, and the partition point of feature B is b, the processor 110 needs to calculate p(A<=a|c=0), p(A<=a|c=1), p(A>a|c=0), p(A>a|c=1), p(B<=b|c=0), p(B<=b|c=1), p(B>b|c=0), p(B>b|c=1). Moreover, when the probability is 0, the processor 110 can use the Laplace correction method for correction. Therefore, the processor 110 can obtain the size relationship between the four groups of p(c=0|A<=a,B<=b)p(c=0) and p(c=1|A<=a,B<=b)p(c=1), p(c=0|A<=a,B>b)p(c=0) and p(c=1|A<=a,B>b)p(c=1), p(c=0|A>a,B<=b)p(c=0) and p(c=1|A>a,B<=b)p(c=1), p(c=0|A>a,B>b)p(c=0) and p(c=1|A>a,B>b)p(c=1), and can obtain the category labels of the four cases according to the probability size.

[0030] In step S260, the processor 110 may execute the classification conversion module 123 to construct a structured semantic knowledge base according to the classification model. In the present embodiment, the structured semantic knowledge base may be a knowledge graph data architecture. Specifically. The classification conversion module 123 may read multiple classification rules from the classification model to generate multiple triple data. The classification conversion module 123 may merge the multiple triple data to construct the structured semantic knowledge base. For example, continuing the above example, assuming that the probability "p(c=0|A<=a,B<=b)" is greater than "p(c=1|A<=a,B<=b)", the category label judgment of this case can be 0, and the following form of triple (A&B, A<=a and B<=b, 0) data can be obtained. In addition, the following situations can be used to merge triple data. Assume that the data of triple 1 includes: (special&o_order&td_s_success, special<=0.383ando_order<=555.23and td_s_success<=202.942, 0). The data of triple 2 includes: (special&o_order&td_s_success, special<=0.383and o_orde<=555.23and td_s_success>202.942, 0). In this regard, the classification conversion module 123 can merge the above triple 1 data and triple 2 data to generate fused triple data: (special&o_order, special<=0.383and o_order<=555.23, 0). In this regard, the classification conversion module 123 can determine whether the factor data and category labels in the data of triple 1 and triple 2 are the same. If they are the same, the classification conversion module 123 can merge the data of triple 1 and triple 2. In other words, the method of generating knowledge graph data in this case can also effectively reduce the amount of redundant data. The multiple triple data respectively include factor data, condition data and conclusion data.

[0031] For another example, first, the data processing module 121 can associate the purchase order table with the receiving order table according to the table fields of the purchase order category, purchase order number and serial number to obtain the first summary data, and can further associate the first summary data with the manufacturer basic information table using the table field of the manufacturer code to obtain the second summary data. The data processing module 121 can calculate each data in the second summary data to obtain the corresponding delivery time (order_delivery, acceptance date to purchase date), whether the delivery is on time (on_time, whether the delivery confirmation date is greater than the purchase date (if so, the value of the characteristic indicator is 1, if not, the value of the characteristic indicator is 0)), delay time (delay_time, purchase date to delivery confirmation date) and other characteristic indicators, and obtain the characteristic indicator data as shown in Table 1 below.

[0032] Delivery time Whether the delivery is on time Delay duration 1 3 5 1 56 19 1 96 1 -20 -74 1 12765421

[0033] Table 1

[0034] Next, the data processing module 121 may remove abnormal data in the feature indicator data and fill in missing values. An example of abnormal data is the delay time "12765421" in the above Table 1. In this regard, if the delivery time is negative and / or the delay time is too long, it is abnormal data. The data processing module 121 may calculate the average value to fill in the missing values. The data processing module 121 may compare the purchase quantity and delivery quantity fields in the second summary data. If the delivery quantity is greater than or equal to the purchase quantity, the label is set to 1, otherwise the label is set to 0. Therefore, the data processing module 121 may generate sample label data as shown in Table 2 below.

[0035] Delivery time Whether the delivery is on time Delay duration Label 40 1 3 0 5 1 56 1 19 1 13 0 96 1 -20 1

[0036] Table 2

[0037] Next, the feature processing module 122 uses variance analysis to select more relevant features for the three features in the sample label data of Table 2, such as delivery time, whether the delivery is on time, and delay time. The feature processing module 122 can first perform maximum and minimum normalization on the values ​​under each feature, calculate the variance corresponding to each feature, set the threshold t to 0.001, and select the features with variance greater than t as delivery time and delay time, so as to obtain the data in Table 3 below.

[0038] Delivery time Delay duration Label 40 3 0 5 56 1 19 13 0 96 -20 1

[0039] Table 3

[0040] The feature processing module 122 can discretize the continuous numerical features (i.e., delivery time and delay time) in Table 3 above. Taking the feature "delivery time" as an example, the feature processing module 122 can sort the feature values ​​in ascending order to obtain 5, 19, 40, and 96. In addition, the feature processing module 122 can obtain three candidate partitioning points 12, 29.5, and 68, and can calculate that the information gain obtained by partitioning at candidate point 12 is the largest. Similarly, the feature "delay time" can obtain the largest information gain by partitioning at 8.5.

[0041] Therefore, the feature processing module 122 may generate data (training data set) as shown in Table 4 below.

[0042] Delivery time Delay duration Label >12 <＝8.5 0 <＝12 >8.5 1 >12 >8.5 0 >12 >8.5 1

[0043] Table 4

[0044] Next, according to the above Table 4, the classification conversion module 123 can calculate the probability p(label=0)=0.5 and the probability p(label=1)=0.5. And, assuming that the features are independent, the processor 110 can calculate the probability "p(order_delivery<=12|label=0)=0.0", the probability "p(order_delivery<=12|label=1)=0.5", the probability "p(order_delivery>12|label=0)=1.0", the probability "p(order_delivery>12|label=1)=0.5", the probability "p(delay_time<=8.5|label=0)=0.5", the probability "p(delay_time<=8.5|label=1)=0.0", the probability "p(delay_time>8.5|label=0)=0.5", the probability "p(delay_time>8.5|label=1)=1.0". The classification conversion module 123 uses Laplace correction for the data with probability 0, and also corrects the probability p(label=0) and the probability p(label=1). Therefore, the classification conversion module 123 can obtain the probability "p(order_delivery<=12|label=0)=1 / (2+2)=0.25" and the probability "p(delay_time<=8.5|label=1)=1 / (2+2)=0.25". In this way, the classification conversion module 123 can calculate the probability “p(lable=0|order_delivery<=12, delay_time<=8.5)<=>p(label=0)*p(order_delivery<=12|label=0)*p(delay_time<=8.5|label=0)=0.5*0.25*0.5=0.0625” and the probability “p(lable=1|order_delivery<=12, delay_time<=8.5)<=>p(label=1)*p(order_delivery<=12|label=1)*p(delay_time<=8.5|label=1)=0.5*0.5*1.0=0.25”.Finally, according to Bayes' theorem, since "p(lable=1|order_delivery<=12,delay_time<=8.5)>p(lable=0|order_delivery<=12,delay_time<=8.5)", it can be concluded that the classification rule is that when the sample satisfies "order_delivery<=12,delay_time<=8.5", the category label can be determined as 1. Similarly, it can be concluded that when the sample satisfies "order_delivery<=12,delay_time>8.5", the category label can be determined as 1. When the sample satisfies "order_delivery>12,delay_time<=8.5", the category label can be determined as 0. When the sample satisfies "order_delivery>12,delay_time>8.5", the category label can be determined as 1.

[0045] Next, the classification conversion module 123 can convert the above four rules into triple data (i.e., data composed of factor data, condition data and conclusion data), which may be, for example, "(order_delivery&delay_time, order_delivery<=12and delay_time<=8.5, no need to chase materials)", "(order_delivery&delay_time, order_delivery<=12and delay_time>8.5, no need to chase materials)", "(order_delivery&delay_time, order_delivery>12and delay_time<=8.5, purchase quantity is not met, and chase materials are needed)" and "(order_delivery&delay_time, order_delivery>12and delay_time>8.5, no need to chase materials)".

[0046] In addition, for example, triple data may be merged if the following situations occur. For example, the triple data includes a triple data "(order_delivery&delay_time, order_delivery<=12and delay_time<=8.5, no need to chase materials)" and another triple data "(order_delivery&delay_time, order_delivery<=12and delay_time>8.5, no need to chase materials)". In this regard, the classification conversion module 123 may merge the above two triple data into "(order_delivery, order_delivery<=12, no need to chase materials)". Finally, the classification conversion module 123 may store the triple data "(order_delivery, order_delivery<=12, no need to chase materials)" into the knowledge graph data, wherein the factor data in this triple data is "order_delivery", the condition data is "order_delivery<=12", and the conclusion data is "no need to chase materials".

[0047] Figure 3 is a flowchart of a prediction method according to an embodiment of the present invention. Figure 4 FIG. 1 is a schematic diagram of a structured rule according to an embodiment of the present invention. Figure 1 , Figure 3 as well as Figure 4 , when the knowledge graph data is constructed, the data processing system 100 can perform the following prediction operations S310 to S330. In step S310, the processor 110 can obtain the parameters of the workflow rules from the structured semantic knowledge base. In step S320, the processor 110 can obtain real-time business field data from the business system 200. In step S330, the processor 110 can compare the real-time data with the parameters to obtain corresponding prediction information.

[0048] For example, during the process of executing a task in the business system 200, the processor 110 may convert the pre-stored knowledge graph data as exemplified in the above embodiment into Figure 4The structured rule 400 of the condition “condition” shown in the figure may include the parameter name ““paramater”: “order_delivery””, the logical judgment symbol ““LogicalJudgmentSymbol”: “LessAndEqueThan””, the logical value ““LogicalValue”: 12” and the result when the condition is true ““consequence”: “No need to chase materials””. In other words, the condition corresponding to the structured rule 400 is “order_delivery<=12”.

[0049] In this regard, assuming that the data of the real-time business field "order_delivery" obtained by the processor 110 from the business system 200 is 11, the processor 110 can determine that the data of the real-time business field "order_delivery" meets the above conditions. Therefore, the processor 110 can obtain the conclusion data of "no need to chase materials", and the processor 110 can reply its conclusion to the business system 200, so that the business system 200 can perform subsequent tasks and / or notify the user accordingly.

[0050] To sum up, the data processing system and data processing method of the present invention can perform data processing on form data to automatically construct knowledge graph data, and in the process of the business system executing tasks, corresponding conditional data can be generated through the knowledge graph data to determine whether the real-time business data generated by the business system in the process of executing tasks meets the conditions, thereby realizing the corresponding prediction function.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing system, characterized in that: include: A storage device, used to store a data processing module, a feature processing module and a classification conversion module; as well as a processor, electrically connected to the storage device and used to execute the data processing module, the feature processing module and the classification conversion module, The data processing module generates summary data according to the plurality of form data, the data processing module performs feature index calculation on the summary data to generate feature index data, and the data processing module performs preprocessing on the feature index data to generate a sample label data set. The feature processing module generates a training data set according to the sample label data set. The classification conversion module constructs a classification model according to the training data set, and constructs a structured semantic knowledge base according to the classification model.

2. The data processing system according to claim 1, characterized in that The processor obtains parameters of workflow rules from a structured semantic knowledge base, obtains real-time business field data from the business system, and compares the real-time data with the parameters to obtain corresponding prediction information.

3. The data processing system according to claim 2, characterized in that: The plurality of form data include purchase order data, receipt data and supplier data, and the forecast information is material tracking information.

4. The data processing system according to claim 1, characterized in that: The data processing module generates the summary data according to the association relationship between the multiple form data, wherein the characteristic index calculation is an ETL (Extract, Transform, Load) engineering calculation.

5. The data processing system according to claim 1, characterized in that: The data processing module performs outlier elimination and missing value filling processing on the feature index data, wherein the data processing module fills the average value of the corresponding feature attribute value and performs labeling processing to generate the sample label data set.

6. The data processing system according to claim 1, characterized in that: The feature processing module performs variance analysis and feature binarization processing on the sample label data set to generate the training data set.

7. The data processing system according to claim 1, characterized in that: The classification conversion module includes a Bayesian classifier, and the Bayesian classifier constructs the classification model according to the training data set.

8. The data processing system according to claim 1, characterized in that: The classification conversion module reads a plurality of classification rules from the classification model to generate a plurality of triple data, wherein the classification conversion module merges the plurality of triple data to construct the structured semantic knowledge base.

9. The data processing system according to claim 8, characterized in that: The structured semantic knowledge base is a knowledge graph data architecture.

10. The data processing system according to claim 8, characterized in that The plurality of triple data respectively include factor data, condition data and conclusion data.

11. A data processing method, characterized in that: include: Executing a data processing module through a processor to generate summary data according to a plurality of form data; Executing the data processing module through the processor to perform feature index calculation on the summary data to generate feature index data; Executing the data processing module through the processor to pre-process the feature indicator data to generate a sample label data set; Executing a feature processing module by the processor to generate a training data set according to the sample label data set; Executing a classification conversion module by the processor to construct a classification model according to the training data set; as well as The classification conversion module is executed by the processor to construct a structured semantic knowledge base according to the classification model.

12. The data processing method according to claim 11, characterized in that: Also includes: Obtaining parameters of workflow rules from the structured semantic knowledge base through the processor; Acquiring real-time business field data from the business system through the processor; as well as The processor compares the real-time data with the parameters to obtain corresponding prediction information.

13. The data processing method according to claim 12, characterized in that: The plurality of form data include purchase order data, receipt data and supplier data, and the forecast information is material tracking information.

14. The data processing method according to claim 11, characterized in that: The steps of generating the summary data include: The summary data is generated according to the association relationship between the multiple form data, wherein the characteristic index calculation is an ETL (Extract, Transform, Load) engineering calculation.

15. The data processing method according to claim 11, characterized in that: The steps of generating the sample label data set include: The feature index data is processed by removing outliers and filling missing values, and the average value of the corresponding feature attribute value is filled, and labeling is performed to generate the sample label data set.

16. The data processing method according to claim 11, characterized in that: The steps of generating the training data set include: The sample label data set is subjected to variance analysis and feature binarization processing to generate the training data set.

17. The data processing method according to claim 11, characterized in that: The classification conversion module includes a Bayesian classifier, and the Bayesian classifier constructs the classification model according to the training data set.

18. The data processing method according to claim 11, characterized in that: The steps of constructing the structured semantic knowledge base include: The classification conversion module reads a plurality of classification rules from the classification model to generate a plurality of triple data; and The plurality of triple data are merged to construct the structured semantic knowledge base.

19. The data processing method according to claim 18, characterized in that: The structured semantic knowledge base is a knowledge graph data architecture.

20. The data processing method according to claim 18, characterized in that: The plurality of triple data respectively include factor data, condition data and conclusion data.

Images