Semantic knowledge base construction system, method and apparatus

Abstract

The application belongs to the field of semantic computing and knowledge base construction, and particularly relates to a semantic knowledge base construction system, method and device, aiming to solve the problem that the existing semantic knowledge base construction system based on a data lake cannot describe rules with more semantic connotations, and the existing rule engine does not allow users to set independently, resulting in that the constructed semantic knowledge base cannot be applied across platforms. The system comprises a data acquisition module, a first rule generation module, a second rule generation module, a cost calculation module, a rule optimization module, a rule subnetwork generation and pre-allocation module, a rule subnetwork division module, a semantic feature data extraction module and a semantic knowledge base construction module. The application extracts rules with more connotations, models the semantic rules of users through a structured rule description language, and further realizes the cross-platform application of the semantic knowledge base.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of semantic computing and knowledge base construction, and particularly relates to a semantic knowledge base construction system, method and device. BACKGROUND

[0002] With the development of public culture and data lake technology, the application of semantic knowledge base based on data lake will become more and more important in the future. At present, hundreds of millions of public cultural participants will form a huge semantic rule network, and the automatic generation of the semantic rule network and the division of the rule network are a major scientific challenge.

[0003] At the same time, all application data will gradually be migrated to a special storage environment such as a data lake. How to build a cross-platform semantic knowledge base in such an environment will face a great difficulty and challenge. In order to solve these difficulties, the application designs a semantic knowledge base construction system. SUMMARY

[0004] In order to solve the above problems in the prior art, that is, to solve the problem that the existing semantic knowledge base construction system based on data lake cannot describe rules with more semantic connotations, and the existing rule engine does not allow users to set it independently, resulting in that the constructed semantic knowledge base cannot be applied cross-platform, the application provides a semantic knowledge base construction system, which comprises a data acquisition module, a first rule generation module, a second rule generation module, a cost calculation module, a rule optimization module, a rule subnetwork generation and pre-allocation module, a rule subnetwork division module, a semantic feature data extraction module and a semantic knowledge base construction module.

[0005] The data acquisition module is configured to extract structured data, semi-structured data and unstructured data of a semantic knowledge base to be constructed from a data lake as input data.

[0006] The first rule generation module is configured to construct semantic rules based on acquired domain expert prior knowledge as first semantic rules.

[0007] The second rule generation module is configured to preprocess the input data by using a natural language processing method for semantic extraction, semantic processing and knowledge mining, to take the preprocessed semantic information as a node, and to construct a rule node according to a preset semantic structure as second semantic rules.

[0008] The cost calculation module is configured to calculate the time cost of each rule node when executed.

[0009] The rule optimization module is configured to combine the time cost, and to merge and replace the rule nodes by using a preset dynamic optimization mechanism based on rule merging and node replacement, to obtain an optimized rule network.

[0010] The rule subnetwork generation and pre-allocation module is configured to divide the optimized rule network into independent rule subnetworks that have no communication relationship with each other, calculate the calculation cost of each independent rule subnetwork, and establish a pre-allocation relationship between each independent rule subnetwork and a preset processor based on an equal distribution principle.

[0011] The rule subnetwork division module is configured to divide the independent rule subnetworks that have a pre-allocation relationship with the plurality of processors into a plurality of segmented networks, and establish an allocation relationship between each segmented network and a corresponding processor.

[0012] The semantic feature data extraction module is configured to extract features from the structured data, semi-structured data and unstructured data extracted from the data lake, remove redundant data, and store the extracted feature data in a database to form structured data.

[0013] The semantic knowledge base construction module is configured to construct a semantic knowledge base based on the first semantic rule and the second semantic rule, in combination with the structured data and the allocation relationship between each segmented network and a corresponding processor.

[0014] In some preferred embodiments, the rule nodes are divided into non-computational rule nodes and computational rule nodes.

[0015] The computational rule nodes include rule selection nodes, rule joint nodes, rule intersection nodes, rule negation calculation nodes, rule connection nodes and rule Cartesian product nodes.

[0016] In some preferred embodiments, the dynamic optimization mechanism based on rule merging and node replacement is as follows:

[0017] Replace a rule node with a low time cost with a rule node with a high time cost.

[0018] If the selection set of a rule node is a subset of the selection set of another rule node or there is a selection condition overlap between the rule nodes, then the rule nodes are merged.

[0019] In some preferred embodiments, the independent rule subnetworks are obtained using an undirected graph traversal method.

[0020] In some preferred embodiments, the calculation cost of each independent rule subnetwork is calculated as follows:

[0021] Calculate the flow of all directed edges of the independent rule subnetwork.

[0022] Calculate the calculation cost of each rule node based on the flow of the directed edges, and obtain the calculation cost of the corresponding independent rule subnetwork.

[0023] In some preferred embodiments, the pre-allocation relationship between each independent rule subnet and the preset processor is established based on the average calculation cost, and the method is as follows:

[0024] Based on the average calculation cost, the preset up-floating ratio value and the preset down-regulation ratio value, the calculation cost interval of a single processor is obtained.

[0025] The single independent rule subnet whose calculation cost falls within the calculation cost interval is allocated with one processor respectively.

[0026] The independent rule subnets whose calculation cost is less than the lower limit of the calculation cost interval are combined, and the combined multiple independent rule subnets whose calculation cost sum falls within the calculation cost interval are allocated with one processor.

[0027] The independent rule subnets whose calculation cost is greater than the upper limit of the calculation cost interval are allocated with multiple processors.

[0028] In some preferred embodiments, the pre-allocation relationship between each independent rule subnet and the preset processor is established based on the average calculation cost, and the method is as follows:

[0029] The independent rule subnet whose calculation cost is greater than the upper limit of the calculation cost interval is allocated with K1 processors if its calculation cost falls within the K1 times interval range of the calculation cost interval; otherwise, the multiple independent rule subnets whose calculation cost is greater than the upper limit of the calculation cost interval are combined, and the combined multiple independent rule subnets are allocated with K2 processors if their calculation cost sum falls within the K2 times interval range of the calculation cost interval.

[0030] In the second aspect of the present application, a semantic knowledge base construction method is provided, and the method comprises the following steps:

[0031] Extracting structured data, semi-structured data and unstructured data to be constructed as input data from a data lake;

[0032] Based on the obtained domain expert prior knowledge, a semantic rule is constructed as a first semantic rule;

[0033] The input data is preprocessed by using a natural language processing method for semantic extraction, semantic processing and knowledge mining, the preprocessed semantic information is taken as a node, and a rule node is constructed according to a preset semantic structure as a second semantic rule;

[0034] The time cost of each rule node when executed is calculated;

[0035] The rule nodes are combined and replaced by a preset dynamic optimization mechanism based on rule combination and node replacement in combination with the time cost, so as to obtain an optimized rule net;

[0036] The optimized rule net is divided into independent rule subnets without communication relationship between each other; the calculation cost of each independent rule subnet is calculated, and the pre-allocation relationship between each independent rule subnet and a preset processor is established based on the balanced allocation principle;

[0037] The independent rule subnets having the pre-allocation relationship with the plurality of processors are subjected to network division, a plurality of division networks are obtained, and the allocation relationship between each division network and the corresponding processor is established;

[0038] The structured data, semi-structured data and unstructured data extracted from the data lake are subjected to feature extraction, redundant data is removed, and the extracted feature data is stored in a database to form structured data;

[0039] Based on the first semantic rule and the second semantic rule, the structured data and the allocation relationship between each division network and the corresponding processor are combined to construct a semantic knowledge base.

[0040] In a third aspect, the application provides a device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the processor, and the instructions are used to implement the semantic knowledge base construction method described above.

[0041] In a fourth aspect, the application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to implement the semantic knowledge base construction method described above.

[0042] The application has the following advantages:

[0043] The application extracts more connotative rules by combining the experience of field experts and artificial intelligence algorithms, and models the semantic rules of users through a structured rule description language, thereby realizing cross-platform application of the semantic knowledge base.

[0044] 1) The application supports different structured data, semi-structured data and unstructured data to use the semantic knowledge base system completely transparently and seamlessly through the data lake application platform, greatly promotes the cross-platform application of the semantic knowledge base technology, and meets and drives the vigorous development of the public cultural semantic knowledge base industry under the storage cornerstone of the future data lake.

[0045] 2) This invention utilizes a rule network generation algorithm based on a computational cost model to calculate the time cost of rule nodes. Combining this time cost, a dynamic optimization mechanism based on rule merging and node replacement is employed to optimize the merging and replacement of rule nodes, extracting more meaningful rules. Furthermore, by incorporating parallel processing techniques based on rule node matching and balanced subgraph partitioning, the computational cost of each rule network is effectively distributed evenly across processors, achieving parallel processing of semantic rules. This keeps the workload of each processor within a relatively balanced and reasonable range, improving the efficiency of system resource utilization and enhancing the real-time performance of information processing in rule-based decision-making systems. Attached Figure Description

[0046] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0047] Figure 1 This is a schematic diagram of the framework of a semantic knowledge base construction system according to an embodiment of the present invention;

[0048] Figure 2 This is a detailed framework diagram of a semantic knowledge base construction system according to an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present invention. Detailed Implementation

[0050] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] This invention provides a semantic knowledge base construction system, such as Figure 1 As shown, it includes a data acquisition module 100, a first rule generation module 200, a second rule generation module 300, a cost calculation module 400, a rule optimization module 500, a rule subnet generation and pre-allocation module 600, a rule subnet partitioning module 700, a semantic feature data extraction module 800, and a semantic knowledge base construction module 900.

[0053] The data acquisition module 100 is configured to extract structured data, semi-structured data and unstructured data to be used to construct a semantic knowledge base from a data lake as input data.

[0054] The first rule generation module 200 is configured to construct semantic rules based on the acquired prior knowledge of the domain experts as first semantic rules.

[0055] The second rule generation module 300 is configured to preprocess the input data by using a natural language processing method for semantic extraction, semantic processing and knowledge mining, to use the preprocessed semantic information as nodes, and to construct rule nodes according to a preset semantic structure as second semantic rules.

[0056] The cost calculation module 400 is configured to calculate the time cost of each rule node when executed.

[0057] The rule optimization module 500 is configured to combine the time cost, and to merge and replace the rule nodes by using a preset dynamic optimization mechanism based on rule merging and node replacement, to obtain an optimized rule network.

[0058] The rule subnetwork generation and pre-allocation module 600 is configured to divide the optimized rule network into independent rule subnetworks that have no connection relationship with each other, to calculate the calculation cost of each independent rule subnetwork, and to establish a pre-allocation relationship between each independent rule subnetwork and a preset processing machine based on an equal distribution principle.

[0059] The rule subnetwork division module 700 is configured to divide the independent rule subnetworks that have a pre-allocation relationship with the plurality of processing machines into a plurality of divided networks, and to establish an allocation relationship between each divided network and a corresponding processor.

[0060] The semantic feature data extraction module 800 is configured to extract features from the structured data, semi-structured data and unstructured data extracted from the data lake, remove redundant data, and store the extracted feature data in a database to form structured data.

[0061] The semantic knowledge base construction module 900 is configured to construct a semantic knowledge base based on the first semantic rules, the second semantic rules, the structured data, and the allocation relationship between each divided network and the corresponding processor.

[0062] In order to more clearly illustrate the present application, the following will be combined with the accompanying drawings to be described in detail. Figure 1 、 2 The embodiments of the present application will be described in detail.

[0063] The data acquisition module 100 is configured to extract structured data, semi-structured data and unstructured data to be used to construct a semantic knowledge base from a data lake as input data;

[0064] The data lake refers to a data aggregate, including structured data, semi-structured data and unstructured data, and mainly contains semantic knowledge base application objects.

[0065] In the embodiment, structured data, semi-structured data and unstructured data to be constructed in the semantic knowledge base are extracted from the data lake as input data.

[0066] The first rule generation module 200 is configured to construct semantic rules based on the acquired domain expert prior knowledge as first semantic rules.

[0067] In the embodiment, the first semantic rules (referred to as simple rules in the application) are constructed based on the experience of the domain expert, and a rule base (referred to as a simple rule base) is formed.

[0068] The second rule generation module 300 is configured to pre-process the input data by using a natural language processing method for semantic extraction, semantic processing and knowledge mining, take the pre-processed semantic information as a node, and construct a rule node according to a preset semantic structure as second semantic rules.

[0069] In the embodiment, the input data is subjected to semantic extraction, which includes entity extraction, relationship extraction, location extraction, time extraction and the like. The multi-granularity semantic information of the input data is extracted to realize semantic mapping representation of the data.

[0070] After the semantic extraction, semantic processing and knowledge mining are performed, that is, the information after the semantic extraction is pre-processed by using synonym / paronym discrimination, semantic disambiguation, semantic fusion, part-of-speech analysis, semantic mining and the like, the pre-processed semantic information is taken as a node, and a rule node is constructed according to a preset semantic structure (preferably, a subject-predicate-object and active complement semantic structure form is adopted in the application) as second semantic rules (referred to as complex rules in the application), thereby forming a semantic rule net.

[0071] In addition, in the application, the rule node includes a non-computing rule node and a computing rule node.

[0072] The non-computing rule node includes a rule relationship node and a rule action node.

[0073] The computing rule node includes a rule selection node, a rule joint node, a rule intersection node, a rule negation calculation node, a rule connection node and a rule Cartesian product node.

[0074] The cost calculation module 400 is configured to calculate time cost when each rule node is executed.

[0075] In this embodiment, the rule node execution time cost is calculated by the rule net generation algorithm based on the computational cost model, as follows:

[0076] The rule node computational cost model is constructed, i.e. the time cost of the processing machine required for the computational rule node in the rule net when performing rule computation.

[0077] 1) Rule selection node computational cost: the time required for the processing machine to perform rule selection operation on the records in the relation table. Its time cost mainly includes the time required for traversing the relation table (schema table) in the database and performing selection judgment. Cost (rule selection node) = Cost (traverse) + Cost (comparison and selection judgment).

[0078] 2) Rule join node computational cost: the time required for the processing machine to perform rule join operation on the records in the relation table. Its time cost mainly includes the time required for traversing the relation table (schema table) in the database and performing comparison judgment. Cost (rule join node) = Cost (traverse) + Cost (comparison and selection judgment).

[0079] 3) Rule intersection node computational cost: the time required for the processing machine to perform rule intersection operation on the records in the relation table. Its time cost mainly includes the time required for traversing the relation table (schema table) in the database and performing comparison judgment. Cost (rule intersection node) = Cost (traverse) + Cost (comparison and selection judgment).

[0080] 4) Rule connection node computational cost: the time required for the processing machine to perform rule connection operation on the records in the relation table. Its time cost mainly includes the time required for traversing the relation table (schema table) in the database and performing judgment and connection operation. Cost (rule connection node) = Cost (traverse) + Cost (comparison and selection judgment) + Cost (connection operation).

[0081] 5) Rule negation node computational cost: the time required for the processing machine to perform rule negation operation on the records in the relation table. Its time cost mainly includes the time required for traversing the relation table in the database and performing comparison judgment. Cost (rule negation node) = Cost (traverse) + Cost (comparison and selection judgment).

[0082] 6) Cost of regular Cartesian product node computation: This refers to the time required for the processor to perform a regular Cartesian product operation on the records in the relational table. Its time consumption mainly consists of the time spent traversing the relational table in the database and performing the Cartesian product operation. Cost (regular Cartesian product node) = Cost (traversal) + Cost (Cartesian product operation).

[0083] The rule optimization module 500 is configured to combine time costs and merge and replace rule nodes through a preset dynamic optimization mechanism based on rule merging and node replacement to obtain an optimized rule network.

[0084] In this embodiment, a preset dynamic optimization mechanism based on rule merging and node replacement is used to merge and replace rule nodes to obtain an optimized rule network.

[0085] The dynamic optimization mechanism based on rule merging and node replacement is as follows:

[0086] 1) Optimization mechanism based on rule merging. If the selection set of one rule node is a subset of the selection set of another rule node, or if there is overlap in the selection conditions between two rule nodes, then merging is performed;

[0087] 2) Optimization mechanism based on rule node replacement. Rule nodes with high computational costs are replaced with low-computational-cost rule nodes without affecting the rule calculation results.

[0088] Based on the optimized rule network, a second semantic rule base (referred to as a complex rule base in this invention) is constructed. Figure 2 The artificial intelligence algorithms required for the complex rules are the methods in the second rule generation module 300 and the rule optimization module 500.

[0089] The rule subnet generation and pre-allocation module 600 is configured to divide the optimized rule network into independent rule subnets that are not connected to each other; calculate the computational cost of each independent rule subnet; and establish a pre-allocation relationship between each independent rule subnet and a preset processor based on the principle of balanced allocation.

[0090] If in a semantic rule network there are some blocks that are not connected to other blocks by any directed edges, then there is no need for communication between these blocks. These blocks can be called independent rule subnets. There is no data dependency between these rule subnets. Independent rule subnets do not need to call the original data or initial calculation results of other rule subnets during the calculation process.

[0091] In the embodiment, the method of undirected graph traversal is adopted to obtain the independent rule sub-network, the optimized rule network is regarded as a graph, and the problem of obtaining the independent rule sub-network is converted into the problem of undirected graph traversal. After the undirected graph traversal, all nodes are traversed, and N connected components are formed. Each connected component is an independent rule sub-network.

[0092] After the independent rule sub-network is divided, the calculation cost of each independent rule sub-network is calculated, and a pre-distribution relationship between each independent rule sub-network and a preset processor is established based on the balanced distribution principle.

[0093] The calculation cost method of each independent rule sub-network is as follows: the directed edge flow of the independent rule sub-network is calculated; the calculation cost of each rule node is calculated based on the directed edge flow, and the calculation cost of the corresponding independent rule sub-network is obtained. Each independent rule sub-network is composed of many rule nodes and directed edges with flow. Based on the directed edge flow, the calculation cost of each calculable rule node can be obtained, and the calculation cost of the independent rule sub-network is obtained by summation.

[0094] The calculation method of the directed edge flow of the independent rule sub-network is as follows:

[0095] 1) Find all rule relationship nodes V0, V1, …, Vm (assuming that there are m relationship nodes in the independent rule sub-network) in the independent rule sub-network;

[0096] 2) Calculate the record number of each rule relationship node, and assign it to the out-degree flow of the edge with the corresponding rule relationship node as the starting point.

[0097] 3) According to the rule traversal algorithm, the out-degree flow of each rule node in the rule sub-network is calculated by traversing each semantic node. (The in-degree flow of the rule node is the out-degree flow of the previous node, so only the out-degree flow needs to be calculated here);

[0098] 4) Repeat step 3) until the corresponding flow of the out-degree edge of all calculable nodes is calculated and assigned with a corresponding value.

[0099] The calculation cost of the independent rule sub-network is calculated as follows:

[0100] 1) The initial value of the calculation cost of the independent rule sub-network Rule_Sub_Network (p) = 0;

[0101] 2) According to the rule traversal algorithm, each semantic node in the independent rule sub-network is traversed, the calculation cost Cost[Pi] of each semantic computable node is calculated through the all-directed edge flow calculation algorithm of the independent rule sub-network, and the value is assigned to Rule_Sub_Network(i) for evaluation, that is, Rule_Sub_Network(i) = Cost[Pi];

[0102] 3) Step 2) is repeated until the cost of each rule node in the independent rule sub-network is calculated, and finally the total cost of the independent rule sub-network is obtained.

[0103] Based on the calculation cost of each independent rule sub-network, the pre-distribution relationship between each independent rule sub-network and the preset processor is established according to the balanced distribution principle.

[0104] Based on the calculation cost of the independent rule sub-network, the total calculation cost of the preset rule network can be obtained after summation, and further based on the number of processors, the average calculation cost of a single processor can be obtained. However, the average calculation cost is a specific numerical value, although the processing cost of each processor is consistent with the numerical value, which is optimal, but this ideal condition cannot be achieved, therefore, based on the average calculation cost, and according to the preset floating ratio value and the preset down-regulation ratio value, the calculation cost interval of a single processor is obtained, which is used as the basis for balanced distribution. The preset floating ratio value and the preset down-regulation ratio value can be set as needed;

[0105] Based on the calculation cost interval, when pre-distributing each independent rule sub-network, the method for the independent rule sub-network less than or equal to the upper limit of the calculation cost interval is as follows: a single independent rule sub-network whose calculation cost falls within the calculation cost interval is respectively distributed to a processor; the independent rule sub-networks less than the lower limit of the calculation cost interval are combined, and the sum of the calculation costs of the combined multiple independent rule sub-networks falls within the calculation cost interval, and the corresponding combined multiple independent rule sub-networks are distributed to a processor. The method for the independent rule sub-network greater than the upper limit of the calculation cost interval is as follows: if the calculation cost of the independent rule sub-network greater than the upper limit of the calculation cost interval falls within the K1 times interval range of the calculation cost interval, the independent rule sub-network is distributed to K1 processors; otherwise, multiple independent rule sub-networks greater than the upper limit of the calculation cost interval are combined, and if the sum of the calculation costs falls within the K2 times interval range of the calculation cost interval, the combined independent rule sub-networks are distributed to K2 processors.

[0106] So far, the pre-allocation relationship between each independent rule subnet and the preset processor is established, and the pre-allocation relationship includes one or more of the following three allocation corresponding relationships: an independent rule subnet independently occupies one processor; a plurality of independent rule subnets combine to occupy one processor; and one independent rule subnet occupies a plurality of processors.

[0107] The rule subnet division module 700 is configured to divide the independent rule subnets having the pre-allocation relationship with the plurality of processors into a plurality of division networks, and establish an allocation relationship between each division network and a corresponding processor.

[0108] In the embodiment, the total standard for dividing the rule network is mainly to make the calculation cost of the divided network meet the pre-subscription design requirement as much as possible, and to make the dependence cost and the communication cost between the divided parts as small as possible, which can be integrated into the following three standards: (1) the processing load of the processor is balanced as much as possible; (2) the waiting (dependence) consumption of the processor is as small as possible; and (3) the communication cost of the processor is as small as possible.

[0109] The independent rule subnet to be divided in the application is an independent rule subnet occupying a plurality of processors in the allocation relationship, which includes the case of one independent rule subnet occupying a plurality of processors, and also includes the case of a plurality of independent rule subnets greater than the upper limit of the calculation cost interval combining to occupy a plurality of processors. When the network is divided, each allocation combination without external communication of the processor is processed separately.

[0110] In the application, the independent rule subnet is divided into a plurality of division networks, which can be one of the following three methods: a rule subnet balanced division algorithm; a rule subnet balanced minimum dependence division algorithm; and a rule subnet balanced, minimum dependence and minimum communication cost division algorithm.

[0111] The rule subnet balanced division algorithm mainly considers the word "balance", and the main purpose is to take whether the processing cost of the processor is balanced as the evaluation standard without considering other factors. The core idea of the algorithm can be summarized as the following three points: considering the balanced distribution of the processing cost of the processor, not considering the waiting (dependence) consumption of the processor, and not considering the communication consumption of the processor. The method adopted is: based on the calculation cost of the rule node of the independent rule subnet to be divided, the rule node is evenly divided into the corresponding pre-allocated processor.

[0112] The rule subnet balanced minimum dependence division algorithm not only considers the word "balance", but also considers the waiting time wasted by the processor due to waiting for the calculation result of the processing task of another processor. The core idea of the algorithm can be summarized as the following three points: considering the balanced distribution of the processing cost of the processor, considering the waiting (dependence) consumption of the processor, and not considering the communication consumption of the processor.

[0113] Rule sub-network balance, less dependent cost and less communication cost segmentation algorithm: This algorithm not only considers the "balance" word, but also considers the waiting cost between processors due to waiting for intermediate calculation results and the communication cost between processors due to the "migration" of calculation tasks as evaluation criteria. The core idea of this algorithm can be summarized as the following three algorithm principles: considering the balance distribution of processor processing cost, considering the waiting (dependence) consumption of processor, and considering the communication consumption of processor on the basis of considering the waiting consumption of processor.

[0114] The preferred embodiment of the present application is the rule sub-network balance, minimum dependence cost and minimum communication cost segmentation algorithm. The rule sub-network balance segmentation algorithm, the rule sub-network balance minimum dependence segmentation algorithm, and the rule sub-network balance, minimum dependence cost and minimum communication cost segmentation algorithm have been described in detail in the prior art literature, and will not be expanded here.

[0115] The semantic feature data extraction module 800 is configured to extract features from the structured data, semi-structured data, and unstructured data extracted from the data lake, remove redundant data, and store the extracted feature data in a database to form structured data.

[0116] In this embodiment, the structured data, semi-structured data, and unstructured data extracted from the data lake are extracted by artificial intelligence algorithms such as CNN and DNN, redundant data is removed, data standardization is performed, and the extracted feature data is stored in a database to form structured data. Combined with the formed structured data, a semantic feature database is constructed; as shown in Figure 2 .

[0117] The semantic knowledge base construction module 900 is configured to construct a semantic knowledge base based on the first semantic rule and the second semantic rule, in combination with the structured data and the allocation relationship established between each segmentation network and the corresponding processor.

[0118] In this embodiment, the simple rule base constructed based on the first semantic rule and the complex rule base constructed based on the optimized second semantic rule are combined with the semantic feature database and the divided rule sub-network to construct a semantic knowledge base.

[0119] It should be noted that the semantic knowledge base construction system provided by the above embodiment is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the modules or steps in the embodiment of the application are further decomposed or combined, for example, the modules of the above embodiment can be combined into one module, or can be further split into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the application are only for distinguishing the respective modules and steps, and should not be considered as an improper limitation of the application.

[0120] A semantic knowledge base construction method of the second embodiment of the application comprises the following steps:

[0121] Extracting structured data, semi-structured data and unstructured data to be constructed as input data from the data lake;

[0122] Constructing semantic rules based on the acquired domain expert prior knowledge as first semantic rules;

[0123] Using natural language processing method to preprocess the input data for semantic extraction, semantic processing and knowledge mining, taking the preprocessed semantic information as nodes, and constructing rule nodes according to the preset semantic structure construction rules as second semantic rules;

[0124] Calculating the time cost of each rule node when executed;

[0125] Combining the time cost, merging and replacing the rule nodes through the preset dynamic optimization mechanism based on rule merging and node replacement, to obtain an optimized rule network;

[0126] Dividing the optimized rule network into independent rule subnetworks without communication relationship between each other; calculating the calculation cost of each independent rule subnetwork, and establishing a pre-allocation relationship between each independent rule subnetwork and a preset processing machine based on the balanced allocation principle;

[0127] Dividing the independent rule subnetworks having a pre-allocation relationship with multiple processing machines into multiple segmentation networks, and establishing an allocation relationship between each segmentation network and the corresponding processor;

[0128] Extracting features from the structured data, semi-structured data and unstructured data extracted from the data lake, removing redundant data, and storing the extracted feature data in a database to form structured data;

[0129] Based on the first semantic rules, the second semantic rules, combined with the structured data and the allocation relationship established between each segmentation network and the corresponding processor, a semantic knowledge base is constructed.

[0130] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related description of the above-described method can refer to the corresponding process in the foregoing system embodiment, and will not be repeated here.

[0131] The device of the third embodiment of the application comprises at least one processor and a memory connected in communication with the at least one processor, wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-described semantic knowledge base construction method.

[0132] The computer readable storage medium of the fourth embodiment of the application stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-described semantic knowledge base construction method.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related description of the above-described storage device and processing device can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here.

[0134] Reference is made below to Figure 3 which shows a structural schematic diagram of a computer system of a server suitable for being used to implement the method, system and device embodiments of the present application. Figure 3 The server shown is merely an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0135] As shown in Figure 3 , the computer system comprises a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or programs loaded from a storage portion 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for system operation are also stored. The CPU 301, the ROM 302 and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0136] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as necessary. A removable recording medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 310 as necessary, so that a computer program read out therefrom is installed in the storage section 308 as necessary.

[0137] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU 301), the above-described functions defined in the methods of the present application are performed. It should be noted that the computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0138] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0139] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0140] The terms "first", "second", etc. are used to distinguish between similar objects, and are not used to describe or indicate a particular order or sequence.

[0141] The terms "include", "comprise" or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus / device.

[0142] The technical scheme of the present application has been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. A semantic knowledge base construction system, characterized in that, The system includes: a data acquisition module, a first rule generation module, a second rule generation module, a cost calculation module, a rule optimization module, a rule subnet generation and pre-allocation module, a rule subnet partitioning module, a semantic feature data extraction module, and a semantic knowledge base construction module; The data acquisition module is configured to extract structured data, semi-structured data, and unstructured data from the data lake to be used as input data for the semantic knowledge base to be built. The first rule generation module is configured to construct semantic rules based on the acquired prior knowledge of domain experts, which serve as the first semantic rule; The second rule generation module is configured to perform semantic extraction, semantic processing and knowledge mining preprocessing on the input data using natural language processing methods, and to construct rule nodes according to a preset semantic structure as the second semantic rule. The cost calculation module is configured to calculate the time cost consumed when each rule node is executed. The rule optimization module is configured to combine time costs and merge and replace rule nodes through a preset dynamic optimization mechanism based on rule merging and node replacement to obtain an optimized rule network. The rule subnet generation and pre-allocation module is configured to divide the optimized rule network into independent rule subnets that are not connected to each other; calculate the computational cost of each independent rule subnet; and establish a pre-allocation relationship between each independent rule subnet and a preset processor based on the principle of balanced allocation. The rule subnetwork partitioning module is configured to partition the independent rule subnetworks that have pre-assignment relationships with multiple processors to obtain multiple segmented networks, and establish an allocation relationship between each segmented network and its corresponding processor. The semantic feature data extraction module is configured to extract features from structured data, semi-structured data, and unstructured data extracted from the data lake, remove redundant data, and store the extracted feature data in the database to form structured data. The semantic knowledge base construction module is configured to construct a semantic knowledge base based on the first semantic rule, the second semantic rule, combined with structured data and the allocation relationship established between each segmentation network and the corresponding processor.

2. The semantic knowledge base construction system according to claim 1, characterized in that, The rule nodes are divided into non-computational rule nodes and computational rule nodes; The non-computational rule nodes include rule relationship nodes and rule action nodes; The calculation rule nodes include rule selection nodes, rule union nodes, rule intersection nodes, rule negation calculation nodes, rule connection nodes, and rule Cartesian product nodes.

3. The semantic knowledge base construction system according to claim 1, characterized in that, The dynamic optimization mechanism based on rule merging and node replacement is as follows: Replace rule nodes with higher time costs with rule nodes with lower time costs; If the selection set of one rule node is a subset of the selection set of another rule node, or if there is overlap in the selection conditions between two rule nodes, then they are merged.

4. The semantic knowledge base construction system according to claim 3, characterized in that, The independent rule subnetwork is obtained using an undirected graph traversal method.

5. The semantic knowledge base construction system according to claim 4, characterized in that, The computational cost of each independent rule subnet is calculated as follows: Calculate the flow of all directed edges in an independent rule subnet; The computational cost of each rule node is calculated based on the directed edge traffic, and the computational cost of the corresponding independent rule subnet is obtained.

6. The semantic knowledge base construction system according to claim 5, characterized in that, "Based on the principle of balanced allocation, a pre-allocation relationship is established between each independent rule subnet and the preset processor." The method is as follows: Based on the average computation cost, a preset upward adjustment ratio, and a preset downward adjustment ratio, the computation cost range of a single processor is obtained. For each independent rule subnet whose computation cost falls within the aforementioned computation cost range, a separate processor shall be allocated; Independent rule subnets that are less than the lower limit of the computation cost range are combined. If the sum of the computation costs of the combined independent rule subnets falls within the computation cost range, the combined independent rule subnets are assigned to a single processor. For independent rule subnets that exceed the upper limit of the computation cost range, allocate multiple processors.

7. The semantic knowledge base construction system according to claim 6, characterized in that, The method for "assigning multiple processors to independent rule subnets that exceed the upper limit of the computational cost range" is as follows: If the computation cost of an independent rule subnet is greater than the upper limit of the computation cost range, and its computation cost falls within the range of K1 times the computation cost range, then the independent rule subnet shall be assigned to K1 processors. Otherwise, if the sum of the computational costs of multiple independent rule subnets that are greater than the upper limit of the computational cost range falls within the range of K2 times the computational cost range, then the combination of the independent rule subnets is allocated to K2 processors.

8. A method for constructing a semantic knowledge base, characterized in that, The method includes the following steps: Structured, semi-structured, and unstructured data to be used as input data are extracted from the data lake to build the semantic knowledge base. Semantic rules are constructed based on the acquired prior knowledge from domain experts and serve as the first semantic rule; Natural language processing methods are used to perform semantic extraction, semantic processing and knowledge mining preprocessing on the input data. The preprocessed semantic information is used as nodes, and rule nodes are constructed according to the preset semantic structure as second semantic rules. Calculate the time cost of executing each rule node; By combining time costs, a pre-defined dynamic optimization mechanism based on rule merging and node replacement is used to merge and replace rule nodes, resulting in an optimized rule network. The optimized rule network is divided into independent rule subnets that are not connected to each other; the computational cost of each independent rule subnet is calculated, and based on the principle of balanced allocation, a pre-allocation relationship between each independent rule subnet and the preset processor is established. The independent rule subnets that have pre-assignment relationships with multiple processors are divided into multiple segmented networks, and each segmented network is assigned a relationship with its corresponding processor. Feature extraction is performed on structured, semi-structured, and unstructured data extracted from the data lake. Redundant data is removed, and the extracted feature data is stored in the database to form structured data. Based on the first semantic rule and the second semantic rule, combined with structured data and the allocation relationship established between each segmentation network and the corresponding processor, a semantic knowledge base is constructed.

9. A device, characterized in that, include: At least one processor; as well as A memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor to implement the semantic knowledge base construction method of claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by the computer to implement the semantic knowledge base construction method of claim 8.

Images