Recognition Model in Knowledge Management Artificial Intelligence System

Abstract

The present invention provides a recognition model based on a knowledge management artificial intelligence system, which is used in an artificial intelligence knowledge management system. The system includes a processor, a computer-readable non-transitory storage medium, and a display. The computer-readable non-transitory storage medium stores the following program modules, and can be processed by the processor to implement the functions of the recognition model based on a heterogeneous graph neural network: a knowledge recognition module, a knowledge selection module, a knowledge parsing module, and an evaluation module. The efficiency of knowledge management is improved, and the participation of knowledge managers in the knowledge recognition process is enhanced by selectively identifying knowledge that users are interested in. A big data foundation is provided for the contributions of different users to the technological development and technological trends in the knowledge field.

Description

Technical Field

[0001] The present invention relates to a recognition model used in a knowledge management system, and in particular to a recognition model in a knowledge management artificial intelligence system, and belongs to the field of knowledge intelligence management. Background Art

[0002] The existing technology uses pictures as the basis to learn the spatial relationship to the physical object relationship, so it is necessary to prepare a lot of models, and thus it is necessary to search for models to find a model that is suitable for recognition. Therefore, from spatial relationship division to model search to adaptive recognition, the overall efficiency is extremely low. Although its advantage is that the recognition accuracy of individual physical objects is relatively high, its disadvantages are also prominent, that is, it is not efficient, and it will not be based on pictures. The fundamental reason is that each model is constructed independently of each other, lacking an overall recognition model, or using a few models to achieve it. At the same time, the existing technology has no selectivity for the recognition of knowledge, and the picture as a whole is recognized and analyzed together, lacking the participation of knowledge managers. The existing technology has a single form of analysis objects, which is limited to pictures, and cannot analyze the current status and trend of knowledge development. Summary of the invention

[0003] Based on the above-mentioned drawbacks and shortcomings of the prior art, the present invention provides a recognition model based on a heterogeneous graph neural network. The idea is to graph the heterogeneous knowledge system to form an overall network. Secondly, the recognition model is designed to be a model for the knowledge document format, so as to enter the corresponding knowledge analysis through format recognition. Finally, the results of the knowledge analysis are selectively projected into the knowledge heterogeneous graph network to achieve the goal of learning from one example and applying it to other situations.

[0004] In view of the above ideas, the present invention provides a recognition model based on a heterogeneous graph neural network in an artificial intelligence knowledge management system, specifically including an artificial intelligence knowledge management system storing the recognition model, the artificial intelligence knowledge management system including a knowledge recognition module for identifying the format of a knowledge document input therein, and when the format is a text or audio format, recognizing words,

[0005] The knowledge selection module is used to receive the knowledge document in the recognized format and the recognized words from the knowledge recognition module, so that the user can select the knowledge of interest in the knowledge document and / or at least part of the words for subsequent knowledge analysis.

[0006] The knowledge parsing module is used to construct a heterogeneous graph neural network (HGNN) of the knowledge system, and to perform knowledge parsing on the knowledge document and / or the word selected by the user, and to project the knowledge parsing result into the heterogeneous graph neural network, call out all meta-paths (that is, theoretically all possible meta-paths that meet the user's customized permutations and combinations), and record the spatial location and time when the user is selected.

[0007] An evaluation module is used to establish an attention mechanism (AM) based on a heterogeneous graph neural network, and to provide the meta-paths of the user belonging to the spatial location and other users around the user according to all the called meta-paths and the spatial location, and to further call out the attention meta-paths with a correlation degree exceeding a first threshold through the attention mechanism, as well as the spatial location and corresponding time corresponding to the attention user corresponding to the attention meta-path, so as to be further used for knowledge evaluation.

[0008] The method for recognizing a word comprises the following steps:

[0009] The first step is to build a natural language processing model NLP and a speech recognition model V for recognizing words in text and audio formats.

[0010] The second step is to input the text directly into the natural language processing model to recognize the words, input the audio into the speech recognition model first to form recognized text, and then input the recognized text into the natural language processing model to recognize the words, and put them into the historical knowledge selection list of the user and / or other users in the spatial location where the user is located and the surrounding area.

[0011] The third step is to select relevant words of interest as keywords based on the historical knowledge selection list (formed by historical knowledge analysis) of the user and / or other users in and around the spatial location where the user is located, and delete the recognized words. If the selection fails, only the words in the historical knowledge selection list of the user are retained to update the list, and the words related to the failed selection are also used as keywords for the user to select in the knowledge selection module.

[0012] Optionally, the attention mechanism is used for a historical knowledge selection list of the user and / or the other users to select the keywords for the user and / or to select the keywords for the user and the other users.

[0013] Optionally, the method of applying the attention mechanism to the historical knowledge selection list of the user and / or the other users specifically includes projecting the inserted word in the historical knowledge selection list into a heterogeneous graph neural network, calling out meta-path neighbors customized by users and / or other users whose association degree exceeds a second threshold; selecting the keywords for the user, and / or selecting the keywords for the user and the other users specifically includes using the inserted word itself and its neighbors as keywords for the user to select in the knowledge selection module.

[0014] Preferably, the words in the historical knowledge selection list are clustered into synonyms and antonyms, and level clustering is performed for the proximity of the synonyms.

[0015] It is easy to understand that the failure of selection corresponds to the degree of association being lower than the second threshold, or the recognized word is not synonymous with the word in any list, and / or the degree of proximity is lower than the level of any clustering, so these two situations are identified as not appearing in the historical knowledge selection list and are defined as selection failure. The selection of keywords is critical because the subsequent heterogeneous graph neural network and the attention mechanism based on the network are constructed based on keywords as nodes. Since the historical knowledge selection list is constantly updated and the degree of attention is constantly changing, that is, the node importance and meta-path association degree of the attention mechanism are constantly changing, the selection results will change, and it is not a simple comparison of whether the words recognized in the voice and video exist in the list.

[0016] Failure in selection means that there are new hot spots or other developments in the area that the user is concerned about. We update the list accordingly to record the changes and developments in knowledge. As for the successfully selected words, it is considered that the contribution to the existing knowledge is limited, because the corresponding meta-path is higher than the preset second threshold, so the words are not recorded but deleted. In summary, by trusting the degree of knowledge association subjectively expressed by the user-defined meta-path, we can sniff out the degree of contribution of the words to the knowledge in the local geographical area through the attention mechanism.

[0017] It should be noted that the concerned users are not the same as the other users, and the basis for the division of the two is different. The former is related to the meta-path formed by projecting the heterogeneous graph neural network with the words (or keywords) selected by the users, while the latter is based on the spatial location of the users, and the two do not necessarily have an intersection.

[0018] Optionally, the format of the knowledge document includes pictures, videos, documents, and voices, the text includes any one of pictures, videos, documents, audios or a combination thereof, the audio is derived from video and / or voice, and the knowledge document in the identified format includes pictures and videos, and the user can select the knowledge of interest in the knowledge document by circling the part of interest from the picture and taking a screenshot of the video and circling the part of interest.

[0019] The heterogeneous graph neural network for constructing the knowledge system includes the following steps:

[0020] S1 builds a knowledge heterogeneous graph network , A collection of keyword nodes for different knowledge fields. is a set of knowledge edges. Knowledge edges represent the association between different keywords;

[0021] S2 Build the meta path: , and define the neighbors of the meta-path, that is, for any node , then connected by meta-path All nodes of are neighbors, where Add 1 to the total number of nodes on the meta-path. , , is a natural number, and , is the association between keywords, each of which is a double arrow representing the adjacent nodes in the meta-path in the knowledge domain, so It also indicates the number of double arrows, while single-pointed arrows It only indicates the order of different nodes in the meta-path.

[0022] It should be emphasized that for any two nodes and belong , the associations in the knowledge domain The double arrows in the figure represent the association of the same field, the same product, the same method, or a combination thereof. They are logical association symbols, and their set is the knowledge edge set. .

[0023] The method for performing knowledge parsing on the knowledge document and / or the word selected by the user comprises:

[0024] S3 (i.e., following S2) builds an artificial intelligence model, specifically, a convolutional neural network (CNN) and a generative adversarial network (GAN), classifies knowledge through the parts of interest circled in the history of all users, and divides them into training sets and validation sets, with the ratio of 7-5:1-3. For the generative adversarial network (GAN), the parts of interest circled in the history of knowledge classification are additionally divided as a real atlas for training the judge in the GAN. The training set is used to train the CNN and GAN respectively, and their output classification is the knowledge classification result. The validation set is used to verify the two models at the same time;

[0025] S4 inputs the circled part of interest to be recognized into CNN and GAN for recognition. When the word is selected for knowledge analysis, the three types of recognition results (i.e., CNN, GAN, and word recognition results) are used as knowledge analysis results. When the word is not selected for knowledge analysis, only the CNN and GAN recognition results are used as knowledge analysis results.

[0026] Optionally, the knowledge classification is to manually compare the words in the historical knowledge selection list with the interesting parts circled in history, and classify the interesting parts circled in history into the word cluster in the historical knowledge selection list that is suitable for describing itself. The method of knowledge parsing of words is to directly compare the words with the synonyms and antonyms of the cluster, and obtain the words named in the closest cluster as the recognition result.

[0027] Therefore, based on a heterogeneous graph network, combined with a historical knowledge selection list, the user's interests are bound to knowledge, and a few natural language models, speech recognition models, CNN, and GAN are used to organize and manage all knowledge fields, so that users can selectively perform related knowledge parsing operations after obtaining knowledge documents, and then obtain knowledge neighbors on related meta-paths, achieving the technical effect of learning from one example and applying it to other cases. We take into account the respective advantages of CNN and GAN models, and use them as identification methods to increase the expansion of knowledge and the possibility of more accurate selection for users.

[0028] Methods for establishing attention mechanisms based on heterogeneous graph neural networks include:

[0029] S5 Any user (i.e., user in all spatial locations) can customize the meta-path based on the selected words, including Size and different sizes The number of types, that is, the number of types of meta-paths, forms the meta-path set , in the constructed heterogeneous graph neural network, the corresponding meta-path set Each meta-path connects a pair of nodes , construct node attention (1) , is a deep neural network with node attention, Node The representation of node That is, it corresponds to the node where the knowledge analysis result generated by the user's selection is projected into the heterogeneous graph neural network;

[0030] S6 Compute Node All meta-path based collections Neighbors The following formula (2) gives the expression of the node attention normalization coefficient of formula (1): (2), is the concatenation operator represented by, Metapath The node attention vector of represents transpose, For Node The number of neighbor nodes of is the activation function.

[0031] It should be understood that Describes the node corresponding to the word selected by the user In the different types of meta path collection based on the user's selection The importance set of nodes. The importance of other more relevant or more important knowledge. It can be composed of nodes under the same knowledge classification, or nodes representing knowledge in other fields under different knowledge classifications. Different choices are made according to the user's customized meta-path. The larger the value, the stronger the correlation between the word selected by the user and the knowledge in this category or even other categories. To a certain extent, it represents the current development status of the knowledge that the user is interested in and the intersection with knowledge in other fields.

[0032] S7 Compute Node Based on meta-path collection The feature representation , and finally repeat the attention mechanism times, get (3), Indicates that the attention mechanism is repeated Splicing is performed again;

[0033] S8 calculates the metapath set Weight (4) Among them Represents the modulus of the corresponding node set, with superscripts and subscripts , with subscript , They are the attention vector, weight, and bias representation of the corresponding semantics respectively;

[0034] S9 Metapath Collection In knowledge heterogeneous graph networks Form subgraphs and calculate nodes All based on The attention of each meta-path in , the following formula (5) obtains the expression of the semantic attention normalization coefficient: (5);

[0035] S10 builds meta-path attention (6) indicates that Any path The meta-path attention under the subgraph is the deep neural network represented by the semantic attention under the corresponding path After learning, the feature representation of the semantic attention mechanism is (7), use Substitution or or a combination thereof to calculate the cross entropy loss ,in Indicates the number of synonyms and synonym cluster labels, is the cluster label set, For the number The cluster labels of is the node classifier parameter, , by training each attention deep neural network until the cross entropy loss stabilizes and minimizes.

[0036] It is easy to understand that the semantic attention normalization coefficient Describes the degree of association between different meta-paths in the subgraph, The larger the value, the greater the correlation between the selected meta-paths, and the greater the correlation between the neighboring nodes on the meta-path. , more relevant knowledge can be called out from the subgraph. Since the attention mechanism is for users in all spatial locations, the algorithm of the mechanism is applicable to the aforementioned users, the other users (users around the user's spatial location), and the focused users.

[0037] Projecting the knowledge parsing results into the heterogeneous graph neural network and calling out all meta-paths. Specific methods include:

[0038] Q1 builds a mapping interface in the nodes of the heterogeneous graph neural network, and the knowledge parsing results are mapped to the corresponding nodes through the mapping interface, that is, Indicates that the corresponding node includes synonym nodes and / or near-synonymous nodes in different knowledge fields;

[0039] Q2 searches for all meta-paths containing the corresponding nodes in the heterogeneous graph neural network based on the user-defined meta-path.

[0040] The attention mechanism is used to further call out the attention meta-paths whose correlation degree exceeds the first threshold, including:

[0041] S11 traverses the set of meta-paths belonging to the spatial location and its surroundings All combinations of , It is the combination number, each number represents a different The combination of the meta-paths and the corresponding nodes are expressed by formulas (1)-(7) Substitute the trained Get multiple , where multiple representations are formed Specifically based on The corresponding nodes on the , in the multiple representations under different combination numbers formed by formulas (1)-(7) ;

[0042] S12 sets the first threshold to ,when (8), call out The corresponding path combination, that is, the meta-path of interest (Subscript represents the set of all meta-paths that satisfy formula (8).

[0043] Optionally, the search for the concerned user corresponding to the concerned meta-path includes searching for the concerned user according to the concerned meta-path. , search for the spatial location and its surrounding areas that have been customized in history Any user of the meta path.

[0044] The method of knowledge evaluation comprises:

[0045] S13 obtains the space location and time corresponding to the user and the user being followed, and calculates the time according to the meta-path of the user being followed. , find out The corresponding nodes in form corresponding representations ;

[0046] S14 According to formulas (1) and (2), we can find , let the second threshold be , find satisfaction When the corresponding node is in the meta-path of interest The neighbor set on (9) Draw a spatial distribution map according to the corresponding spatial locations, that is, Distribution, and draw a time distribution diagram in chronological order, that is, Distribution, according to Draw the correlation degree distribution diagram in increasing or decreasing order, that is distributed;

[0047] S15 distributed, distributed, The location of at least one of the user, the other users, and the concerned user is marked in any one of the distributions or a combination thereof.

[0048] It is easy to understand that Distribution is to mark the empty position, that is, the spatial location, for Distribution marks the time point in the chronological order. The distribution is marked in an increasing or decreasing sequence.

[0049] For projecting the inserted word in the historical knowledge selection list into the heterogeneous graph neural network, calling out the meta-path neighbors customized by the user and / or other users whose association degree exceeds the second threshold; selecting the keyword for the user, and / or selecting the keyword for the user and the other users specifically includes taking the inserted word itself and its neighbors as keywords, and specifically taking the inserted word in the historical knowledge selection list according to steps Q1-Q2, and for the user and / or the user, continuing to follow S11-S14 to obtain a neighbor set as shown in formula (9), and taking it and the inserted word itself as the selected keyword.

[0050] Beneficial Effects

[0051] The recognition model stored in the system of the present invention can identify, classify and distribute the knowledge of interest to users in different geographical locations based on heterogeneous graph neural networks and using a small number of modeling methods, thereby improving the efficiency of knowledge management, enabling selective identification of the knowledge of interest to users and enhancing the participation of knowledge managers in the knowledge identification process. This provides a big data foundation for the contributions of different users to the technological development and technological trends in the knowledge field. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The embodiment 1 of the present invention shows an implementation method of a recognition model based on a heterogeneous graph neural network in an artificial intelligence knowledge management system;

[0053] Figure 2 The program implementation interface diagram of the recognition model based on heterogeneous graph neural network in the artificial intelligence knowledge management system according to the first embodiment of the present invention is shown, wherein ad is a schematic diagram of the interface windows under the map label, knowledge selection label, knowledge analysis label, and evaluation label respectively;

[0054] Figure 3 A schematic diagram of local visualization of a heterogeneous graph neural network for constructing a knowledge system according to Embodiment 2 of the present invention is shown;

[0055] Figure 4 A flowchart showing a method for performing knowledge parsing on a knowledge document and / or the word selected by a user and projecting the knowledge parsing result into the heterogeneous graph neural network and calling out a specific method of all meta-paths according to an embodiment of the present invention is shown;

[0056] Figure 5A simplified flowchart of a method for establishing an attention mechanism based on a heterogeneous graph neural network according to an embodiment of the present invention is shown;

[0057] Figure 6 A simplified flowchart of a specific method for further calling out an attention meta-path with a correlation degree exceeding a first threshold through an attention mechanism in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0058] Example 1

[0059] This embodiment describes the implementation of the recognition model based on heterogeneous graph neural network. Figure 1 An implementation method of a recognition model in a knowledge management artificial intelligence system is given, which specifically includes the use of an artificial intelligence knowledge management system that stores the recognition model. The artificial intelligence knowledge management system is hardware, and the functions of the recognition model are realized by processing the following program modules stored in a computer-readable non-temporary storage medium by the included processor: knowledge recognition module (represented by a light green shaped box), knowledge selection module (represented by a black shaped box), knowledge parsing module (represented by a red shaped box), and evaluation module (represented by a blue rectangular box). The recognition model based on a heterogeneous graph neural network is composed of the above program modules.

[0060] The knowledge recognition module is used to identify the format of the knowledge documents input, including pictures, videos, documents, and voices. It can identify the format of pictures and videos, the text format of documents, and the audio format of voices. For the latter two, NLP and V are used to recognize words and put them into Figure 2 The historical knowledge selection list of the user shown in a and other surrounding users ( Figure 1 The user selects keywords using AM (indicated by a + sign in the example 2). If the selection is successful, the keywords are classified as keywords and the words are deleted. If the selection fails, only the user's list is updated and the words that failed to be selected are also classified as keywords. The knowledge documents of the format, i.e., pictures and videos, and the recognized words, i.e., keywords, are identified and input into the knowledge selection module.

[0061] Knowledge selection module, used to accept pictures and videos ( Figure 1 in dark green) and keywords ( Figure 1 In the figure, the user can select the knowledge of interest in the picture or video (with light green guide lines), that is, circle the part of interest from the picture or video frame (also with dark green guide lines), and the words that are at least part of the keywords (also with light green guide lines), for subsequent knowledge analysis.

[0062] The knowledge parsing module is used to construct the HGNN, and perform knowledge parsing on the circled parts of interest and keywords selected by the user, and project the knowledge parsing results into the heterogeneous graph neural network, call out all meta-paths, and record the spatial location and time when the user is selected.

[0063] The evaluation module is used to establish an attention mechanism (AM) based on a heterogeneous graph neural network, and to provide meta-paths belonging to the spatial location and its surroundings (within a radius of 1-30 km with the spatial location as the center) based on all the called meta-paths (guided by purple arrows) and the spatial location (guided by lake blue arrows). AM is used to call out the attention meta-paths with a correlation degree exceeding a first threshold, as well as the spatial locations corresponding to the attention users corresponding to the attention meta-paths and the corresponding times, for further use in knowledge evaluation.

[0064] The above module functions are implemented by software on the system, such as Figure 2 As shown, a schematic interface on the system display is given. Figure 2 a shows the topmost function menu bar, including four labels that can be selected by clicking the mouse: map, knowledge selection, knowledge analysis, and evaluation. It also includes the "×", chamfered rectangle, and short horizontal line "—" on the right, which respectively represent the closing, maximization, and minimization of the interface window. The interface window can be scaled to any size ratio by clicking and dragging the corners or edges of the interface with the mouse. Double-clicking the blank space of the menu bar can also maximize the interface. Clicking the blank space with the mouse and dragging the mouse will move the interface. These functions are similar in Figure 2 b-It is implemented on the corresponding interface window in 2d.

[0065] Among them, the last three of the four tags correspond to the entry points for the knowledge selection module, knowledge analysis module, and evaluation module functions. Under the map tag, a digital map is used (the map tag is gray to indicate that the currently activated tag is a map tag) to describe the geographic locations of other users around the user's machine, so as to Figure 2 The mouse cursor shown in a clicks the position where the word "user" is located, and the corresponding historical knowledge selection list is displayed on the right side of the menu bar. The coordinates of the user, that is, the spatial location, are given. In the middle, there are synonym and near-synonym buttons. Click them to view the synonym clusters and corresponding near-synonym clusters of the keywords selected by the user in the past. Similarly, click Figure 2 The red dots around any user in a represent other users around them, and the historical knowledge selection lists of other users are displayed accordingly.

[0066] The button with a "+" in the lower right corner of the digital map is the knowledge recognition module. Clicking it will prompt you to import the knowledge document. After the import is successful, the above functions of the knowledge recognition module will be realized.

[0067] If you click the knowledge selection label, you will enter Figure 2 b's exemplary interface window (omitted the menu bar, Figure 2 c and Figure 2 d). After knowledge recognition, the left side of the interface displays all the keywords 1, keyword 2, keyword 3 for users to choose, until the middle is omitted, and finally the picture / video area button. Use the dark gray slider on the left side of the icon to drag and browse the keywords. You can also type keywords in the keyword search bar at the top left of the icon, and click the magnifying glass button on the right side of the keyword search bar to search for keywords. If the search fails, a dialog box will pop up ( Figure 2 b), the user can choose whether to add it as a new keyword as needed. You can also right-click the keyword and choose to delete it ( Figure 2 not shown in b).

[0068] Click the picture / video area button and a pop-up window will appear. Figure 2 b. The arrows in the lower right corner are for browsing the input pictures forward (dark green left arrow) and backward (dark green right arrow) in sequence. Starting from the left, the lower left corner is for video play, pause, and close buttons. When the close button is pressed, the arrow is activated to browse the pictures. When the play button is pressed, the playback of a video is started. Press the pause button, and different videos can be switched by the arrows. The switched video is displayed in the picture / video area window in the form of the first frame, so that by selecting play or pause, you can arbitrarily select the frame of interest in any video. Then use the mouse to take a screenshot in the manner shown in the figure (click a selected position and drag in the lower right corner direction) to circle the part of interest in the picture or video frame.

[0069] Figure 2 The right side of the b interface window is the dictionary, where users can add synonyms and near-synonyms, query (via the built-in electronic dictionary or online query) the explanations of synonyms and near-synonyms, define the degree of proximity of near-synonyms, and perform clustering updates.

[0070] Click the Knowledge Analysis tab to display the following Figure 2 c shows the interface, where you click the light green "HGNN" button on the upper right side, and the corresponding light green HGNN construction area window pops up, in which the heterogeneous graph neural network of the knowledge system is constructed. Click the light ochre "Analysis" button on the upper right side, and the corresponding light ochre knowledge analysis area window pops up. In the knowledge analysis area window, you can further view the Figure 2 b's keywords are displayed in a pull-up manner, so that the user can finally confirm whether the keywords are correct. Otherwise, enter Figure 2 b. Go back to the interface window under the knowledge selection tab to add and delete again.

[0071] There are CNN and GAN buttons on the right side of the knowledge parsing area window, which are used for training CNN and GAN models in the implementation of knowledge parsing methods. Figure 2 c Take clicking CNN as an example, and a black CNN window pops up. It displays the CNN network parameters of parameter 1 and parameter 2, as well as the training / verification button at the bottom, and the classification and division buttons, which are used for knowledge classification and the division ratio setting of the training set, verification set, and real atlas. Clicking the training / verification button pops up the list of training sets and verification sets (not shown in the figure). The user can select the number of training sets for the training set, so that the division ratio of the corresponding verification set number changes, and starting training starts automatic training and verification. During the training process, the user can manually adjust the parameters in the window to assist in optimizing the network parameters.

[0072] There is also a knowledge analysis button on the right side of the knowledge analysis area window. After clicking it, the currently trained CNN and GAN are used for knowledge analysis, and then a dialog box pops up to show that the analysis is complete. Then the user clicks the projection button to complete the projection of the knowledge analysis results into the heterogeneous graph neural network. Finally, click the call / record button to complete the call of all meta-paths and record the spatial location and time when the user selects.

[0073] Click the Evaluation tab to enter Figure 2 d shows the evaluation window. Similarly, click the corresponding AM button, Follow Metapath button, Follow User button, and Evaluation button on the right, and the corresponding window will pop up. Figure 2 d shows that when you click AM, the AM creation area button pops up. When you click the Follow Meta Path button and Follow User button, they are displayed in the upper and lower windows (i.e., Follow Meta Path area and Follow User area) of the same window. You can also drag the middle horizontal bar with the mouse to adjust the display area of ​​the two areas. The Follow Meta Path area can display the Follow Meta Path. .

[0074] Example 2

[0075] This embodiment will illustrate the method for implementing the functions of each module in Embodiment 1.

[0076] first, Figure 1 The heterogeneous graph neural network for constructing a knowledge system in the knowledge parsing module shown includes the following steps:

[0077] S1 builds a knowledge heterogeneous graph network , A collection of keyword nodes for different knowledge fields. is a set of knowledge edges, which represent the associations between different keywords.

[0078] S2 The meta path constructed in . Figure 3 Example shown The constructed meta path shown in the local area: black ,red , respectively representing meta-paths of two different or similar fields (a six-branch and a four-branch, each branch contains one keyword, for example, more keywords are polygons with more sides). And define the neighbors of the meta-path, that is, for any node , then connected by meta-path All nodes of are neighbors. Neighbors of and ,red Neighbors of and . Black and red , is the association between keywords, each of which is a double arrow representing the adjacent nodes in the meta-path in the knowledge domain, so It also indicates the number of double arrows, while single-pointed arrows It only indicates the order of different nodes in the meta-path. Figure 3 It is just an example of visualization to help understand the abstract mathematical heterogeneous graph network.

[0079] like Figure 4 As shown, Figure 1 The method for performing knowledge parsing on the knowledge document and / or the word selected by the user in the knowledge parsing module shown includes:

[0080] Following S2, step S3 is performed to build an artificial intelligence model, specifically, a convolutional neural network CNN and a generative adversarial network GAN are constructed, knowledge is classified through the parts of interest circled in the history of all users, and a training set and a validation set are divided, with the ratio of the two being 5:1. For the generative adversarial network GAN, the parts of interest circled in the history of knowledge classification are additionally divided as a real atlas for training the judge in the GAN. The training set is used to train the CNN and GAN respectively, and their output classification is the knowledge classification result. The validation set is used to verify the two models at the same time.

[0081] S4 inputs the circled part of interest to be recognized into CNN and GAN for recognition. When the word is selected for knowledge analysis, the three types of recognition results, namely, the recognition results of CNN, GAN, and word, are used as knowledge analysis results. When the word is not selected for knowledge analysis, only the recognition results of CNN and GAN are used as knowledge analysis results.

[0082] Secondly, if Figure 5 As shown, Figure 1The methods for establishing the attention mechanism based on heterogeneous graph neural network in the evaluation module include:

[0083] S5 Any user, that is, a user in all spatial locations, customizes the meta-path based on the selected words, including Size and different sizes The number of types, that is, the number of types of meta-paths, forms the meta-path set , in the constructed heterogeneous graph neural network, the corresponding meta-path set Each meta-path connects a pair of nodes , construct node attention (1) , is a deep neural network with node attention, Node The representation of node That is, it corresponds to the node where the knowledge analysis result generated by the user's selection is projected into the heterogeneous graph neural network;

[0084] S6 Compute Node All meta-path based collections Neighbors The following formula (2) gives the expression of the node attention normalization coefficient of formula (1): (2), is the concatenation operator represented by, Metapath The node attention vector of represents transpose, For Node The number of neighbor nodes of is the activation function;

[0085] S7 Compute Node Based on meta-path collection The feature representation , and finally repeat the attention mechanism times, get (3), Indicates that the attention mechanism is repeated Splicing is performed again;

[0086] S8 calculates the metapath set Weight (4) Among them Represents the modulus of the corresponding node set, with superscripts and subscripts , with subscript , They are the attention vector, weight, and bias representation of the corresponding semantics respectively;

[0087] S9 Metapath Collection In knowledge heterogeneous graph networks Form subgraphs and calculate nodes All based on The attention of each meta-path in , the following formula (5) obtains the expression of the semantic attention normalization coefficient: (5);

[0088] S10 builds meta-path attention (6) indicates that Any path The meta-path attention under the subgraph is the deep neural network represented by the semantic attention under the corresponding path After learning, the feature representation of the semantic attention mechanism is (7), use Substitution The cross entropy loss is calculated in ,in Indicates the number of synonyms and synonym cluster labels, is the cluster label set, For the number The cluster labels of is the node classifier parameter, , by training each attention deep neural network until the cross entropy loss stabilizes and minimizes. Figure 5 The logic of the above construction process of formulas (1)-(7) and the training process of the attention mechanism is sorted out.

[0089] The knowledge parsing results are projected into the heterogeneous graph neural network, and the specific method of calling all meta-paths is still Figure 4 As shown, after obtaining the knowledge parsing results, proceed as follows:

[0090] Q1 In heterogeneous graph neural networks (still based on Figure 3 Visual Mapping interfaces 1 to 6 are constructed in the nodes of the local area as an example, and the knowledge parsing results are mapped to the corresponding nodes through these mapping interfaces, that is, The corresponding nodes include synonym nodes and / or near-synonymous nodes in different knowledge domains. The mapping interface is easily implemented in the form of pointers, for example. The historical knowledge selection list is selected by AM, and then mapped to the corresponding node through the corresponding pointer.

[0091] Q2 searches for all meta-paths containing the corresponding nodes in the heterogeneous graph neural network based on the user-defined meta-path.

[0092] like Figure 6As shown, Figure 1 In the evaluation module, the attention mechanism is used to further call out the attention meta-paths whose correlation degree exceeds the first threshold, including:

[0093] S11 traverses the spatial location and its surroundings (respectively from Figure 1 The meta-path collection of the spatial location described by the user and other users around him All combinations of , It is the combination number, each number represents a different A combination of meta-paths, and the corresponding nodes. Figure 6 The black origin and red triangle represent the six-branch and four-branch knowledge fields. In the example, multiple combinations of the two types of knowledge areas are possible.

[0094] Depend on Figure 5 Formulas (1)-(7) in the above formula form multiple expressions Substitute the trained Get multiple , where multiple representations are formed Specifically based on The corresponding nodes on the , and then by Figure 5 Formulas (1)-(7) in the above formulas form multiple representations under different combination numbers. Among them, Figure 6 Medium Black and red To exemplarily give the sequential positions of the corresponding nodes in the two user-defined meta-paths in the combination.

[0095] S12 sets the first threshold to ,when (8), call out The corresponding path combination, that is, the meta-path of interest , represents the set of all meta-paths that satisfy formula (8).

[0096] exist Figure 1 In the evaluation module, the concerned user corresponding to the concerned meta-path is obtained by searching, and the specific searching method includes the concerned meta-path obtained according to S12. ,use Figure 1 The records of the knowledge parsing module in the above-mentioned space are searched for the historically customized records of the above-mentioned space location and its surroundings. Any user of the meta path.

[0097] Finally, still Figure 6 As shown, Figure 1In the evaluation module, the method of knowledge evaluation includes:

[0098] S13 obtains the space location and time corresponding to the user and the user being followed, and calculates the time according to the meta-path of the user being followed. , find out The corresponding nodes in form corresponding representations ;

[0099] S14 According to formulas (1) and (2), we can find , let the second threshold be , find satisfaction When the corresponding node is in the meta-path of interest The neighbor set on (9) Draw a spatial distribution map according to the corresponding spatial locations, that is, Distribution, and draw a time distribution diagram in chronological order, that is, Distribution, according to Draw the correlation degree distribution diagram in increasing or decreasing order, that is distributed,

[0100] S15 distributed, distributed, Any one or a combination of the distributions may mark the locations of the user, the other users, and the concerned user.

[0101] Regarding the selection of keywords in Example 1, the words in the historical knowledge selection list are also put in according to steps Q1-Q2, and for the user and / or the user, the neighbor set as shown in formula (9) is obtained in accordance with S11-S14, and the neighbor set is put into the word itself as the selected keyword.

[0102] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An artificial intelligence knowledge management system, comprising a processor, a computer-readable non-transitory storage medium, and a display, characterized in that: The computer-readable non-transitory storage medium stores the following program modules, and can be processed by the processor to implement the functions of the recognition model based on the heterogeneous graph neural network: The knowledge recognition module is used to recognize the format of the knowledge document input therein, and when the format is text or audio format, recognize the words, The knowledge selection module is used to receive the knowledge document in the recognized format and the recognized words from the knowledge recognition module, so that the user can select the knowledge of interest in the knowledge document and / or at least part of the words for subsequent knowledge analysis. The knowledge parsing module is used to construct a heterogeneous graph neural network HGNN of the knowledge system, and to perform knowledge parsing on the knowledge document and / or the word selected by the user, and to project the knowledge parsing result into the heterogeneous graph neural network, call out all meta-paths, and record the spatial location and time when the user selects. An evaluation module is used to establish an attention mechanism AM based on a heterogeneous graph neural network, and to provide the meta-paths of the user belonging to the spatial location and other users around the spatial location according to all the meta-paths and the spatial location called, and to further call out the attention meta-paths with a correlation degree exceeding a first threshold through the attention mechanism, as well as the spatial location and corresponding time corresponding to the attention user corresponding to the attention meta-path, so as to be further used for knowledge evaluation, wherein the recognition model based on the heterogeneous graph neural network is composed of the above-mentioned program modules; The heterogeneous graph neural network for constructing the knowledge system includes the following steps: S1 builds a knowledge heterogeneous graph network , A collection of keyword nodes for different knowledge fields. is the knowledge edge set; S2 Build the meta path: , and define the neighbors of the meta-path, that is, for any node , then connected by meta-path All nodes of are neighbors, where , , is a natural number, and , is the association between keywords, each of which is a double arrow representing the adjacent nodes in the meta-path in the knowledge domain, so It also indicates the number of double arrows, while single-pointed arrows It only indicates the order of different nodes in the meta-path; The method for performing knowledge parsing on the knowledge document and / or the word selected by the user comprises: S3 builds an artificial intelligence model, specifically a convolutional neural network (CNN) and a generative adversarial network (GAN). Knowledge is classified through the parts of interest circled in the history of all users, and the training set and the validation set are divided in a ratio of 7-5:1-3. For the generative adversarial network (GAN), the parts of interest circled in the history of knowledge classification are additionally divided as a real atlas for training the judge in GAN. The training set is used to train CNN and GAN respectively, and their output classification is the knowledge classification result. The validation set is used to verify the two models at the same time. S4 inputs the circled part of interest to be recognized into CNN and GAN for recognition. When the word is selected for knowledge analysis, the three types of recognition results are used as knowledge analysis results. When the word is not selected for knowledge analysis, only the CNN and GAN recognition results are used as knowledge analysis results. The three types of recognition results include the recognition results of CNN, GAN, and word. Methods for establishing attention mechanisms based on heterogeneous graph neural networks include: S5 Any user can customize the meta path based on the words they choose, including Size and different sizes The number of types, that is, the number of types of meta-paths, forms the meta-path set , in the constructed heterogeneous graph neural network, the corresponding meta-path set Each meta-path connects a pair of nodes , construct node attention (1) , is a deep neural network with node attention, Node The representation of node That is, it corresponds to the node where the knowledge analysis result generated by the user's selection is projected into the heterogeneous graph neural network; S6 Compute Node All meta-path based collections Neighbors The following formula (2) gives the expression of the node attention normalization coefficient of formula (1): (2), is the concatenation operator represented by, Metapath The node attention vector of represents transpose, For Node The number of neighbor nodes of is the activation function; S7 Compute Node Based on meta path collection The feature representation , and finally repeat the attention mechanism times, get (3), Indicates that the attention mechanism is repeated Splicing is performed again; S8 calculates the metapath set Weight (4) Among them Represents the modulus of the corresponding node set, with superscripts and subscripts , with subscript , They are the attention vector, weight, and bias representation of the corresponding semantics respectively; S9 Metapath Collection In knowledge heterogeneous graph networks Form subgraphs and calculate nodes All based on The attention of each meta-path in , the following formula (5) obtains the expression of the semantic attention normalization coefficient: (5); S10 builds meta-path attention (6) indicates that Any path The meta-path attention under the subgraph is the deep neural network represented by semantic attention under the corresponding path After learning, the feature representation of the semantic attention mechanism is (7), use Substitution or or a combination thereof to calculate the cross entropy loss ,in Indicates the number of synonyms and synonym cluster labels, is the cluster label set, For the number The cluster labels of is the node classifier parameter, , by training each attention deep neural network until the cross entropy loss stabilizes to the minimum; Projecting the knowledge parsing results into the heterogeneous graph neural network and calling out all meta-paths. Specific methods include: Q1 builds a mapping interface in the nodes of the heterogeneous graph neural network, and the knowledge parsing results are mapped to the corresponding nodes through the mapping interface, that is, Indicates that the corresponding node includes synonym nodes and / or near-synonymous nodes in different knowledge fields; Q2 searches for all meta-paths containing the corresponding nodes in the heterogeneous graph neural network according to the meta-path defined by the user; The attention mechanism is used to further call out the attention meta-paths whose correlation degree exceeds the first threshold, including: S11 traverses the set of meta-paths belonging to the spatial location and its surroundings All combinations of , It is the combination number, each number represents a different The combination of the meta-paths and the corresponding nodes are expressed by formulas (1)-(7) Substitute the trained Get multiple , where multiple representations are formed Specifically based on The corresponding nodes on the , and then use formulas (1)-(7) to form multiple representations under different combination numbers ; S12 sets the first threshold to ,when (8), call out The corresponding path combination, that is, the meta-path of interest , represents the set of all meta-paths that satisfy formula (8); The method of knowledge evaluation comprises: S13 obtains the space location and time corresponding to the user and the user being followed, and calculates the time according to the meta-path of the user being followed. , find out The corresponding nodes in form corresponding representations ; S14 According to formulas (1) and (2), we can find , let the second threshold be , find satisfaction When the corresponding node is in the meta-path of interest The neighbor set on (9) Draw a spatial distribution map according to the corresponding spatial locations, that is, Distribution, and draw a time distribution diagram in chronological order, that is, Distribution, according to Draw the correlation degree distribution diagram in increasing or decreasing order, that is distributed; S15 distributed, distributed, The location of at least one of the user, the other users, and the concerned user is marked in any one of the distributions or a combination thereof.

2. The system according to claim 1, characterized in that The method for recognizing a word comprises the following steps: The first step is to build a natural language processing model NLP and a speech recognition model V for word recognition in the text and audio formats. The second step is to input the text directly into the natural language processing model to recognize the words, input the audio into the speech recognition model first to form recognized text, and then input the recognized text into the natural language processing model to recognize the words, and put them into the historical knowledge selection list of the user and / or other users in the spatial location where the user is located and the surrounding area. The third step is to select relevant words of interest as keywords based on the historical knowledge selection list of the user and / or other users in and around the spatial location where the user is located, and delete the recognized words. If the selection fails, only the words in the historical knowledge selection list of the user are retained to update the list, and the words related to the failed selection are also used as keywords for the user to select in the knowledge selection module.

3. The system according to claim 2, characterized in that The attention mechanism is used for the historical knowledge selection list of the user and / or the other users to select the keywords for the user and / or to select the keywords for the user and the other users.

4. The system according to claim 3, characterized in that The method of applying the attention mechanism to the historical knowledge selection list of the user and / or the other users specifically includes projecting the inserted word in the historical knowledge selection list into a heterogeneous graph neural network, calling out the user and / or other user-defined meta-path neighbors whose correlation degree exceeds a second threshold; selecting the keyword for the user, and / or selecting the keyword for the user and the other users specifically includes using the inserted word itself and its neighbors as keywords for the user to select in the knowledge selection module.

5. The system according to any one of claims 2 to 4, characterized in that: In the historical knowledge selection list, the words are clustered into synonyms and antonyms, and the level clustering is performed for the proximity of the synonyms.

6. The system according to claim 5, characterized in that The formats of the knowledge document include pictures, videos, documents, and voices; the text includes any one of pictures, videos, documents, and audios or a combination thereof; the audio is derived from video and / or voice; and the knowledge document in the identified format includes pictures and videos, and the user can select the knowledge of interest in the knowledge document by circling the part of interest from the picture and taking a screenshot of the video and circling the part of interest.

7. The system according to claim 1, characterized in that The search for the concerned user corresponding to the concerned meta-path includes: , search for the meta-paths that have been customized historically in the spatial location and its surroundings Any user of .

8. The system according to claim 1, characterized in that The knowledge classification is to manually compare the words in the historical knowledge selection list with the interesting parts circled in history, and classify the interesting parts circled in history into the word cluster in the historical knowledge selection list that is suitable for describing itself. The method of knowledge parsing of words is to directly compare the words with the synonyms and antonyms of the cluster, and obtain the words named in the closest cluster as the recognition result.

Images