Industrial IoT equipment collaborative management and control system based on data and knowledge drive

Abstract

The present invention relates to a data- and knowledge-driven collaborative management and control system for industrial Internet of Things devices, comprising Internet of Things devices, a knowledge acquisition module, a knowledge parsing module, an input module, an evaluation module, and a display module. A composite heterogeneous graph network is constructed through the knowledge parsing module, including a heterogeneous graph network G=(V, E) based on real space locations, a knowledge heterogeneous graph network H=(K, F) based on V, and an Internet of Things device heterogeneous graph network I=(U, W) based on V. An attention mechanism represented by a deep neural network model is established, and collaborative management of all knowledge from the whole to the part, from geographic space locations to equipment and natural and social sciences, is obtained, so that each knowledge subject participates in the collaborative management of Internet of Things devices, and the Internet of Things is expanded in terms of time, space, and the depth of logical relationships, thereby reflecting the history and development of knowledge.

Description

Technical Field

[0001] The present invention relates to a collaborative management and control system for Internet of Things devices, and in particular to a collaborative management and control system for industrial Internet of Things devices based on data and knowledge drive, and belongs to the field of Internet of Things management. Background Art

[0002] The existing Internet of Things management system generates online evaluation and decision-making through model construction and data mining, thereby realizing the collaborative management of the Internet of Things and equipment. However, it only focuses on the indicators of the equipment and responds to environmental parameters. Its essence is an operation and maintenance system, which cannot reflect the role of knowledge itself in the learning and development of knowledge in the equipment. The learning and development of knowledge, including the progress of research and study, requires the use of equipment knowledge, relevant scientific history and status, knowledge creators, disseminators, and learners as big data. Only from the indicators and environment of the equipment itself, it is only a contribution to the knowledge of equipment use. In addition, the existing technology has not proposed how the specific model is constructed, the principle of data mining of the model, and the decision-making aspect have not been analyzed in detail. Moreover, the collaborative management of equipment only relies on the overall equipment collaboration, and does not consider the local collaboration of the knowledge generation source of the equipment, and the local collaboration based on the correlation, so it cannot reflect the role and changes of the fields where different knowledge generation sources are located in the generation and development of knowledge, and the role of equipment in knowledge.

[0003] Therefore, how to associate devices with knowledge becomes the fundamental problem of the Internet of Things, and the knowledge data in this application is a heterogeneous data, that is, the knowledge data is closely related to its creator, disseminator, learner, and user, forming a complex heterogeneous network. Therefore, the key to solving the fundamental problem is to solve the distribution problem of devices and knowledge in time and space, especially the spatial and temporal distribution problem of the degree of association between knowledge and devices. The heterogeneous graph network is a graph form that is most suitable for heterogeneous data organization. Through the attention mechanism, each node in the graph can be analyzed to study the relationship between them. This expands the Internet of Things in terms of time, space, and the depth of logical relationships, rather than just the problem of device indicators and environmental parameters. With this extension, the problem of device indicators and environmental parameters is only a component of its application, and naturally, device indicators and environmental parameters can also be discovered and monitored through distribution. It is a direction for collaborative management of Internet of Things devices. Summary of the invention

[0004] In view of the above problems in the prior art, the present invention will develop the following two key points around the heterogeneous graph network. First, on the basis of constructing a heterogeneous graph network of knowledge generation sources, knowledge itself and the interactive traversal between them, the device Internet of Things is additionally introduced to construct a device heterogeneous graph network to realize the interactive traversal of the three major knowledge data of knowledge generation sources, knowledge and devices. Second, use points to lead lines, lines to lead surfaces, and use nodes, paths, sub-paths, etc. to realize the interactive traversal of the three major knowledge data of knowledge generation sources, knowledge and devices. Figure 3 The heterogeneous data associations between knowledge data are studied at different levels, so as to realize the collaborative management of IoT devices as a whole and realize the additional collaborative management effects of associating knowledge generation sources and knowledge.

[0005] Based on the considerations of the above solutions, the present invention provides a data- and knowledge-driven collaborative management and control system for industrial IoT devices, including IoT devices;

[0006] A knowledge acquisition module is used to acquire knowledge from a knowledge generation source and record the spatial location and generation time of knowledge generation. The knowledge includes knowledge related to the device, specifically including the name of the device, the knowledge generation source and / or user to which it belongs, the purpose, technical principle, usage record, historical and real-time indicators and environmental parameters related to the device, device management information, as well as scientific and technological resources, social and humanities resources, specifically including literature, pictures, videos, and audio;

[0007] A knowledge parsing module, connected to the knowledge acquisition module, is used to construct a Complex Heterogeneous Graph Neural Networks (CHGNN), and use the knowledge and spatial locations input by the received knowledge generation source to construct a Complex Heterogeneous Graph Neural Networks Based Attention Mechanism (CHGNNBAM), and after the construction is completed, the knowledge context is retrieved again based on the new input (in most cases, the new input is a search operation, that is, the relevant knowledge input is searched for any relevant knowledge information of interest, or there are other purposes of use, learning, and organization), and the analysis result of the attention mechanism is given based on the knowledge context, and the CHGNN and CHGNNBAM are updated within a predetermined time period;

[0008] The input module is connected to the computer network and is used to receive the newly input knowledge of the user, and input it into the knowledge analysis module through the computer network. The knowledge is clustered into a preset knowledge field with a field label. When the user newly inputs knowledge, the knowledge field needs to be selected for the knowledge, and the field label of the knowledge is selected.

[0009] The evaluation module is used to conduct a comprehensive analysis of the retrieved knowledge context and the analysis results of the attention mechanism, and give a final evaluation that is suitable for the user's true search intention, and the display module is used to display the analysis results of the knowledge context, the attention mechanism and the comprehensive analysis results and operate the visualized CHGNN; wherein,

[0010] The knowledge generation source periodically inputs the knowledge generated according to the confidentiality classification into the knowledge acquisition module, and transmits it to the knowledge analysis module for the update.

[0011] Optionally, the knowledge generation sources include: administrative departments, enterprises, institutions, companies, school research institutions, social groups and organizations, and individuals.

[0012] It is easy to understand that the purpose includes the knowledge field and equipment functions used by the equipment, the usage records include users, usage periods, historical and real-time indicators and environmental parameters include historical and real-time technical parameters of equipment operation, and physical parameters of the equipment operating environment, equipment management information, including information on equipment purchase, operation and maintenance, update, borrowing, and relocation.

[0013] Optionally, the knowledge acquisition module parses the IP address or obtains the spatial location based on the name and address of the input knowledge generation source. The generation time is obtained by recording the time of receiving the knowledge through the knowledge acquisition module. The knowledge is reviewed manually or by computer to decide whether to adopt the knowledge and record the spatial location and generation time. The operation includes dragging the visualized CHGNN, clicking on it to pop up a dialog box or search bar to search and / or view, modify, delete and / or restore the knowledge.

[0014] The method of constructing a composite heterogeneous graph neural network and using the knowledge and spatial locations input by the received knowledge generation source to construct an attention mechanism based on the composite heterogeneous graph neural network includes:

[0015] S1 establishes a geographical map, marks the spatial locations on the geographical map as nodes of the heterogeneous graph neural network, and constructs a heterogeneous graph network based on real spatial locations , forming a heterogeneous graph network neural network of real space locations, where It is a collection of nodes. Once recorded, it cannot be dragged or modified, but can be deleted and restored. is a set of spatial edges. Spatial edges represent the spatial relationship between different nodes.

[0016] That is, with the help of geographic maps, the spatial location of IoT devices, knowledge, and knowledge generation sources are defined, thus laying the foundation for spatial distribution and providing a big data foundation for the research on knowledge creation, dissemination, learning, and use. However, the existing heterogeneous graph neural network is only a node in logical abstraction, and does not correspond to the actual geographical location of the node. Through the real geographical location, that is, the correspondence of the spatial location, it is possible to carry out collaborative association from the trinity of space, time, and degree of association, so that the mathematical model is clear and the principle is clear. The source of knowledge generation can be either the user or the user, so the system increases the degree of participation of various knowledge subjects in system updates and improvements.

[0017] S2 is built on Knowledge Heterogeneous Graph Networks , is a collection of knowledge nodes in different knowledge fields. is a knowledge edge set. Knowledge edges represent the relationship between knowledge in different knowledge fields or in the same knowledge field, and are constructed based on Heterogeneous Graph Network of IoT Devices , is a collection of device nodes representing a single device or a collection of devices. is a device edge set. The device edge represents the association between devices or device sets belonging to the same knowledge domain or similar knowledge domain. This creates a data structure foundation for collaborative management between devices or device sets owned by any user.

[0018] It is easy to understand that for large equipment, a single device is often represented as a node, such as a multi-axis CNC machine tool, an electron microscope, etc. For completing a project or facing a class of problems, the equipment required or related to each other is represented as a node by a collection of equipment, such as the collection of equipment owned by the R&D department of an organization or the research group of a scientific research institution, or the collection of all household appliances of a household, the collection of chemical utensils and analytical chemistry equipment, etc. The IoT device can also be a smart mobile terminal, such as a smart phone, smart bracelet, smart watch, etc. These terminals are Real-time motion or stillness, then generated in Heterogeneous Graph Network of IoT Devices A collection of nodes For smartphones, this is usually the geographical or administrative area bound to the SIM card, or it can be the real-time geographical location of the holder or owner publicly available online. That is, the reversible movement trajectory of the smartphone holder or owner. In the present invention, it is not necessarily a fixed spatial location, but can also be a node containing variable motion. After visualization, Visualize movement or stillness.

[0019] S3 , , Construct meta-paths I separately: 、Meta Path II: 、Meta Path III: 、Traversing Meta-Path IV: And define the neighbors of the meta-path, that is, for any node , then connected by meta-path All nodes of are neighbors, where , , , , , is a natural number, and , is the spatial relationship of the nodes, each of which is a double-arrow vector representing an adjacent node in the meta-path I on the geographic map; is the relationship between knowledge, each of which is a double arrow representing the adjacent nodes in the meta-path II in the knowledge field of scientific and technological resources and social and humanities resources (indicating that the knowledge fields are interconnected), is the association between IoT devices, each of which is a double arrow representing the adjacent nodes in the meta-path III in the device-related knowledge domain (indicating that the knowledge domains to which the devices belong are related to each other);

[0020] For the crossing metapath IV neighbors, define the crossing neighbors. , , Select the focus node from one of the ,or ,or ,and ,set up and The value range is less than or equal to Greater than 0, set ,but ,in, for The complement of the range of To focus on the traversal relationship between nodes and traversal neighbors and between traversal neighbors (local traversal and remote traversal), it is expressed as and The double arrows between form a crossing edge, where ,and , ,but , for The value range complement of , in this case, the traversing metapath IV is called a full traversing metapath, and the non-full traversing metapath is called a partial traversing metapath. That is, the nodes in the full traversing metapath are distributed in all three heterogeneous graph networks; the nodes in the partial traversing metapath are partially in any two of the at most three heterogeneous graph networks.

[0021] For equipment , its neighbors can be divided according to the property rights attributes of the equipment and knowledge generation source, the usage attributes, and the relationship between partners. The traversal path reflects the relationship between the device itself and knowledge and spatial location.

[0022] S4 First , , In the above, attention mechanisms are established according to meta-path I, meta-path II, and meta-path III respectively. First, the node attention mechanism is constructed, then the semantic attention mechanism is constructed, and finally the sub-graph attention mechanism is constructed.

[0023] Secondly, based on the traversal metapath IV, , , Establish a cross-attention mechanism between , , One of them selects the node to focus on, and builds the node attention mechanism first, then the semantic attention mechanism, and finally the subgraph attention mechanism according to the meta-path traversal, and then selects , , The nodes of each of the remaining two are taken as the focus nodes, and the node attention mechanism is first constructed according to the traversing meta-path, and then the semantic attention mechanism is constructed, and finally the CHGNNBAM is obtained, where the number of nodes of meta-path I, meta-path II, meta-path III, and traversing meta-path IV can be adjusted between 3 and 20.

[0024] Optionally, the node attention mechanism construction method is as follows:

[0025] S4-1 For the node pairs connected by meta-path I, meta-path II, meta-path III, and traversal path IV ,for Meta-path I, meta-path II, meta-path III, and traversal path IV are used to construct four types of node attention respectively. , , , (1), , , , They are four types of deep neural networks with node attention. , , , They are respectively, Metapath I, Metapath II, Metapath III, Nodes when crossing path IV The representation of which, For Node Select as focus node The expression,

[0026] S4-2 Compute Node All meta-path based Neighbors The following formula (2) gives the unified expression of the normalized coefficients of the attention of the four types of nodes in formula (1): (2), is the concatenation operator represented by, Metapath The node attention vector superscript represents transpose, For Node The number of neighbor nodes of is the activation function, , and Node The unified expression of attention in four types of nodes, as well as the unified representation,

[0027] S4-3 Compute Node Based on four types of meta-paths The unified representation of features , and finally repeat the attention mechanism times, get (3), Indicates that the attention mechanism is repeated Splicing is performed again;

[0028] The semantic attention mechanism is constructed as follows:

[0029] S4-4 Calculate the weights of four types of meta-paths (4) Among them represents the mode of the corresponding node set, in the fourth formula Determined based on the specific node set to which the selected focus node belongs (if the node belongs to any two or more of the three, that is, including the module of the union of the nodes), with superscripts and subscripts , with subscript , are the corresponding semantic attention vector, weight, and bias representation, respectively.

[0030] S4-5 For In the corresponding subgraph, the calculation node All based on each meta path The following formula (5) gives the unified expression of the normalization coefficients of the four types of semantic attention: (5), where is the unified expression of formula (4),

[0031] S4-6 Constructing four types of meta-path attention (6) means that for any path The meta-path attention under the corresponding path is represented as Deep Neural Networks via Semantic Attention After learning, the characteristics of the semantic attention mechanism are uniformly expressed as (7);

[0032] The subgraph attention mechanism is constructed as follows:

[0033] S4-7 is the same as the node attention mechanism and the semantic attention mechanism. For each subgraph corresponding to the knowledge domain to which the knowledge input by the knowledge generation source belongs (that is, the subgraph set composed of nodes in each domain), constructing the subgraph attention mechanism involves the following corresponding formulas (8)-(11):

[0034] (8), is the subgraph association degree, is a deep neural network for subgraph attention,

[0035] (9), is the subgraph weight, with superscripts and subscripts , , are the corresponding sub-graph attention vector, weight, and bias representation, respectively.

[0036] (10)

[0037] (11), is a unified representation of the sub-graph attention mechanism.

[0038] It can be seen that for When the equal sign is obtained, it corresponds to the overall equipment collaborative management based on the whole map. When it corresponds to local collaborative management, the meta-path and sub-graph can be constructed through four types of meta-paths, so as to obtain the importance and relevance of the prediction in the local sub-graph through the corresponding attention mechanism, so as to know the importance or relevance of each node in the neighbor, meta-path, and sub-graph, that is, the status. In this way, the source of knowledge, the knowledge itself, and the related IoT devices are completely included in a composite heterogeneous graph network for collaborative management.

[0039] It should be noted that , The solution is obtained through the corresponding degree of association and attention mechanism, and the optimized network parameters are obtained through training through the following steps.

[0040] use Substitution , , , , , Any one or a combination of them, i.e., multi-layer sensing mechanism, , , ( is a real number) to calculate the cross entropy loss ,in represents the number of the field label, is a set of domain labels, For the number The field label, is the node classifier parameter, , by training each attention deep neural network until the cross entropy loss stabilizes and minimizes, we get CHGNNBAM.

[0041] because It belongs to the subgraph attention mechanism representation, which contains the components of the semantic (fifth item) and node (first four items) attention mechanisms below it, or in other words, it comes from the semantic and node attention mechanisms. Therefore, there may be a certain degree of deviation when different attention mechanisms are substituted. Therefore, there is an adjustment of the combination coefficient. When only one of the attention mechanisms is substituted, although it does not hinder training, it will deviate from the attention mechanism trained by the representation of a better corresponding level. Therefore, different combinations are selected according to different accuracy requirements, that is, the adjustment of the combination coefficient. Generally speaking Each item of the combined attention mechanism represents the prediction result of the degree of synergy correlation of the attention mechanism at the corresponding level. More specific meta-paths and sub-graphs have different combination coefficient values. For example, when the meta-path does not include the crossing path, then , when the meta-path selection includes only If the meta-path I in .

[0042] The training methods include:

[0043] S4-8 The knowledge generation source inputs the knowledge generated according to the classification into the input module regularly through the computer network, and the input module transmits the knowledge to the knowledge acquisition module to record the spatial location and generation time of the knowledge generation;

[0044] S4-9 The knowledge parsing module obtains the , , In, and , , The trained attention mechanisms of the four types of meta-paths traversed between them receive the input knowledge and the generated spatial locations, and divide them into training set and validation set, with a ratio of 8-3:3-1. The training set is processed to form node representation, and the domain label is formed according to the knowledge domain input by the knowledge generation source, and the number of meta-path nodes is defined. All the corresponding meta-paths of the knowledge corresponding nodes are obtained, and the corresponding normalization coefficients are predicted according to formulas (1)-(3) and , and the corresponding normalization coefficients are obtained according to formulas (4)-(7) and , and according to formulas (8)-(11), the corresponding subgraph association degree, that is, the normalization coefficient and , and the cross entropy loss values ​​are calculated respectively,

[0045] S4-10 back-propagates the cross entropy loss value to optimize the parameters of each deep neural network, uses the verification set to verify the accuracy, and repeats steps S4-7-S4-9 to optimize the network parameters and verify the accuracy.

[0046] The method of retrieving the knowledge context based on the newly input knowledge and providing the analysis result of the attention mechanism based on the knowledge context is as follows:

[0047] S5 inputs the knowledge into the trained CHGNNBAM to obtain the predicted knowledge domain.

[0048] S6 draws the knowledge context in the predicted knowledge domain according to the corresponding spatial location and generation time of the knowledge, and retrieves the drawn knowledge context.

[0049] S7: marking nodes and / or subgraphs corresponding to the new input knowledge in the drawn knowledge context.

[0050] The predicted knowledge domain described in S5 is specifically based on the knowledge domain to which the domain label with the largest normalization coefficient under each domain label may belong. The method of drawing in S6 is to draw on the geographical map, , Select all nodes and / or subgraphs belonging to the predicted knowledge domain, and sort them according to the chronological order of knowledge generation in the predicted knowledge domain to form a historical context , S7 is marked on the geographical map and visualization , Winning bid.

[0051] Optionally, the knowledge context corresponding to at least one other field among the fields with non-maximum normalization coefficients under each field label is selected for drawing. The visualization is performed under the The nodes are displayed on the corresponding geographical map.

[0052] Optionally, the comprehensive analysis includes statistically analyzing at least one of the following types of distributions: a first distribution on the geographical map in the predicted knowledge domain corresponding to the knowledge and the at least one other domain, i.e. distributed,

[0053] In the visualization The second distribution in distributed,

[0054] In the visualization The third distribution in distributed,

[0055] , , The crossing distribution between distribution, and

[0056] According to the user's knowledge generation source, the corresponding knowledge field of the knowledge generation source is displayed on the geographical map. distributed, distributed, distributed, At least one of the distributions, and

[0057] The historical context of various distributions selected for statistical analysis , that is, in the selected distributions, each belongs to The historical context of knowledge in the nodes The final evaluation adapted to the real meaning of the user's search includes comparing the predicted knowledge domain and the domain labels corresponding to the at least one other domain with the domain labels of the user input knowledge, and taking the comparison result as the final evaluation.

[0058] It is easy to understand that the knowledge domain can be artificially divided into closeness in advance, so as to judge the result of the comparison according to the closeness, whether it is a complete match, a partial match, or a complete mismatch, etc. The final evaluation can also be used to judge whether the user has a misunderstanding of the field he is concerned about, whether the field he is engaged in can be borrowed from other fields, so as to inspire the inspiration of the innovation subject.

[0059] Beneficial Effects

[0060] By constructing a heterogeneous graph network based on real-world locations ,based on Knowledge Heterogeneous Graph Networks ,based on Heterogeneous Graph Network of IoT Devices , a composite heterogeneous graph network, an attention mechanism represented by a deep neural network model, and a collaborative management system for IoT devices based on a composite heterogeneous graph neural network. It is a collaborative management of all knowledge from the whole to the part, from geographic space locations to equipment and natural and social sciences, allowing various knowledge subjects to participate in the collaborative management of IoT devices, expanding the depth of IoT in terms of time, space, and logical relationships, thereby reflecting the history and development of knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Configuration diagram of industrial IoT equipment collaborative management and control system based on data and knowledge drive;

[0062] Figure 2 and A schematic diagram of the sub-graph visualization of , where the north direction is given;

[0063] Figure 3 , , Schematic diagram of the sub-plot visualization, where the north direction is given;

[0064] Figure 4 a- Figure 4 e is the corresponding node for inputting new knowledge in the five-category meta-path III A schematic diagram of and its neighbors, Figure 4 f When the corresponding node of the new knowledge is input Schematic diagram of the and its neighbors;

[0065] Figure 5 A simplified flow chart of methods S1-S3 for constructing CHGNNBAM;

[0066] Figure 6A simplified diagram of the node attention mechanism construction method;

[0067] Figure 7 A simplified diagram of the construction process of the semantic attention mechanism and subgraph attention mechanism;

[0068] Figure 8 A flowchart of a specific method for retrieving a knowledge context based on newly input knowledge and providing an analysis result of an attention mechanism based on the knowledge context;

[0069] Fig. 9 Flowchart of methods for various statistical distributions, historical contexts, and final evaluations adapted to the true meaning of the user's search. DETAILED DESCRIPTION

[0070] Figure 1 As shown, a configuration diagram of the collaborative management and control system of industrial IoT equipment driven by data and knowledge is given, including IoT devices with black rounded corners in a gray background and a knowledge acquisition module, which is used to acquire knowledge from the knowledge generation source and record the spatial location (indicated by the blue arrow) and generation time (indicated by the red arrow) of knowledge generation. The two-way single-sided green arrows between the knowledge generation source and the user and the IoT device indicate the ownership relationship. The knowledge includes the name of the device, the knowledge generation source and user to which it belongs, the purpose related to the device, the technical principle, the usage record, the historical and real-time indicators and environmental parameters, the equipment management information, as well as the scientific and technological resources, the social and humanities resources, including literature, pictures, videos and audio.

[0071] It also includes a knowledge parsing module based on a composite heterogeneous graph neural network attention mechanism (CHGNNBAM) represented by a black rounded box with a gray background, which is connected to the knowledge acquisition module and is used to construct a composite heterogeneous graph neural network CHGNNB (indicated by a black arrow), and construct CHGNNBAM based on the knowledge (indicated by a purple arrow) and spatial location (indicated by a blue single-line arrow) input by the received knowledge generation source by the CHGNNB, and after the construction is completed, the knowledge context is retrieved again based on the knowledge newly input by the user (indicated by a purple one-way arrow) (indicated by a lake blue single-line arrow), and the analysis result of CHGNNBAM is given based on the knowledge context (indicated by a red arrow), and CHGNNB and CHGNNBAM are updated within a predetermined time period (not shown in the figure).

[0072] The input module represented by the black gray box with rounded corners is connected to the computer network (indicated by the lake blue line segment) to receive new knowledge input by the user (indicated by the purple one-way arrow) and input it into the knowledge parsing module through the computer network (indicated by the lake blue arrow).

[0073] The evaluation module represented by the black gray box with rounded corners is used to conduct a comprehensive analysis of the retrieved knowledge context and the analysis results of the attention mechanism, and give a final evaluation (indicated by the black arrow) that is suitable for the user's true search meaning. The display module represented by the black gray box with rounded corners is used to display the analysis results of the knowledge context and the attention mechanism (indicated by the black line arrow) and the results of the comprehensive analysis and the final evaluation (indicated by the black arrow) and to display and operate the visualized CHGNN.

[0074] The knowledge generation source inputs the knowledge generated according to the confidentiality classification into the knowledge acquisition module regularly (1-3 years), and transmits the knowledge analysis module for the update ( Figure 1 not shown).

[0075] Next, we will Figure 1 The methods for implementing the three types of functions represented by the red, blue and green dotted rectangular boxes are described.

[0076] First, if Figure 2-Figure 4 As shown, the method of constructing CHGNN and constructing CHGNNBAM based on the knowledge and spatial location input by the heterogeneous graph neural network to receive the knowledge generation source includes:

[0077] like Figure 2 As shown, S1 establishes a geographical map and places the spatial location on the geographical map. - , , , Calibrate the nodes of the heterogeneous graph neural network and build a heterogeneous graph network based on real space locations ,and Figure 2 An example is shown in Node - , The subgraphs formed form a heterogeneous graph network neural network of real space locations, where For a node collection, is the set of spatial edges.

[0078] At the same time Figure 3 Built from a geographical map , and then proceed to step S2 to construct Knowledge Heterogeneous Graph Networks , Figure 2 The corresponding exemplary display part Sub-graph, is a collection of knowledge nodes in different knowledge fields, is a set of knowledge edges. The subgraph is composed of Figure 3 Shown with the node and For example (for other Figure 2 The same is true for the nodes in the figure), and the corresponding two black nodes and its surroundings are represented by concentric lake blue and grass green dotted circles of different radii, with different degrees of proximity to it. These neighbors of different distances represent knowledge clusters of different degrees of proximity in different fields (based on multi-level subject classification and its branches, or patent IPC or CPC classification as the division standard), specifically, knowledge clusters represented by scientific and technological resources and social humanities resources. Subgraph visualization, for The whole graph can be visualized in the same way. It can be seen that the abstract knowledge heterogeneous graph structure is visualized as concentric circles.

[0079] like Figure 2 and 3 As shown, for heterogeneous graph networks of IoT devices The visualization is also represented by concentric circles of different radii with dashed lines, and it is represented at the node - , , , Nearby, it can be displayed in the display module and the user can drag the mouse and click on the node to pop up a dialog box for searching, viewing and modifying. Graphs can also perform these operations, but nodes - , , , Once recorded, it can be dragged and modified. Figure 4 As shown in a-4d, with 5 red nodes It is represented by multiple concentric circles of different colors (black, purple) with the center of the circle Different distances Neighbors - , where the neighbors on the black dashed circle (Black) Same field, Nearest neighbors, neighbors on the purple dotted circle (Purple) Same field, The next nearest neighbor. Figure 3 Although not shown in the figure, There are multiple other different lake blue and grass green dotted circles in the sub-image Neighbors of (if present on meta-path II).

[0080] like Figure 4As shown in e, there are other lesser neighbors on other concentric circles, as well as other neighbors on black and purple dotted circles (as long as they are on the set meta-path III), which are omitted by three black and red dots respectively. Figure 4 f, if the user inputs new knowledge belonging to The same applies , , The three neighbors are the nearest neighbor, the next nearest neighbor, and the next nearest neighbor. Similarly, Figure 3 In In the subgraph, inputting new knowledge can also belong to non For other nodes, the lake blue and grass green dotted circle visualization effects change, as shown in Figure 4 f and Figure 4 a- Figure 4 Changes between e. Figure 2 and Figure 3 In In the subgraph, at the node - , Nearby, followed by Figure 4 5 out of a-4e A node and its neighbors are represented by five types of meta-paths The subgraph of Figure 4 e) does not show the meta-path edges, i.e. Figure 4 The double-sided arrows in a-4d represent the metapath III (same below): , , ,as well as The order of the single arrows of each node corresponds to the defined meta-path III: In the node, for example middle Corresponding , and so on for the corresponding relationships of nodes in other defined meta-path types.

[0081] In the following S3, it is based on , , Four types of meta-paths are defined. Figure 2 As shown, the node - , The meta-path I is composed of: , the two types of meta-paths II in Figure 3 (with as the starting point), Figure 4 a-4e five-category metapath III, and Figure 3 Two types of traversal meta-paths are defined in IV: (two purple lines), (a lake blue line), and define the neighbors of each type of meta-path. Two types of traversal paths are full traversal paths, and the focus nodes are and The local traversal path is , as well as , and the purple ones are all local crossings, while the lake blue ones For local crossing, and the lake blue To travel to a different place.

[0082] For example and Different signal detection equipment in two places , the accuracy is different, in order to obtain more accurate test results, it may be generated Cross-regional traversal, i.e. nodes representing scientific and technological data Equipment in both locations First and last (i.e. purple crossing path , and the lake blue crossing path ) conducted a test experiment (as reflected in a published journal article or other literature represented by experimental records). The device usage information was generated and the network was updated. If other Most users on the node run to and The device with higher precision is used, which means It is expected that the knowledge domain in the subgraph will obtain a better detection result. Objectively, it shows that there are new high requirements for the accuracy of the detection results, which shows the development of knowledge. Therefore, for any crossing path that passes through a place with high equipment accuracy, the attention mechanism in the following text will predict that its correlation with other crossing paths running to places with high accuracy is higher than that of other crossing meta-paths running to places with low equipment accuracy. Therefore, it can be judged that any crossing path that passes through a place with high equipment accuracy belongs to the knowledge domain and the related knowledge evaluation, that is, the statistical distribution will be denser. Figure 5 An outline of the steps S1-S3 is given.

[0083] Then proceed to S4 first and In the paper, attention mechanisms are established according to meta-path I and meta-path II respectively. First, the node attention mechanism is constructed, then the semantic attention mechanism is constructed, and finally the sub-graph attention mechanism is constructed.

[0084] Secondly, based on the traversal metapath IV, , , Establish a cross-attention mechanism between , , One of them selects the node to focus on, and builds the node attention mechanism first according to the meta-path, then the semantic attention mechanism, and finally the sub-graph attention mechanism, and then selects , , The nodes of each of the remaining two are taken as the focus nodes, and the node attention mechanism is first constructed according to the traversal meta-path, and then the semantic attention mechanism is constructed, and finally the CHGNNBAM is obtained.

[0085] like Figure 6 As shown, Figure 1 The node attention mechanism construction method represented in the red dashed rectangle is as follows:

[0086] S4-1 For the node pairs connected by meta-path I, meta-path II, meta-path III, and traversal path IV ,for Meta-path I, meta-path II, meta-path III, and traversal path IV are used to construct four types of node attention respectively. , , , (1), , , , They are four types of deep neural networks with node attention. , , , They are respectively, Metapath I, Metapath II, Metapath III, Nodes when crossing path IV The representation of which, For Node Select as focus node The expression,

[0087] S4-2 Compute Node All meta-path based Neighbors The following formula (2) gives the unified expression of the normalized coefficients of the attention of the four types of nodes in formula (1): (2), is the concatenation operator represented by, Metapath The node attention vector superscript represents transpose, For Node The number of neighbor nodes of is the activation function, , and Node The unified expression of attention in four types of nodes, as well as the unified representation,

[0088] S4-3 Compute Node Based on four types of meta-paths The unified representation of features , and finally repeat the attention mechanism times, get (3), Indicates that the attention mechanism is repeated Splice again.

[0089] like Figure 7 As shown, Figure 1 The construction method of the semantic attention mechanism represented by the red dashed rectangle is as follows:

[0090] S4-4 Calculate the weights of four types of meta-paths (4) Among them represents the mode of the corresponding node set, in the fourth formula Determined according to the specific node set to which the selected focus node belongs, with superscripts and subscripts , with subscript , are the corresponding semantic attention vector, weight, and bias representation, respectively.

[0091] S4-5 For In the corresponding subgraph, the calculation node All based on each meta path The following formula (5) gives the unified expression of the normalization coefficients of the four types of semantic attention: (5), where is the unified expression of formula (4),

[0092] S4-6 Constructing four types of meta-path attention (6) means that for any path The meta-path attention under the corresponding path is represented as Deep Neural Networks via Semantic Attention After learning, the characteristics of the semantic attention mechanism are uniformly expressed as (7);

[0093] The subgraph attention mechanism construction method is as follows:

[0094] S4-7 is the same as the node attention mechanism and the semantic attention mechanism. For each subgraph corresponding to the knowledge domain to which the knowledge input by the knowledge generation source belongs (that is, the subgraph set composed of nodes in each domain), constructing the subgraph attention mechanism involves the following corresponding formulas (8)-(11):

[0095] (8), is the subgraph association degree, is a deep neural network for subgraph attention,

[0096] (9), is the subgraph weight, with superscripts and subscripts , , are the corresponding sub-graph attention vector, weight, and bias representation, respectively.

[0097] (10)

[0098] (11), which is a unified representation of the sub-graph attention mechanism.

[0099] use Substituting into the combined network, that is, the multi-layer induction mechanism, The cross entropy loss is calculated in ,in represents the number of the field label, is a set of domain labels, For the number The field label, is the node classifier parameter, , In order to select within the previous numerical range, each attention deep neural network is trained until the cross entropy loss stabilizes and minimizes, and CHGNNBAM is obtained.

[0100] The training methods include:

[0101] S4-8 The knowledge generation source inputs the knowledge generated according to the classification into the input module regularly through the computer network, and the input module transmits the knowledge to the knowledge acquisition module to record the spatial location and generation time of the knowledge generation;

[0102] S4-9 The knowledge parsing module obtains the , , In, and , , The trained attention mechanisms of the four types of meta-paths traversed between them receive the input knowledge and the generated spatial locations, and divide them into a training set and a validation set with a ratio of 4:1. The training set is processed to form a node representation. According to the knowledge domain input by the knowledge generation source, a domain label is formed, and the number of meta-path nodes is defined. All the corresponding meta-paths of the knowledge corresponding nodes are obtained, and the corresponding normalization coefficients are predicted according to formulas (1)-(3) and , and the corresponding normalization coefficients are obtained according to formulas (4)-(7) and , and according to formulas (8)-(11), the corresponding subgraph association degree, that is, the normalization coefficient and , and the cross entropy loss values ​​are calculated respectively,

[0103] S4-10 back-propagates the cross entropy loss value to optimize the parameters of each deep neural network, uses the verification set to verify the accuracy, and repeats steps S4-7-S4-9 to optimize the network parameters and verify the accuracy.

[0104] like Figure 8 As shown, Figure 1 The blue dotted rectangle box represents the retrieval of the knowledge context based on the newly input knowledge and the analysis result of the attention mechanism based on the knowledge context. The specific method is:

[0105] S5 inputs the knowledge into the trained CHGNNBAM to obtain the predicted knowledge domain.

[0106] S6 draws the knowledge context in the predicted knowledge domain according to the corresponding spatial location and generation time of the knowledge, and retrieves the drawn knowledge context.

[0107] S7: marking nodes and subgraphs corresponding to the new input knowledge in the drawn knowledge context.

[0108] The predicted knowledge domain described in S5 is specifically based on the knowledge domain to which the domain label with the largest normalization coefficient under each domain label may belong. The method of drawing in S6 is to draw on the geographical map, , Select all nodes and subgraphs belonging to the predicted knowledge domain from Figure 1 The chronological order of knowledge generation forms a historical context , S7 is marked as Figure 3 The geographical map and visualization , Winning bid.

[0109] like Fig. 9 As shown, in the predicted knowledge field corresponding to the knowledge and the at least one other field, statistical analysis is performed on the geographical map distributed, distributed, distributed, Then, according to the user's knowledge generation source, the corresponding knowledge field on the geographical map is displayed. distributed, distributed, distributed, , and statistically analyze the historical context of various distributions selected Finally, the predicted knowledge domain and the domain labels corresponding to the at least one other domain are compared with the domain labels of the user input knowledge, and the comparison result is used as the final evaluation. Figure 1-Figure 3 Some of the geographical maps are given in distributed, Figure 2 and 3 Some of the distributed, distribution, and distributed.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. The collaborative management and control system of industrial IoT equipment driven by data and knowledge is characterized by: Including IoT devices, The knowledge acquisition module is used to acquire knowledge from the knowledge generation source and record the spatial location and generation time of the knowledge generation. The knowledge includes the knowledge related to the device, specifically including the name of the device, the knowledge generation source and / or user to which it belongs, the purpose, technical principle, usage record, historical and real-time indicators and environmental parameters related to the device, device management information, as well as scientific and technological resources, social and humanities resources, specifically including literature, pictures, videos, and audio. The knowledge parsing module is connected to the knowledge acquisition module, and is used to construct a composite heterogeneous graph neural network CHGNN, and use the knowledge and spatial locations input by the received knowledge generation source to construct an attention mechanism CHGNNBAM based on the composite heterogeneous graph neural network. After the construction is completed, the knowledge context is retrieved again based on the newly input knowledge, and the analysis result of the attention mechanism is given based on the knowledge context, and CHGNN and CHGNNBAM are updated within a predetermined time period. The input module is connected to the computer network and is used to receive the newly input knowledge of the user, and input it into the knowledge analysis module through the computer network. The knowledge is clustered into a preset knowledge field with a field label. When the user newly inputs knowledge, the knowledge field needs to be selected for the knowledge, and the field label of the knowledge is selected. The evaluation module is used to conduct a comprehensive analysis of the retrieved knowledge context and the analysis results of the attention mechanism, and give a final evaluation that is suitable for the user's true search meaning, and The display module is used to display the analysis results of the knowledge context, attention mechanism and comprehensive analysis and to operate the visualized CHGNN. The knowledge generation source periodically inputs the knowledge generated according to the confidentiality classification into the knowledge acquisition module, and transmits it to the knowledge analysis module for the update; The method of constructing a composite heterogeneous graph neural network and using the knowledge and spatial locations input by the received knowledge generation source to construct an attention mechanism based on the composite heterogeneous graph neural network includes: S1 establishes a geographical map, marks the spatial locations on the geographical map as nodes of the heterogeneous graph neural network, and constructs a heterogeneous graph network based on real spatial locations , forming a heterogeneous graph neural network of real-space locations, where It is a collection of nodes. Once recorded, it cannot be dragged or modified, but can be deleted and restored. is the set of spatial edges; S2 is built on Knowledge Heterogeneous Graph Networks , is a collection of knowledge nodes in different knowledge fields. is a knowledge edge set, and constructs Heterogeneous Graph Network of IoT Devices , is a collection of device nodes representing a single device or a collection of devices. is the device edge collection; S3 , , Construct meta paths in , meta path , meta path , traverse the metapath And define the neighbors of the meta-path, that is, for any node , then connected by meta-path All nodes of are neighbors, where , , , , , is a natural number, and , is the spatial relationship of nodes, each of which represents a meta-path A double-arrow vector of adjacent nodes on the geographic map; is the relationship between knowledge, each of which represents a meta-path The double arrows of adjacent nodes in the knowledge fields of scientific and technological resources and social and humanities resources, is the association between IoT devices, each of which represents a meta-path Double arrows between adjacent nodes in the device-related knowledge domain; For traversal metapath Neighbors define crossing neighbors in , , Select the focus node from one of the ,or ,or ,and ,set up and The value range is less than or equal to Greater than 0, set ,but ,in, for The complement of the range of To focus on the traversal relationship between the node and the traversal neighbor and between the traversal neighbors, it is expressed as and The double arrows between form a crossing edge, where ,and , ,but , for The value range complement of is called a full-traversal metapath, and a non-full-traversal metapath is called a partial-traversal metapath; S4 First , , According to the meta path , meta path , meta path Establish an attention mechanism, first build a node attention mechanism, then build a semantic attention mechanism, and finally build a subgraph attention mechanism. Secondly, based on the traversal metapath ,exist , , Establish a cross-attention mechanism between , , One of them selects the node to focus on, and builds the node attention mechanism first, then the semantic attention mechanism, and finally the subgraph attention mechanism according to the meta-path traversal, and then selects , , The remaining two nodes in each of the two are taken as the focus nodes, and the node attention mechanism is first constructed according to the meta-path, and then the semantic attention mechanism is constructed, and finally the CHGNNBAM is obtained, where the meta-path , meta path , meta path , traverse the metapath The number of nodes can be adjusted between 3 and 20.

2. The system according to claim 1, characterized in that The sources of knowledge generation include: administrative departments, enterprises, institutions, companies, school research institutions, social groups and organizations, and individuals.

3. The system according to claim 1 or 2, characterized in that: The knowledge acquisition module parses the IP address or obtains the spatial location based on the name and address of the input knowledge generation source. The generation time is obtained through the time of receiving the knowledge recorded by the knowledge acquisition module. The knowledge is reviewed manually or by computer to decide whether to adopt the knowledge and record the spatial location and generation time. The operation includes dragging the visualized CHGNN, clicking on it to pop up a dialog box or search bar to search and / or view, modify, delete and / or restore the knowledge.

4. The system according to claim 1, characterized in that The node attention mechanism construction method is as follows: S4-1 For meta-path , meta path , meta path , traverse the metapath Connected node pairs ,for Meta Path , meta path , meta path , traverse the metapath , construct four types of node attention respectively , , , (1), , , , They are four types of deep neural networks with node attention. , , , They are Meta Path , meta path , meta path , Traversing the Meta Path Node at time The representation of which, For Node Select as focus node The expression, S4-2 Compute Node All meta-path based Neighbors The following formula (2) gives the unified expression of the normalized coefficients of the attention of the four types of nodes in formula (1): (2), is the concatenation operator represented by, Metapath The node attention vector superscript represents transpose, For Node The number of neighbor nodes of is the activation function, and Node In the unified expression and unified representation of the attention of four types of nodes, S4-3 Compute Node Based on four types of meta-paths The unified representation of features , and finally repeat the attention mechanism times, get (3), Indicates that the attention mechanism is repeated Splicing is performed again; The semantic attention mechanism is constructed as follows: S4-4 Calculate the weights of four types of meta-paths (4) Among them represents the mode of the corresponding node set, in the fourth formula The node sets of all nodes on the traversal metapath are combined, with superscripts and subscripts. , with subscript , are the corresponding semantic attention vector, weight, and bias representation, respectively. S4-5 For In the corresponding subgraph, the calculation node All based on each meta path The following formula (5) gives the unified expression of the normalization coefficients of the four types of semantic attention: (5), where is the unified expression of formula (4), S4-6 Constructing four types of meta-path attention (6) means that for any path The meta-path attention under the corresponding path Deep Neural Networks via Semantic Attention After learning, the characteristics of the semantic attention mechanism are uniformly expressed as (7); The subgraph attention mechanism is constructed as follows: S4-7 is the same as the node attention mechanism and the semantic attention mechanism. For each subgraph corresponding to the knowledge domain to which the knowledge input by the knowledge generation source belongs, constructing the subgraph attention mechanism involves the following corresponding formulas (8)-(11): (8), is the subgraph association degree, is a deep neural network for subgraph attention, (9), is the subgraph weight, with superscripts and subscripts , , are the corresponding sub-graph attention vector, weight, and bias representation, respectively. (10)， (11), is the unified representation of the subgraph attention mechanism, use Substitution , , , , , Any one or a combination of them, i.e. multi-layer sensing mechanism The cross entropy loss is calculated in ,in , ; represents the number of the field label, is a set of domain labels, For the number The field label, is the node classifier parameter, , by training each attention deep neural network until the cross entropy loss stabilizes and minimizes, we get CHGNNBAM; The training methods include: S4-8 The knowledge generation source inputs the knowledge generated according to the classification into the input module regularly through the computer network, and the input module transmits the knowledge to the knowledge acquisition module to record the spatial location and generation time of the knowledge generation; S4-9 The knowledge parsing module obtains the , , In, and , , The trained attention mechanisms of the four types of meta-paths traversed between receive the input knowledge and the generated spatial locations, divide them into training set and validation set, process the training set to form node representation, form domain labels according to the knowledge domain input by the knowledge generation source, and define the number of meta-path nodes, obtain all the corresponding meta-paths of the knowledge corresponding nodes, and predict the corresponding normalization coefficients according to formulas (1)-(3) and , and the corresponding normalization coefficients are obtained according to formulas (4)-(7) and , and according to formulas (8)-(11), the corresponding subgraph association degree, that is, the normalization coefficient and , and the cross entropy loss values ​​are calculated respectively, S4-10 back-propagates the cross entropy loss value to optimize the parameters of each deep neural network, uses the verification set to verify the accuracy, and repeats steps S4-7-S4-9 to optimize the network parameters and verify the accuracy.

5. The system according to claim 4, characterized in that in, The knowledge context is retrieved based on the newly input knowledge, and the analysis result of the attention mechanism is given based on the knowledge context. The specific method is: S5 inputs the knowledge into the trained CHGNNBAM to obtain the predicted knowledge domain. S6 draws the knowledge context in the predicted knowledge domain according to the corresponding spatial location and generation time of the knowledge, and retrieves the drawn knowledge context. S7 marks nodes and / or subgraphs corresponding to the new input knowledge in the drawn knowledge context; The comprehensive analysis includes statistically analyzing at least one of the following types of distributions: a first distribution on the geographic map in the predicted knowledge domain and at least one other domain corresponding to the knowledge, i.e. distributed, In the visualization The second distribution in distributed, In the visualization The third distribution in distributed, , , The crossing distribution between distribution, and According to the user's knowledge generation source, the corresponding knowledge field of the knowledge generation source is displayed on the geographical map. distributed, distributed, distributed, At least one of the distributions, and The historical context of various distributions selected for statistical analysis , that is, in the selected distributions, each belongs to The historical context of knowledge in the nodes The final evaluation adapted to the real meaning of the user's search includes comparing the predicted knowledge domain and the domain labels corresponding to the at least one other domain with the domain labels of the user input knowledge, and taking the comparison result as the final evaluation.

6. The system according to claim 5, characterized in that in, The predicted knowledge domain described in S5 is specifically based on the knowledge domain to which the largest normalized coefficient under each domain label represents the domain to which the domain label that may be involved belongs. The method of drawing in S6 is to draw the knowledge domain on the geographical map, , Select all nodes and / or subgraphs belonging to the predicted knowledge domain, and sort them according to the chronological order of knowledge generation in the predicted knowledge domain to form a historical context , S7 is marked on the geographical map and visualization , Winning bid.

7. The system according to claim 6, characterized in that Select to draw the knowledge context corresponding to at least one other field among the fields with non-maximum normalization coefficients under each field label, and the visualization is in the field belonging to The nodes are displayed on the corresponding geographical map.

Images