Device for monitoring the behavior of a technical system

Abstract

A controller (102) adapted to carry out a method for monitoring the behavior of a technical system (100), the method comprising: Determining (501) embeddings of nodes of a knowledge graph (105) using at least one embedding model, wherein the knowledge graph (105) has nodes and edges between nodes, wherein each node represents one or more components of the technical system (100), one or more control parameters (107) of the technical system (100) and / or one or more objects processed by the technical system (100), and the edges represent roles that the one or more components, the one or more control parameters (107) or the one or more objects have in relation to one another; Determining (502), using the at least one embedding model, an embedding of a query (108) to the knowledge graph (105) with respect to a behavior of the technical system (100), wherein the query (108) is a query for one or more nodes of the knowledge graph (105) and specifies desired roles of nodes of the knowledge graph (105), and wherein the query (108), when viewed as a knowledge graph (105), has, for at least two nodes, multiple paths between the two nodes; wherein the at least one embedding model comprises a neural network trained to merge role embeddings, and determining the embedding of the query (108) comprises determining a role embedding for each of the plurality of paths and merging the role embeddings of the plurality of paths by passing the role embeddings to the neural network; Selecting (503) one or more nodes of the knowledge graph (105) depending on a similarity of the embedding of the query (108) and the embeddings of the nodes of the knowledge graph (105); and Determining (504) a monitoring result (104) of the behavior of the technical system (100) of the one or more selected nodes.

Description

[0001] The present disclosure relates to devices for monitoring a behavior of a technical system.

[0002] The advancement of data processing technologies has led to the development of sophisticated systems for monitoring and controlling technical systems, such as processing assemblies, e.g., manufacturing machines. Conventional methods are often based on relational databases and rule-based systems, whose ability to process complex, interconnected data can be limited.

[0003] Knowledge graphs, consisting of entities (nodes) and relationships (edges), have proven to be a powerful tool for overcoming these challenges. They enable complex interrelationships and efficient data analysis in the context of monitoring and control. The information in a knowledge graph is accessed by querying the knowledge graph. This can be done efficiently by comparing a query embedding with an embedding of knowledge graph components. This involves representing entities and relationships as one or more vectors to preserve semantic meaning, transforming the query into one or more vectors that reflect its logical structure, and using similarity measures to compare query embeddings with knowledge graph embeddings to identify the most relevant answers.

[0004] The publications • [1] Hongyu Ren, Weihua Hu and Jure Leskovec, “Query2box: Reasoning over Knowledge Graphs in Vector Space Using Box Embeddings”, 8th International Conference on Learning Representations, ICLR2020, Addis Ababa, Ethiopia, April 26-30, 2020 • [2] Zhanqiu Hang, Jie Wang, Jiajun Chen, Shuiwang Ji and Feng Wu, “Cone: Cone embeddings for multi-hop reasoning over knowledge graphs”, in Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, and • [3] Hongyu Ren and Jure Leskovec, “Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, describe embedding techniques for querying a knowledge graph as explained above.

[0005] However, embedding methods such as those described in the above-mentioned publications do not allow for the (exact) processing of queries that contain conjunctions, i.e., that, when represented as subgraphs of a knowledge graph, contain parallel paths (i.e., do not have a tree shape). Accordingly, approaches that enable the processing of such queries are desirable, thus improving the possibilities for monitoring and controlling technical systems through a more accurate and reliable use of the information available in a knowledge graph.

[0006] The publication M. Zaheer et al, “Deep Sets”, arXiv:1703.06114v3, 2017, hereinafter referred to as reference [4], describes the neural network operator Deep Sets.

[0007] According to various embodiments, a controller adapted to carry out a method for monitoring the behavior of a technical system is provided, the method comprising • Determining, using at least one embedding model, embeddings of nodes of a knowledge graph, wherein the knowledge graph has nodes and edges between nodes, wherein each node represents one or more components of the technical system, one or more control parameters of the technical system and / or one or more objects processed by the technical system, and the edges represent roles that the one or more components, the one or more control parameters or the one or more objects have in relation to one another • Determining an embedding of a query in the knowledge graph with respect to a behavior of the technical system using the at least one embedding model, wherein the query is a query for one or more nodes of the knowledge graph and specifies desired roles of nodes of the knowledge graph, and wherein the query, when viewed as a knowledge graph, has multiple paths between the two nodes for at least two nodes ◯ wherein the at least one embedding model comprises a neural network trained to merge role embeddings, and determining the embedding of the query comprises determining a role embedding for each of the plurality of paths and merging the role embeddings of the plurality of paths by passing the role embeddings to the neural network • Selecting one or more nodes of the knowledge graph depending on a similarity between the embedding of the query and the embeddings of the nodes of the knowledge graph, and • Determining a monitoring result of the behavior of the technical system of the one or more selected nodes.

[0008] The method described above enables the efficient processing of queries in the context of the behavior of a technical system against a knowledge graph that exhibit conjunctions, i.e., do not have a tree-like form (i.e., are not acyclic), since there are multiple paths between the nodes of the query (when the query itself is represented in graph form). This improves the possibilities for monitoring and controlling technical systems, as more complex relationships between components, control parameters, and objects processed by the system can be considered, because information on role combinations can be retrieved from the knowledge graph with high accuracy.

[0009] Various examples are listed below.

[0010] Example 1 is a method for monitoring the behavior of a technical system as explained above.

[0011] Example 2 is the method of Example 1, wherein a role that a component of the technical system, a control parameter of the technical system, or an object processed by the technical system has with respect to another component of the technical system, another control parameter of the technical system, or another object processed by the technical system is to exhibit a particular behavior when interacting with the other component of the technical system, the other control parameter of the technical system, or the other object processed by the technical system.

[0012] Example 3 is the method from Example 2, where the behavior consists in the interaction leading to a specific result of an operation of the technical system.

[0013] For example, a certain quality of a product produced by the technical system or an action performed by the technical system is achieved, or the technical system has a certain energy consumption, etc.

[0014] Example 4 is the method of any one of Examples 1 to 3 and includes training the at least one embedding model using a loss that includes a penalty for an embedding of a role conjunction that is not within the embeddings of any of the roles.

[0015] In other words, during training of the at least one embedding model (and in particular the neural network), the monotonicity of the embeddings is promoted. This improves the embedding generalization capability of the at least one embedding model.

[0016] Example 5 is the method of any one of Examples 1 to 4 and includes training the at least one embedding model using a loss having a penalty for embedding a conjunction of roles that differs from an intersection of the embeddings of the roles.

[0017] In other words, when training the at least one embedding model (and in particular the neural network), constrained conjunction preservation is encouraged. This improves the embedding generalization capability of the at least one embedding model.

[0018] Example 6 is the method of any of Examples 1 to 5, which comprises controlling the technical system depending on the monitoring result.

[0019] For example, the monitoring results show that a current behavior is undesirable and / or how a desired behavior can be achieved (e.g., by specifying suitable control parameter values), and the technical system is controlled accordingly.

[0020] Example 7 is the method according to any one of examples 1 to 6, which comprises receiving sensor data representing the behavior of the technical system and determining the query depending on the sensor data.

[0021] For example, if sensor data shows undesirable behavior, e.g., if maximum temperatures or pressures are exceeded or a processing result does not meet a quality requirement, the knowledge graph can be queried for a reason for the undesirable behavior and / or an appropriate response to the undesirable behavior.

[0022] Example 8 is a controller adapted to perform a method according to any one of examples 1 to 7.

[0023] Example 9 is a computer program including instructions that, when executed by a computer, cause the computer to perform a method according to any one of Examples 1 to 7.

[0024] Example 10 is a computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform a method according to any one of Examples 1 to 7.

[0025] In the drawings, similar reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being generally placed upon illustrating the principles of the invention. In the following description, various aspects are described with reference to the following drawings, in which: Fig. 1 shows a processing arrangement. Fig. 2 illustrates a computational graph of a query. Fig. 3 illustrates a computational graph of another query. Fig. 4 illustrates a computational graph that exhibits a role conjunction. Fig. Figure 5 shows a flowchart illustrating a method for monitoring the behavior of a technical system.

[0026] The following detailed description refers to the accompanying drawings, which, by way of illustration, show specific details and aspects of this disclosure in which the invention may be practiced. Other aspects may be utilized, and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various aspects of this disclosure are not necessarily mutually exclusive, as some aspects of this disclosure may be combined with one or more other aspects of this disclosure to form new aspects.

[0027] Various examples are described in more detail below.

[0028] Fig. 1 shows a processing arrangement 100 (as an example of a technical system to be monitored and / or controlled) comprising one or more processing devices 102, e.g., robots, manufacturing machines, etc., arranged, e.g., in an order corresponding to a sequence of processing steps.

[0029] The one or more processing devices 102 perform a physical or chemical process, which may include any type of technical processing, such as a manufacturing process (e.g., the manufacture of a product or intermediate product) and / or a machining process (e.g., the machining of a workpiece).

[0030] The processing arrangement 100 comprises a controller 101 adapted, for example, to monitor and control the processing performed by the one or more processing devices 102 by providing control parameters 107 (e.g., process parameters such as temperature, exposure time, force, torque) to the one or more processing devices 102.

[0031] The term "control device" (also referred to as "controller") can be understood as any type of logical implementation unit, which may, for example, comprise a circuit and / or a processor, and which is capable of executing software, firmware, or a combination thereof stored on a storage medium and which can issue instructions, for example to a device for executing a process in the present example. For example, the control device can be configured by program code (e.g., software) to control the operation and / or adjustment (e.g., calibration) of one or more processing devices 102, such as production system components, such as a robot, etc., to perform a specific (e.g., manufacturing and / or processing) task.

[0032] A control parameter may be a parameter that describes an input variable, such as a physical or chemical variable, an applied voltage, an opening of a valve, etc. The control parameter may also be, for example, a process-relevant property of one or more materials, such as hardness, thermal conductivity, electrical conductivity, density, microstructure, macrostructure, chemical composition, etc., and can thus, for example, control which material is processed by the one or more processing devices 102.

[0033] The controller 101 monitors, for example, the operation of the processing arrangement 100. This may in particular include determining a result of the task during or after the execution of a processing task.

[0034] For this purpose, the system 100 may, for example, comprise one or more sensors 103. The sensor(s) 103 may be configured to detect a result of the target task, in particular a physical or chemical process. A result of the process may, for example, be a property of a manufactured product or machined workpiece (e.g., a hardness, a strength, a density, a microstructure, a macrostructure, a chemical composition, etc.), a success or failure of a capability (e.g., picking up an object) of a robot, a resolution of an image captured by a camera, etc. The result of the process may be described using at least one (i.e., exactly one or more) output variables.The one or more sensors 103 may be configured to detect the at least one output variable, and from this, the controller 101 may determine a monitoring result 104. It should be noted that the monitoring result 104 may also include information about how a processing task is performed, e.g., a temperature reached during processing or a specific alarm (e.g., when a processing temperature or pressure is exceeded), etc.

[0035] Based on the monitoring result 104, the controller 101 can set or adjust the control parameters 107, e.g., to ensure that a specific property of a manufactured product is achieved. Depending on the monitoring result 104, the controller 101 can also react to the monitoring result 104 in other ways, e.g., trigger a notification to a user if a malfunction or alarm is detected or a product does not have the required properties, inform a user that a component of a processing device 102 or an entire processing device 102 should be replaced, etc. Furthermore, the controller 101 can react not only to monitoring results 104, but also to other inputs, such as required properties of a processing result, i.e., incorporate these via a user interface or a communication interface.

[0036] To perform monitoring and control decisions, e.g., to select control parameters 107 based on monitoring results and / or required properties or to send a notification signal (e.g., to a user interface), the controller 101 must have information about relationships between entities of the processing arrangement 100 (e.g., machines, products, control parameters, sensor data, etc.).

[0037] An efficient way to collect and make available such information is a knowledge graph 105. The knowledge graph 105 can be stored in a database component (e.g., a software module) of the controller 101 (as in Fig. 1) or stored in a database 106 connected to the controller 101.

[0038] The controller 101 can query the knowledge graph of the database 106 for information about the processing arrangement 100 to obtain information for decision-making (e.g., how to set or adjust control parameters, or whether to send a notification signal). The database 106 responds to a query 108 received from the controller 101. However, depending on the complexity of the query 108, responding to the query 108 by consulting the knowledge graph 105 may not be a trivial task.

[0039] Providing an answer to a complex query 108 using a knowledge graph 105, so-called complex reasoning, involves solving queries by decomposing them into subqueries. To solve this problem, query embedding (QE) methods exist that encode queries with low-dimensional vectors and use neural logical operators to define the embedding of a query as the combination of the embedding of its subqueries. In other words, the database 106 may contain a machine learning model 108 that it uses to encode the query 108 and parts of the knowledge graph 105, compare the query encoding with the knowledge graph encodings, and respond to the query with a comparison result (typically, a part of the knowledge graph 105 whose encoding has the greatest similarity to the encoding of the query 108 is responded to).

[0040] However, conventional query embedding techniques can only handle a limited number of first-level logic queries that have a single unquantified variable (called target), SROI− correspond to description logic queries, and are called tree-form queries because their computational graphs are trees if one considers only the nodes that represent variables.

[0041] Consider the following first-order query ϕ(y) that asks for products (e.g., intermediate products) y that were processed by machines x1, x2 that are involved in a processing O (e.g., appear in a sequence of processing stages) and that, while executing tasks in processing the product, triggered a respective alarm a1, a2 (e.g., alarm of different types).

[0042] This knowledge graph query can be written as follows: Φ(y);;=∃x1, ∃x2:contains (O, x1)∧triggered_a1(x1, y)∧contains (O, x2)∧triggered_a2 (x2, y)

[0043] Fig. 2 illustrates the computational graph 200 of the query ϕ(y).

[0044] The above query example is used below; another example is, for example, the query for a set of control parameters which, when applied to a first machine x1 (which can produce a product O), satisfies a requirement for energy consumption and, when applied to a second machine x2 (which can produce a product O), satisfies a property of the product O.

[0045] The query ϕ(y) in Fig. 2 has a tree form because there is at most one path from x1 to y and at most one path from x2 to y. Since the query has a tree form, it can be expressed as a conjunction Φ1(y)∧Φ2(y) where the subqueries ϕ1(y) and ϕ2(y) also have tree form: Φ1(y);;=∃x1 contains(O, x1)∧triggered_a1(x1, y) Φ2(y);;=∃x2 contains(O, x2)∧triggered_a2(x2, y)

[0046] In SROI− the query ϕ is expressed as C = C1 ⊓ C2, ​​where the subqueries C1 and C2 are: C1;;=∃triggered_a1−.(∃contains−.{O}),Φ2(y) C2;;=∃triggered_a2−.(∃contains−.{O}).

[0047] Complex query answering methods use the embeddings of subqueries ϕ1 and ϕ2 to compute the embedding of query ϕ. To this end, these methods define a neural logical operator representing the logical operation Λ. Therefore, the ability to decompose queries into subqueries and express the logical connectives with neural logical operators is crucial for existing complex reasoning methods.

[0048] However, there are queries where such a decomposition of queries is no longer possible. Consider the query that asks for products manufactured by a machine contained in O that triggered both alarms (i.e., a machine contained in O has both roles: trigger of alarm 1 and trigger of alarm 2). Compared to the previous query, this new query enforces x1 = x2, which can be encoded by renaming the two variables x1 and x2 to x: Ψ(y);;=∃x: contains(O, x)∧triggered_a1(x, y)∧triggered_a2(x, y)

[0049] From the computational graph of the query ψ(y), it is clear that ψ(y) is no longer a tree-like query, since there are two paths from the variable x to the variable y. In the following, such a query type, i.e., queries from the set of queries that result from relaxing the constraint that only one path may exist between nodes (as in tree-like queries), will be referred to as directed graph queries (DAG queries).

[0050] The query ψ(y) cannot be decomposed into two tree-like queries because the conjunction between the query atoms triggered_a1 (x, y) and triggered_a1 (x, y) triggered_a1(x, y)∧triggered_a2(x, y) requires the consideration of two target variables in the subtask on complex reasoning or reasoning. Likewise, the query ψ(y) cannot be SROI− be expressed because SROI− Conjunctions in concept descriptions, but not in role descriptions, which are required to indicate that x "triggered Alarm 1 and Alarm 2 when producing" y. A description logic that allows conjunctions in role descriptions, ALCOIR called, the query (y) can be expressed as the following concept description D: D::=∃(triggered_a1⊓triggered_a2)−.(∃contains-.{O})

[0051] Note that the terms “concept description” and “query” are used synonymously herein, since a concept description defines the answer to a particular query.

[0052] Unlike existing methods, the database 106 for calculating the embedding of the query D, according to various embodiments, does not decompose into two subqueries but calculates an embedding of the relation description triggered_a1⊓released_a2 (i.e., a role conjunction) created with an additional neural operator, a so-called relational combinator, to represent the intersection between relationships. With this extension of existing embedding methods, the database 106 can compare the above-mentioned query D with the Fig. 4 shown calculation graphs.

[0053] Fig. 4 illustrates a computational graph 400 having a role conjunction.

[0054] Like the computational graphs of tree-like queries, the computational graph 400 of query D has a single path from the variable x to the variable y. Therefore, the database can reuse existing query embedding techniques to recursively define the embedding of a DAG query. In contrast, conventional embedding techniques without the extension to embed role conjunctions cannot be applied to DAG queries. Instead, they can be applied by reducing DAG queries to tree-like queries. However, this detour typically leads to less accurate results because the answers to the relaxed query ϕ(y) do not necessarily satisfy x1 = x2, as is the case with the answers to the query ψ(y).

[0055] To define the relational combinator for determining the change of role conjunctions, the embedding model is adapted 108 to contain a set of ALCOIR tautologies fulfilled, which include role conjunctions.

[0056] In the following, queries are presented as ALCOIR concepts. For the following definitions, pairwise disjoint sets C, R, and ε assumed, whose elements are called concept names, role names and individual names.

[0057] Syntax of ALCOIR concept and role descriptions: ALCOIR concept descriptions C and role descriptions R are defined by the following grammar C::=T|A|{a}|¬C|C⊓C|∃R, C R::=T|R−|R⋅R|R⊓R|R+ where the symbol T is a special concept description, and the symbols A, a, and r stand for concept names, individual names, and role names, respectively. Concept descriptions {a} are called nominals. In the following, ⊥, C⊔D, ∀R, C as abbreviations for ¬T, ¬(¬C⊓¬D)or,- ¬∃R.¬C used.

[0058] Syntax of ALCOIR knowledge bases: With two ALCOIR concept descriptions C and D and two role descriptions R and S are the expressions C C⊑D and R⊑S a concept inclusion or a role inclusion. The expression C≡D is used as an abbreviation for two concept inclusions C⊑D and D⊑C used. The expression R≡S is used analogously. For two individual names a and b, a concept description C and a relation description R, the expression C(a) is a concept assignment and the expression R(a,b) is a role assignment. ALCOIR knowledge base is a triple K=(R,T,A), where R a set of role inclusions, T is a set of concept inclusions, and A is a set of concept and role assignments.

[0059] An interpretation I is a tuple (ΔI,.I), where ΔI a lot and .I a function with a domain E∪C∪R which is called the interpretation function and assigns each individual name a ∈ ℇ to an element aI∈ΔI, each concept name A∈C ∈ a set AI⊆ΔI and each role name r∈R a relation rI⊆ΔI×ΔI assigns.

[0060] The interpretation function is recursively applied to ALCOIR concept descriptions and role descriptions by defining the semantics of each connector.

[0061] Semantics of ALCOIR knowledge bases: Given a given interpretation I is I "a model of" • a role axiom R⊑S exactly when RI⊆SI • a conceptual axiom C⊑D exactly when CI⊆DI • a concept assignment C(a) if and only if aI∈CI • a role assignment R(a, b) if and only if (aI, bI)∈RI • one ALCOIR knowledge base K=(R,T,A) exactly when I a model of each element in R∪T∪A is

[0062] Implication: Given two knowledge bases K1 and K2, It is said that the knowledge base K1 implies K2, designated with K1⊨K2, if for each interpretation I, if I K1 modeled, then K1 K2 This definition is extended to axioms and assignments (e.g. K⊨C(a), if all models of K are also models of C(a)).

[0063] According to ALCOIR A knowledge graph G is a ALCOIR knowledge base, whose RBox is empty, whose TBox has a unique concept inclusion T⊑{a1}⊔…⊔{an} which is called the domain closure assumption, where {a1,...,a n} is the set of all individual names occurring in the ABox, and whose ABox contains only role assignments.

[0064] Knowledge graph query answers: Given a knowledge graph G, the answers to a ALCOIR concept description C the individual names a ∈ ℇ, so that G⊨C(a).

[0065] Tree-shaped queries can be SROI concept descriptions The computational graphs of the first-order formulas corresponding to these concept descriptions have at most one path to the target variable for each quantified variable. As explained above, this does not hold when the relational intersection ⊓ is added to queries. In the following, a tree form and DAG are considered as subsets of the ALCOIR description logic defined, its computational graph is described, and the mitigation of non-tree-shaped DAG queries as tree-shaped queries is described in the ALCOIR description logic described.

[0066] Tree-shaped and DAG queries: DAG queries are the subset of ALCOIR concept descriptions C, which are defined by the following grammar C::={a}|¬C|C⊓C|∃R,C R::=r|R−|R⋅R|R⊓R

[0067] A DAG query is said to be tree-like if it does not contain the ⊓ operator in role descriptions.

[0068] As opposed to ALCOIR concept descriptions DAG queries do not contain concept names, the T concept, the transitive closure, or the relationships. These constructors can be excluded because they are not present in queries supported by existing query embeddings.

[0069] For two role descriptions R and S and an individual name a, the following equivalences apply: Commutativity: R⊓S≡S⊓R, Monotonicity: R⊓S⊑R, limited conjunction preservation: ∃(R⊓S).{a}≡∃R.{a}⊓∃S.{a}

[0070] These arise directly from the semantics of ALCOIR concept and role descriptions.

[0071] The following is a formulation for a graph representation of DAG queries (and thus for tree-shaped queries).

[0072] A computational graph is a labeled directed graph Γ = (N, E, λ, τ) such that N is a set whose elements are called nodes,

[0073] E ⊆ N × N is a set whose elements are called edges, λ is a function that assigns a label to each vertex in N, and τ is a distinguished vertex in N called the target.

[0074] For example, the computational graph Γ = (N, E, λ, τ), with N = {u1,u1}, E = {(u1,u2)}, λ = {u1 ↦ {is being processed},u2 ↦ ∃contains -}, and τ = u2 can be represented as (u1 : {is being processed})→(u2 : ∃contains−)

[0075] Intuitively, node u1 computes the concept {is being processed}, and node u2 computes the concept ∃ contains- {is being processed}, which corresponds to the answers to the query about which machines are involved in the processing.

[0076] To define the computational graphs of DAG queries, we define the composition of a computational graph Γ with a role description R, denoted by Γ [R]. Intuitively, the composition is the concatenation of Γ with the graph representing the role description.

[0077] For example, consider the computational graph Γ of (11). The computational graphs Γ [ausgelöst_a1 - ] and Γ [( triggered_a1 - ⊓ triggered_a1 - ] are shown in (12) and (13) respectively. (u1 : {is being processed})→(u2 : ∃contains−)→(u3 : ∃triggered_a1−) (u1 : {is being processed})→(u2 : ∃contains−)→(u3 : ∃triggered_a1−)→(u4 : ∃triggered_a2−)→(u5 : ⊓)

[0078] Consider the tree-shaped query C=C1⊓C2, which is defined by equations (5) and (6). The computational graph of C can be represented as follows: (u1 : {is being processed})→(u2 : ∃contains−)→(u3 : ∃triggered1−)(u5 : {is being processed})→(u6 : ∃contains−)→(u7 : ∃triggered2−)→(u4 : ⊓)

[0079] Similarly, the computational graph for the DAG query in equation (9) is represented by (13).

[0080] Note that in (14) different nodes can have the same label and represent the same concept (e.g., nodes u2 and u6 represent the concept ∃contains -{is being processed}). Intuitively, the duplication of labels means that a response can be a product produced by one machine in the processing that raised alarm a1 and another machine in the processing that raised alarm a2. On the other hand, since there is only a single node in the computational graph in (13) for this concept, namely node u2, the product must have been produced by a machine that raised both alarms a1 and a2.

[0081] Restricted conjunction preservation no longer applies when the nominal {a} is replaced by a general concept description C. In fact, the above example is a counterexample to the generalized version of conjunction preservation. The fact that the conjunction is not generally preserved is the reason why a different neural operator is required for role conjunction than for concept conjunction. Alternatively, if the neural operator is not used, the concept description can be weakened with a weakened version of restricted conjunction preservation:

[0082] Given two role descriptions R and S and a concept description C, the following concept inclusion applies: ∃(R⊓S).C⊑∃RC⊓SC

[0083] Intuitively, the weakening of the role conjunction consists in not assuming that the instances of the concept C on the concept defined on the right-hand side must be equal. An example of this is the above example, where the machine raising alarm a1 and the machine raising alarm a2 need not be the same machine involved in the processing. Thus, the tree-like query with the computational graph in (14) weakens the non-tree-like query with the computational graph in (13).

[0084] The approximated tree-like query of a DAG query Q, denoted by tree(Q), is the tree-like query resulting from removing all conjunctions of role descriptions using the inclusion in (15).

[0085] It is easy to see that for any DAG query Q without a complement constructor (i.e., without ¬), Q ⊓ tree(Q), i.e., the query tree(Q) attenuates the query Q. This is not required for queries with complement, as they are not necessarily monotonic. Therefore, query embeddings that use tree-like queries to predict answers to DAG queries are expected to produce both more false positives and more false negatives.

[0086] Many query embedding methods (see, for example, references [1] to [3]) predict query responses by comparing the embedding of individuals with the embedding of the query. Thus, individuals closer to the query in the embedding space are more likely to receive a response. These query embeddings are learnable parameterized objects and are computed via neural operations corresponding to the logical connectives in the queries.

[0087] The following describes an interface for embedding methods that can serve as a basis for embedding conjunctions. Query embedding methods such as the Query2Box, ConE, and BetaE embedding methods from references [1], [2], and [3] satisfy this interface.

[0088] In the following, three vector spaces E d , R d and Q dwhere E, R and Q are fields (which depends on the query embedding method used) and d is the dimension of the vectors. It is assumed that each individual a is embedded in a vector Emb a ∈ E d embedded, each role name r and its inverse r - in vectors Emb r , Emb r - ∈ R d , and each tree-shaped query Q is embedded in a vector Emb Q ∈ Q d is embedded. While the embedding function Emb is defined for individuals and role names, as well as the inverse of role names, because these are directly defined by the parameters to be learned, the function Emb for composite role and concept descriptions is not directly defined.

[0089] The embedding of a role description R is recursively constructed from the embedding of role names and their inverses as in a neural operator with the signature RComposition: R d × Rd → R d calculated as follows: EmbR∘S::=RComposition(EmbR,EmbS) EmbR−−::=EmbR Emb(R∘S)−::=EmbS−∘R−

[0090] The embedding of a query is calculated from the embedding of individual names and role names using neural operators that represent the logical connectors in queries. The signatures of these neural operators are as follows: Nominal: E d → Q d RelT: Q D × R E → Q E Intersection: Q d × ... Q d → Q d Complement: Q d → Q d These neural operators define the query embedding of tree-shaped queries as follows: Emb{a}::=Nominal(Emba) Emb∃r,C::=RelTr(EmbC) EmbC1⊓C2…⊓Cn::=Intersect(EmbC1,EmbC2,…,EmbCn) Emb¬C::=Complement(EmbC)

[0091] Given a query Q and a knowledge graph G, the goal of query embedding approaches is to maximize the predictions of positive responses to the query Q (i.e., individuals a such that G = Q (a)) and to maximize the prediction of negative responses to the query Q (i.e., individuals b such that G⊨Q(a) ). Due to the open-world semantics of G, it cannot be known which answers are negative. However, learning the query embedding requires negative answers. Therefore, for any positive answer a, query embedding methods typically assume that a random individual b that differs from a is a negative answer.

[0092] In the representation space, the evaluation of how likely it is that an individual is a response to a query is done with a function with the signature Insideness: Q d × Q d→ ℝ, which returns higher numbers for individuals that are answers to the query than for individuals that are not answers to the query. That is, given a query Q with a positive answer a and its corresponding randomly generated negative answer a' that differs from a, the goal of query embedding approaches is to minimize the following loss: Li(Q)::=∑a∈E(−logσ(γ−Insideness(EmbQ,Emb{a}))+∑jk1klogσ(γ−Insideness(EmbQ,Emb{a'}))) where {a'} is the negative sample, γ is a marginal hyperparameter, and k is the number of random negative samples for each positive query-response pair.

[0093] So far, an interface has been described that is provided by neural operators implemented by existing query embeddings. These neural operators enable the computation of tree-shaped query embeddings, but not DAG queries that exhibit the conjunction of roles. To improve the performance of these methods for DAG queries, a relational combination operator with the signature RCombinerk:(Rd)k→Rd introduced, where k is a positive natural number. With this operator, the embedding of a role description R1⊓…⊓Rk (with k > 0): EmbR1⊓…⊓Rk::=RCombinerk(EmbR1,…,EmbRk) where RCombiner is a commutative neural network. For example, the Deep Sets neural operator from reference [4] is used to implement RCombiner. Rcombinerk(EmbR1,…,EmbRk)=∑1≤i≤kαi⋅MLP(EmbRi)) where the (trainable) weights α1, ..., α k add up to 1. In particular, αi=exp(MLP(EmbRi))∑1≤j≤kexp(MLP(EmbRj))

[0094] For two role descriptions R and S the following applies: RCombiner2(EmbR, EmbS)=RCombiner2(EmbS, EmbR)

[0095] This ensures that the embedding of the role description ALCOIR tautologies of commutativity and idempotence. Furthermore, the embedding generalization capability can be improved by promoting query embeddings such that the inference of the embeddings obeys monotonicity and the preservation of restricted conjunctions.

[0096] Monotonicity: According to various embodiments, the embedding model 108 is promoted such that the query embeddings of a DAG query Q=∃(R⊓S) by query embedding the query Q'=∃RC subsume by using the following loss component when training the embedding model 108: Lm(Q)=∑r∈R,s∈RInsideness(EmbQ,EmbQ')

[0097] The insideness function depends on the embedding method used, e.g., embedding methods according to [1], [2] and [3]. For example, in Query2Box, where boxes p i in the embedding vector space are used as embeddings, the interior function or insideness function is defined as follows: Insideness(p1, p2)=BoxVolume(Intersect(p1, p2))BoxVolume(p1)

[0098] Constrained conjunction preservation according to various embodiments, wherein the embedding model 108 is promoted to respect the tautology ∃(r ⊓s).{e}≡∃r.{e} ⊓ ∃s.{e} by using the following loss component when training the embedding model 108: Lr::=Diff(Emb∃(r⊓s).{e},Intersect(Embr.{e},Embs.{e})))

[0099] Diff measures the distance between two query embeddings (depending on the embedding method used, e.g. according to [1], [2] and [3]).

[0100] By including the loss components (28) and (29) in the training loss of the embedding model 108, the monotonicity and restricted conjunction preserving tautologies are encoded into geometric constraints that represent soft constraints over the embedding space. Therefore, the two loss component loss terms (28) and (29) can also be considered as regulatory terms that reduce the embedding search space. For a training dataset, D To answer queries, the training loss (i.e., the optimization objective) for the embedding model 108 is specified, for example, as follows: L(D)::=∑i=1|D|Li(Q)+λ1Lm+λ2Lr where Li(Q) is the loss of the query embedding (i.e., the loss between an embedding output by the embedding model 108 and a target embedding (i.e., a label) of a training example), and λ1 and X2 are the weights of the regularization terms.

[0101] In summary, according to various embodiments, a method is provided as described in Fig. 5 shown.

[0102] Fig. 5 shows a flowchart 500 illustrating a method for monitoring the behavior of a technical system.

[0103] In 501, embeddings of nodes of a knowledge graph are determined using at least one embedding model. The knowledge graph has nodes and edges between nodes, wherein each node represents one or more components of the technical system, one or more control parameters of the technical system, and / or one or more objects processed by the technical system, and the edges represent the roles that the one or more components, the one or more control parameters, or the one or more objects have in relation to one another.

[0104] In 502, an embedding of a query to the knowledge graph regarding a behavior of the technical system is determined using the at least one embedding model. The query is a query for one or more nodes of the knowledge graph and specifies the desired roles of the nodes (for which it performs a query) of the knowledge graph. If the query is considered a knowledge graph, it has multiple paths between the two nodes for at least two nodes (each path has one or more edges—in other words, the query has a conjunction of roles).

[0105] The at least one embedding model comprises a neural network trained to merge role embeddings, and determining the embedding of the query comprises determining a role embedding for each of the plurality of paths and merging the role embeddings of the plurality of paths by passing the role embeddings to the neural network. An example of the neural network is given in Equation (26).

[0106] In 503, one or more nodes of the knowledge graph are selected depending on a similarity between the query embedding and the embeddings of the knowledge graph nodes. For example, a node is selected if the similarity of its embedding to the query embedding is above a predetermined threshold, or a certain number of the most similar nodes are selected.

[0107] In 504, a monitoring result of the behavior of the technical system is determined from the selected node(s). For example, the selected node(s) carry information (or are connected to specific nodes that carry information) that allows the monitoring result to be determined, such as, in the example used above, the result that a specific product has triggered both alarms a1 and a2 on the same machine. A control action, e.g., a safety measure, can then be triggered, such as replacing a processing device or a component, stopping a processing device, notifying a human operator, etc. For example, a combination of the two alarms a1 and a2 indicates that a product may have a quality defect or is even dangerous and will be rejected.

[0108] The Fig. The method illustrated in Figure 5 can thus be used to monitor the behavior of a processing arrangement and, in response to a monitoring result, in particular information about how control parameters influence the behavior, generate a control signal for controlling a technical system of the processing arrangement, for example a computer-controlled machine, such as a robot, a vehicle, a power tool, a manufacturing machine or an access control system.

[0109] Various embodiments may receive and use image data (i.e., digital images) from various visual sensors (cameras) such as video, radar, LiDAR, ultrasound, thermal imaging, motion, sonar, etc., to monitor the technical system. For example, the query may depend on collected sensor information; e.g., a query could be how to react when the temperature and pressure of a product on the same machine have exceeded their respective thresholds.

[0110] The procedure from Fig.3 may be executed by one or more data processing devices (e.g., computers or microcontrollers) having one or more data processing units. The term “data processing unit” may be understood to mean any type of entity that enables the processing of data or signals. For example, the data or signals may be processed according to at least one (i.e., one or more) specific functions performed by the data processing unit. A data processing unit may include or consist of an analog circuit, a digital circuit, a logic circuit, a microprocessor, a microcontroller, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or any combination thereof.Any other means for implementing the respective functions described in more detail herein may also be understood as a data processing unit or logical circuit arrangement. One or more of the method steps described in more detail herein may be performed (e.g., implemented) by a data processing unit through one or more specific functions performed by the data processing unit.

[0111] Accordingly, according to one embodiment, the method is computer-implemented. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Hongyu Ren, Weihua Hu, and Jure Leskovec, “Query2box: Reasoning over Knowledge Graphs in Vector Space Using Box Embeddings,” 8th International Conference on Learning Representations, ICLR2020, Addis Ababa, Ethiopia, April 26–30, 2020

[0004] Zhanqiu Hang, Jie Wang, Jiajun Chen, Shuiwang Ji, and Feng Wu, “Cone: Cone embeddings for multi-hop reasoning over knowledge graphs,” in Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6–14, 2021

[0004] Hongyu Ren and Jure Leskovec, “Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs,” in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020

[0004] M. Zaheer et al, “Deep Sets”, arXiv:1703.06114v3, 2017

[0006]

Claims

[1] Controller (102) adapted to carry out a method for monitoring the behavior of a technical system (100), the method comprising: Determining (501) embeddings of nodes of a knowledge graph (105) using at least one embedding model, wherein the knowledge graph (105) has nodes and edges between nodes, wherein each node represents one or more components of the technical system (100), one or more control parameters (107) of the technical system (100) and / or one or more objects processed by the technical system (100), and the edges represent roles that the one or more components, the one or more control parameters (107) or the one or more objects have in relation to one another; Determining (502), using the at least one embedding model, an embedding of a query (108) to the knowledge graph (105) with respect to a behavior of the technical system (100), wherein the query (108) is a query for one or more nodes of the knowledge graph (105) and specifies desired roles of nodes of the knowledge graph (105), and wherein the query (108), when viewed as a knowledge graph (105), has, for at least two nodes, multiple paths between the two nodes; wherein the at least one embedding model comprises a neural network trained to merge role embeddings, and determining the embedding of the query (108) comprises determining a role embedding for each of the plurality of paths and merging the role embeddings of the plurality of paths by passing the role embeddings to the neural network; Selecting (503) one or more nodes of the knowledge graph (105) depending on a similarity of the embedding of the query (108) and the embeddings of the nodes of the knowledge graph (105); and Determining (504) a monitoring result (104) of the behavior of the technical system (100) of the one or more selected nodes. [2] Controller according to claim 1, wherein a role that a component of the technical system (100), a control parameter of the technical system (100) or an object processed by the technical system (100) has in relation to another component of the technical system (100), another control parameter of the technical system (100) or another object processed by the technical system (100) consists in that it has a certain behavior when interacting with the other component of the technical system (100), the other control parameter of the technical system (100) or the other object processed by the technical system (100). [3] Controller according to claim 2, wherein the behavior consists in that the interaction leads to a certain result of an operation of the technical system (100). [4] The controller of any one of claims 1 to 3, comprising training the at least one embedding model using a loss including a penalty for an embedding of a role conjunction that is not within the embeddings of any of the roles. [5] The controller of any one of claims 1 to 4, comprising training the at least one embedding model using a loss comprising a penalty for embedding a conjunction of roles that differs from an intersection of the embeddings of the roles. [6] Control according to one of claims 1 to 5, comprising controlling the technical system (100) in dependence on the monitoring result (104). [7] Controller according to one of claims 1 to 6, comprising receiving sensor data representing the behavior of the technical system (100), wherein the query (108) is determined as a function of the sensor data.

Images