Method and device for providing a recommender system

A shared recommender system using graph neural networks and federated learning addresses privacy and personalization challenges in engineering design by training on collective data and updating with user-specific gradients, enhancing design accuracy and efficiency.

JP2024544681AActive Publication Date: 2024-12-03SIEMENS LNDUSTRY SOFTWARE INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024533140
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-12-03
Publication Date
2024-12-03
Estimated Expiration
2041-12-03

AI Technical Summary

Technical Problem

Existing recommender systems for engineering design processes face challenges due to data privacy concerns, lack of personalization, and the need for collective learning from many users, which is hindered by privacy issues and the impracticality of individually training models for each user.

Method used

A shared recommender system using a graph neural network architecture is deployed on a central server, trained with shared training data, and updated with user-specific gradient information to provide personalized recommendations while maintaining user privacy, utilizing federated learning to adapt to individual user preferences.

Benefits of technology

The system efficiently provides context-sensitive recommendations, enhancing design accuracy and efficiency by learning from collective user data without exposing individual user data, thus addressing privacy concerns and improving design quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024544681000001_ABST
    Figure 2024544681000001_ABST
Patent Text Reader

Abstract

The present invention relates to a computer-implemented method of providing a recommender system (SRS) for the design process of complex systems. The recommender system is shared by multiple users (U1C1, U2C1, U1C2). The complex system comprises multiple connectable components and is designed in a design process by a series of design steps (DS1, DS2). At each design step a partial design (PD) is created until a completed design (CD) is obtained. The partial design of one step and the partial design of a subsequent step differ by a design difference (DELTA) reflecting a difference in at least one element including components or / and component connections. The shared recommender system provides a prediction of the subsequent design difference at each design step.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a computer-implemented method for providing a recommender system for a design process, and further to a corresponding computer program and a recommendation device. [Background technology]

[0002] In industrial applications, engineers are often required to design complex systems or engineering projects that comprise a large number of interconnected components. The design of such systems is usually performed in computer-implemented engineering tools and can be described as an iterative process of identifying interacting components that will satisfy functional requirements resulting from the intended use of the overall system, introducing the identified components into the project, and connecting them together so that the resulting interconnected components enable the intended real-world use.

[0003] Due to the large number of available components and ways to connect them, this process is time consuming and requires technical expertise, domain knowledge, and effort to perform correctly. One way to assist engineers in this process is to integrate recommender systems into engineering tools that suggest suitable and compatible components to be added to engineering projects.

[0004] A recommendation or recommender system can be realized by using a model based on a neural network architecture trained with data from the design process. In such a created model that predicts the next component or components to be added or the next connection or connections, the prediction or recommendation is data-driven and depends on the data available for training. Therefore, it will greatly benefit from learning from the data generated by a large number of its users.

[0005] However, data privacy issues are an obstacle to users agreeing to share their data or usage patterns with the engineering tool provider, i.e., those responsible for designing and maintaining the recommendation system, and with other user groups, e.g. from other companies.

[0006] Currently, many engineering tools suffer from poor user knowledge as a result of users being overwhelmed with menu items that do not fully capture the user's context, such as the current project state or the user's preferences, such as the preferred order of operations when presenting the menu items, to name just a few.

[0007] Considering the complexity of the engineering domain, the most appropriate type of recommender system relies on the concept of collaborative filtering, which requires data about engineering tool usage patterns. Collaborative filtering is a technique in which the unknown preferences of a single user are inferred from the known preferences ("ratings") of a group of users who have a certain number of ratings in common with said single user. Thus, there is no personalization, only guesswork about user preferences.

[0008] Nevertheless, for satisfactory performance, personalizing recommendations according to user preferences is desirable. However, personalization requires individual user data. Engineering recommender systems must learn to recommend appropriate components among hundreds of thousands of items and understand complex relationships between conditions. This requirement requires a lot of training data, which makes training a model for each user individually infeasible.

[0009] Solutions leading to satisfactory recommender systems require collective learning from many users, however, likely spread across multiple organizational units and companies, and privacy concerns preclude the possibility of centralizing multi-user training data and applying routine machine learning training procedures.

[0010] In view of the above, it is an object of the present invention to provide the possibility of improving recommender systems that can be used by a large number of users, and to overcome the disadvantages of individual training, collaborative filtering, or sharing of training data in the context of recommender systems. Summary of the Invention

[0011] This object is achieved by the embodiments disclosed in the independent claims. Advantageous specific embodiments are the subject of the dependent claims.

[0012] According to a first aspect, the present invention relates to a computer-implemented method for providing a recommender system.

[0013] The recommender system is used for a design process and is shared among multiple users. In the design process, which is performed, for example, by using an engineering tool, a complex system, for example an electronic component or a hybrid vehicle, is created in a series of design steps. The complex system may be described by a number of components, for example memory chips and processors, which are at least partially interconnected electrically or inductively.

[0014] In a design step, an intermediate or partial design is achieved by adding one or more elements to the partial design of a previous step. An element consists of at least one component, or at least one connection, or both. The recommender system predicts design or element differences between one design step and a subsequent design step.

[0015] According to an advantageous aspect, this is provided to the user of the recommender system as a context-sensitive menu: if the predictions of the recommender system are good, i.e. technically sound and compatible with the user's requirements, this enhances the design process in terms of speed and quality, since at a certain stage only relevant menu items will be proposed.

[0016] To facilitate good predictions by the recommender system, a recommender system is provided by a computer-implemented method that involves the steps of: A global or shared recommender system is provided at a central server, such as an engineering tool provider's facility or a cloud service, which is global or shared in that it spans a large number of users.

[0017] The shared recommender system encodes the available partial designs, e.g., in the form of a knowledge graph consisting of nodes representing components and links representing connections between the components, e.g., by using a graph neural network architecture, and the result of the encoding is information about the components and their interconnections.

[0018] The global recommender system further provides predictions of subsequent design variations and, for this purpose, is trained using shared training data that influence parameters of the global or shared recommender system. These are meant as "shared training data" in that multiple users have access to these data, e.g., for controlled purposes, and the creator of the data, e.g., an engineering tool provider, has no privacy concerns regarding this sharing.

[0019] Parameters, e.g., weights used in the graph neural network architecture of the shared recommender system or parameters of the graph neural network architecture, are transmitted to users or clients. The users use these transmitted parameters to initialize their version of the shared recommender system. For example, the users may have received a version of the shared recommender system by transmission from a central server to their local equipment or it may be provided to them as a service.

[0020] Users can adapt the shared recommender system to their needs by performing user-specific training with their own special data to obtain a personalized recommender system. Some users send gradient information from this user-specific training to the central server. The gradient information provides information on the evolution of parameters, such as the weight changes used to reduce the error between the prediction of the design difference and the design difference actually chosen, during the user-specific training.

[0021] Providing gradient information that is inconclusive for the training data used has the advantage that a shared model can be updated by using training performed by multiple users, eliminating the need to share training data between them, which can raise privacy issues.

[0022] At the central server, this gradient information is used to update the parameters of the shared recommender model, and this updated shared recommender system is beneficially provided as a new shared recommender system.

[0023] According to an advantageous aspect, these updated parameters are provided again to at least a portion of the users.

[0024] According to another beneficial aspect, the shared recommender system comprises an encoder network, for example consisting of a graph neural network. The encoder network encodes information about the components of the complex system and the connections between them. The shared recommender system further comprises a decoder network, which derives from this information the probability that any design difference will be selected at any design step during the design of the complex system. This has the advantage that such a separation on the user's side only requires adjustment of the decoder parameters, since the underlying encoded information, i.e. the components and their relations, is the same.

[0025] According to another aspect, the invention relates to a computer program which, when executed on a computer, performs the above-mentioned method.

[0026] According to another aspect, the invention relates to a recommendation device on which a computer program is stored and / or provided, for example said recommendation device being connectable by an interface, for example an API, to an engineering tool for the design of complex systems. [Brief description of the drawings]

[0027] Further aspects, features, and advantages of the present invention will become apparent from the following description and dependent claims, taken in conjunction with the accompanying drawings. [Figure 1] An example of a system design process performed in an engineering tool, where a system is constructed through a series of design steps that are decomposed into a set of design deltas that define the operations corresponding to transforming the design of a previous step into the design of a subsequent step. [Diagram 2] We show the training procedure and information flow between a global shared recommender system and individual users' personalized recommender systems. [Diagram 3] 1 illustrates an example architecture for a recommender system model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] One objective of the present invention is to provide a recommender system that can guide engineers during the design of a system to the next component they need. For example, the recommender system may be implemented in an engineering tool, for which a context-sensitive menu is displayed during the design process, the context-sensitive menu suggesting which element should be added next.

[0029] In this context, a system can be anything ranging from a printed circuit board to an autonomous vehicle. These complex systems are composed of a variety of interconnectable components, each with a set of technical characteristics.

[0030] For example, in the case of a memory module, its technical characteristics may include clock frequency, write cycle time, access time, and required voltage supply, and the connections may be realized through various bus systems.

[0031] Software suites, i.e. collections of software available to support the design and / or configuration of complex systems, are offered for various applications such as construction tasks, industrial automation design, or chemical. Examples from Siemens are SimCenter™ and the TIA (totally integrated automation) portal. These tools can be used to create a wide variety of systems ranging from hybrid vehicles and quadcopters to factory automation systems and electronic components / chips. For an efficient engineering and design process, it is important that these tools provide the support required by a particular engineer at a particular stage of a particular project.

[0032] The engineering or design process is accomplished by sequentially selecting components and adding them to an existing system design, where each component may be connected to several other components using various link types, e.g., mechanical, electrical, via a particular bus, etc.

[0033] The recommender system is aware of the current project state and provides a ranked list of suitable components or connections for selection as the next item, for example in a context-sensitive menu. The ranking reflects selectability, with the highest ranked items being the most likely to be selected, i.e., added to the existing system design in the next step.

[0034] Each engineer has its own preferences. This can be reflected in the order of operations. For example, one user may prefer to start with the most central components, while another prefers to start with the peripheral components. Regarding connections between components, one user may prefer to select all components first and then make the appropriate connections, while another user may prefer to select one component and subsequently establish all the necessary links to this component. The recommender system must be able to learn across a large number of users, while also adapting to the personal preferences of each engineer.

[0035] According to an advantageous embodiment of the invention, the following components are used for the implementation of the recommendation system: Global or shared recommender system models A set of personalized recommender system models A training process to update the global system model parameters Training process to personalize the global model to user preferences Sampling procedure for selecting clients for shared model updates

[0036] "Design Process Using Recommender Systems"

[0037] Figure 1 shows an example of a system design process carried out with the aid of an engineering tool.

[0038] A system is a complex object, e.g. a hybrid vehicle or an electronic component, made up of various connectable components that must be used, combined and connected in a way that satisfies the requirements set for the complex object.

[0039] A system is constructed through a series of design steps starting from an initial combination. The design process can be decomposed into a set of design steps that define the operations corresponding to transforming the design of a previous step into the design of a subsequent step.

[0040] In Figure 1, in the first design step DS1 there is only a component "Vehicle" V, which has properties such as mass, number of front and rear wheels, which are represented by its ports and rectangles within the different ports, and which is connected mechanically, electrically or otherwise to other components, which are represented by bounding rectangles. In addition there is a component axle A, which can be connected to the front or rear axle.

[0041] Proceeding from the first design step DS1 to the second design step DS2, one or more elements or connections DELTA(1,2), also called design difference or design delta, are added, in the illustrated example a new element rear axle RA is added and connected to element axle A. From the second design step DS2 further design differences DELTA(2,...) are added, resulting in a subsequent design.

[0042] All these intermediate designs before the complete design CD is achieved are called partial designs PD.

[0043] During the development process, elements are added and connected to the partial design PD at each design step, and after a series of design steps DS···, the completed design CD is obtained at the final design step DS_final.

[0044] The completed design CD is used for the realization of the complex object when the requirements for the complex object, for example the specific performance of an electrical component or part thereof, are met.

[0045] Thus, the term "completed design" CD includes a completed system architecture, e.g., a completed hybrid vehicle or completed electronic components, along with intermediate designs that may be transferred to another user, company, etc., e.g., for further processing.

[0046] The goal of a recommender system is to predict the possible next design delta with sufficient accuracy, which means it should learn from the context, i.e., the current design step, and the user's preferences, to predict the subsequent design delta, i.e., the components and connections to be added.

[0047] "Recommender System Architecture"

[0048] A high-level schematic diagram of the architecture of a recommender system or model is shown in Figure 3. As input data X, the partial design PD and the completed design are used.

[0049] For training, all possible part designs PD consisting of one or more elements in the component catalog CC and the completed designs CD are used as input data X. As output data Y, a ranking of the elements to be added or the design differences DELTA is obtained, i.e. the respective probabilities for each design difference.

[0050] When the recommender system is trained and being used, the input data X is a particular sub-design PD, and the output data can be a ranking of the design differences DELTA that are added to this particular sub-design.

[0051] As an example of input data in Fig. 3, a completed design CD of a hybrid vehicle V is shown as a knowledge graph KG, optionally including attributes ATT for the individual nodes. The hybrid vehicle V is represented by a central node. The knowledge graph KG, with nodes representing elements and links representing connections between elements, and optional attributes ATT, allows a specific system design of a complex system to be described in a permutation invariant manner suitable for use by a graph neural network.

[0052] In the encoder network EN, the representation of the nodes of the knowledge graph KG and their relationships with their neighboring nodes are obtained by feeding the input data X to the graph neural network. First, the input data X is input to the first graph neural network GNN1.

[0053] Input data X is H (0) Also denoted as , which is a representation of the node features and link structure of the data architecture, and the adjacency matrix A ~ Therefore, H (0) contains features or characteristics that refer only to a particular node, such as motor characteristics or available connection types, in other words, everything related to the identity of a particular node within a given complex system. ※A ~ teeth Represents JPEG2024544681000002.jpg88.

[0054] For example, these data may represent a motor with its weight, and possibly electrical or mechanical connections.

[0055] In the first graph neural network GNN1, the features of nodes with one hop distance are encoded in the representation of a particular node. By repeating this process many times, more distance information is taken into account for the encoding of a particular node.

[0056] The output of the first graph neural network GNN1 is a matrix H (1) which becomes the input of the second graph neural network GNN2.

[0057] As mentioned above, the matrix H (1) The value of reflects the first-order correlation between two nodes, i.e., having one edge between them. Therefore, in addition to the node features, the first-order correlations are also included in this matrix H (1) As mentioned above, a first-order correlation has an edge going directly from the source node to the target node, a second-order correlation has an edge going from the source node via a first edge to the interruption code, via a second edge to the target node, etc.

[0058] H as input to the second graph neural network GNN2 (1) By using,the second-order correlation between two nodes, i.e., two nodes that have a node between them, and thus, go through two edges, is the output,H,. (2) This H (2) is a matrix with dimensions #n*, the number of nodes in the graph convolutional neural network, and #LD, the number of latent dimensions. (2) encodes node features and information from nodes one-hop and two-hop distance from the considered node.

[0059] Experiments have shown that considering first-order and second-order relations, i.e., considering relations with nodes that are one or two hops away, produces good results, i.e., the derived indicators reflect reality very well. Depending on the data architecture, in other embodiments, higher-order correlations are also beneficial. The usefulness depends, for example, on the strength of correlation between nodes or the number of connections between a node and other nodes, because at higher orders, node features and information about nearby nodes are smoothed out while more distant distance relations are examined.

[0060] In terms of architecture, the graph neural network may comprise one convolutional layer, or may have a more complex operation, for example including other layers, such as other convolutional layers or other types of layers. The first graph neural network GNN1 and the second graph neural network may differ from each other in architecture and / or training.

[0061] "Standard graph convolution"

[0062] According to an advantageous embodiment, the convolution operation used in either the first or the second graph neural network GNN1, GNN2 is JPEG2024544681000003.jpg1168. H is the representation of the node. l is a running variable that represents the number of latent dimensions in the graph convolutional neural network or the convolutional layer of the graph convolutional network. When l=0, H (0) represents node features, e.g., the number and type of ports or types that can be "components". H is iteratively updated and for values ​​l>0 also represents the relationships between nodes. σ is a sigmoid function used as the activation function for the GNN. The matrix D ~-1 is used for normalization and can be derived from the input and diagonal matrices. ※D ~-1 teeth Represents JPEG2024544681000004.jpg910.

[0063] A ~ is a matrix reflecting the topology of the data structure, e.g., the complete design CD or the partial design PD. For example, A ~ W is an adjacency matrix that describes the connections between one node and another for every node in the graph representation, and thus essentially represents the link structure. (l) are parameters or weights that represent the strength of connections between units in a neural network. The advantage of this convolution operation is that in its basic form, aggregation, i.e., the collection of information related to one particular node, is based on average values.

[0064] Alternatively, other convolution operations can be used that are tailored to a particular problem, for example the design process of an electronic component or compound.

[0065] "Consolidation"

[0066] Thus, the node representation H (1) and H (2) represents the structural identity of each node and its surroundings by encoding its neighboring information. The node representation H (1) and H (2) are concatenated with CC, thus obtaining concatenated data.

[0067] For example, two matrices H (1) and H (2) When stacked, the concatenated data is H (1) Number of columns in and H (2) Therefore, the dimension of the connected data is determined by the number of original nodes in the data architecture, the number of latent dimensions of the first graph neural network GNN1, and the number of dimensions of the second graph neural network GNN2, and how many degrees of correlation are considered, i.e., how many matrices H (l) is added. Using the combined data, decoding is then performed in the decoder neural network DN.

[0068] "Decoding"

[0069] In the decoder neural network DN, the respective probabilities are extracted from the node encoding for each design difference DELTA by using a neural network NN.

[0070] The decoder network can be of several types. One example is a dot product or scalar product decoder, where each partial design is scored using a dot product or scalar product operator for all components in the catalog, followed by a softmax function to obtain a probability. The softmax function converts a vector with numbers as entries into a vector with probabilities as entries. This can be achieved, for example, by using a normalized exponential function.

[0071] The probability assigned to a design delta reflects how likely it is that a particular design delta will be added to a particular part design PD. This probability can be viewed as a function of the part design PD and the design delta.

[0072] "Ranking"

[0073] By classifying or ranking R the design differences DELTA according to their respective probabilities for each partial design PD, for each partial design DELTA as output Y, it is possible to determine the group of design differences DELTA that are most likely to be included in the next design step.

[0074] Thus, only the most relevant design differences can be displayed in the context-sensitive menu of the engineering tool, making the design process more efficient and contributing to avoiding errors.

[0075] In summary, the exemplary architecture of the recommender system includes an encoder network where graph-formatted data is input and encoded, a decoder network that extracts probabilities from the encoded information, and a ranking entity that ranks the design difference deltas according to their probabilities. The exemplary encoder network architecture includes:

[0076] "Training procedures and information flow between a global recommender system and a personalized recommender system"

[0077] As mentioned above, one of the objectives of the present invention is to provide a recommender system that, for a particular design step, suggests elements that are most likely to be added in a subsequent design step. Therefore, the recommender system should learn from the context, i.e., the sub-design PD of the current design and the user's preferences for predicting elements of the subsequent design delta. To achieve this, a combination of centralized and individualized training is proposed, which is described with reference to FIG. 2.

[0078] Figure 2 shows the flow of information between a global recommender system and several personalized recommender systems derived from the global recommender system.

[0079] At the centralized server CS, training and evaluation data T / ED are utilized. The training data is used to train the model, and the evaluation or validation data is data taken from the set of training data to be used to test the hyperparameters of the model. A hyperparameter is a parameter whose value cannot be estimated from the data provided to the model, but is used to manage the learning process. This is, for example, the learning rate when training a neural network.

[0080] Furthermore, the centralized server CS utilizes a component catalog CC, which contains elements that can be added during the design process, i.e., at any partial design PD. For example, this component catalog CC is hosted on the server side and contains information about every item that can be recommended to a user, including technical characteristics (e.g., resistance value of each resistor component, power rating of an electrical component, etc.).

[0081] According to an advantageous embodiment, items of the component catalogue CC are sent to the user together with the shared recommender model or updates thereto.

[0082] These data, the training and evaluation data T / ED, and the component catalog CC, enter the training and evaluation procedure of the global recommender model. A model update MU is performed after training where the original parameters are replaced by parameters derived from the training process.

[0083] The global recommender system model SRS must be able to encode the partial design PD illustrated in Figure 1 and rank the items or elements or design differences DELTA to be added accordingly. Since the system design can be well described by a graph, a graph neural network or other graph learning based approach is suitable.

[0084] The training and evaluation data T / ED used for the training and evaluation procedure T / EP of the shared recommender system SRS is data that can be shared between each user and the company, e.g. because the respective generators of the data have agreed to the sharing, or because the data has been created by simulation, the data has been created for tutorial purposes, etc., i.e., the data contained on the server side is not considered to be user sensitive.

[0085] The global recommender system, or model SRS, uses federated transfer learning, described below, to learn from the experiences of all users without direct exposure to user data.

[0086] The parameters of the global or shared recommender model SRS are sent to each user using the shared recommender model SRS for parameter initialization PI. The users initialize the shared recommender model, i.e., set the parameters to their proposed values. The parameters can be, for example, the weights of individual neurons.

[0087] The shared recommender system SRS thus initialized is used as a starting point for personalization of the shared recommender system SRS by using user-specific training data in the shared model training SMT.

[0088] "Personalized training procedures"

[0089] To personalize the parameters of the shared recommender system (SRS) to each user's preferred working mode, a personalization training procedure PTP is executed based on each user's data UD. The personalization training procedure PTP adapts the initialized parameters obtained from the shared recommender system SRS according to the client's usage data UD obtained, for example, from a previous design process, to obtain a personalized recommender system PRS. Thus, the general strategy and hyperparameters of this training procedure are different from those of the training procedure for the shared recommender system SRS, and the goal here is to optimize the parameters of the shared recommender system according to the user's personal usage data UD, so that the design difference DELTA proposed in the design step meets the user's best needs and preferences.

[0090] In contrast, achieving optimal performance across all users is not the goal of the personalizing training procedure PTP.

[0091] By the personalization training procedure PTP, a personalized recommender model is generated by updating only the decoder network DN model parameters, e.g., the weights used in this neural network NN, while keeping the encoder parameters, e.g., the weights of the first neural network GNN1 and the second neural network GNN2, unchanged. Thus, the probabilities of the design differences DELTA are adapted to change for individual users, and the ranking of the proposed design differences is modified accordingly.

[0092] "Improving shared recommender systems through individual-use server-side shared model training"

[0093] The shared recommender model SRS is updated according to what is learned by the usage data UD of each client. The clients can be a first user U1C1 in a first company, a second user U2C1 in the first company, a first user U1C2 in a second company, etc. Users within a first company may want to use together the data generated by them, but the exchange of data between companies is unlikely.

[0094] In the personalized training process PTP, the gradients of the parameters of the decoding network DN are calculated. The gradients represent the change in all weights with respect to the change in error. The error is expressed as the true result vs. the input / training dataset x i Results obtained by a personalized recommender model for y i The difference between is shown.

[0095] As shown in Figure 2, the calculated user gradients UG are sent to the central server CS. The usage data UD itself is never passed to the global recommender model training, only the gradient information. Thus, the privacy of the users is maintained. Furthermore, when sending only the gradients instead of the set of training data created by the users, the amount of data sent is reduced. In addition, updating the shared recommender model using the gradients requires less computational effort than using a new set of training data.

[0096] On the side of the central server CS, the user gradient information UG sent by each client or user is used to form updates to the model parameters of the shared recommender model in a shared model training procedure SMTP. The recommender loss function is L i (w,x i ,y i ), where w is the set of weights used in the personalized recommender model, e.g., the matrix w jk , x i is the set of input data, i.e., intermediate or partial design PD, y i is the set of results, i.e. the proposed factor or design difference DELTA. The recommender loss function is a function of a set of model parameters w and training examples x i ,y i The recommender loss function can be calculated, for example, by using binary cross-entropy. The total loss over all examples is defined as: where n denotes the number of training examples. JPEG2024544681000005.jpg2031

[0097] The shared recommender model parameter update using gradient descent is defined as follows: JPEG2024544681000006.jpg936In the formula, ∇L denotes the gradient of the loss function. γ is a parameter indicating the learning rate or step size. That is, the weights are changed between step t and step t+1 depending on the magnitude of the scalar product of the parameter γ and the gradient of the loss function. This gradient is in a multidimensional space, and the derivative described by the gradient is the loss function with respect to the model parameters. For example, one entry is the weight w jk ,dL / dw jk , which is the derivative of . Thus, for example, one can find a local minimum of the loss function and determine an appropriate set of parameters.

[0098] For each individual user or client, the gradient ∇L c denotes the gradient computed by one client, c.

[0099] To update the shared recommender system or model SRS parameters, an average is taken across all considered clients. JPEG2024544681000007.jpg1750, N c Let N denote the number of training samples at client c, and N denote the total number of training examples across all clients considered. The weighted average allows users or clients with more training examples to influence the updates more heavily.

[0100] According to another embodiment, the weightings may be individualized, for example weights may be assigned to particular users or groups of users depending on experience, quality of designs, time the engineering tool has been used, etc.

[0101] Depending on the embodiment, either all or a subset of the clients are considered. The advantage of considering all clients is that we get a large number of gradients.

[0102] "Selecting which clients contribute to updates"

[0103] According to another embodiment, each update of the shared recommender model or system SRS is performed by obtaining gradient information only from a subset of clients. Thus, in addition to reducing the effort on the client side, the amount of data transmitted and the computational effort for the update can be reduced. The selection of the subset or user sampling US must be made such that an update of the weights based on the gradient information of one user also improves the shared recommender system SRS.

[0104] When many clients for a particular development tool exist within the same organization, many clients may contain very similar system designs, and large systems are often developed collaboratively by a team of engineers who share the design. It is inefficient to learn from all clients because there may not be any variance in the data across some clients.

[0105] The shared recommender model SRS is used to calculate performance metrics on local client data at each client prior to personalization in order to gather information on when gradient information obtained by local training at the client may affect parameters of the shared recommender model SRS that is distributed to all customers.

[0106] The performance metric measures the accuracy of the prediction, i.e., how accurate the prediction of the design difference is for a particular user or client, in other words, the magnitude of the error in the prediction for a particular client.

[0107] According to an advantageous embodiment, the error E is determined by any part design PD i It is calculated as the sum of the prediction errors for JPEG2024544681000008.jpg1726

[0108] These performance metrics are sent to the server and used in a sampling approach. Clients that are particularly likely to possess gradient information that would improve the performance of the shared model, e.g., gradient information that would reduce the average error of the user, are more likely to be sampled. For example, these are clients with poor performance metrics, i.e., clients for which the predictions of the personalized recommender model are not working satisfactorily.

[0109] An alternative approach is to train a reinforcement learning agent to select the client. According to an advantageous embodiment, the reward is based on a performance metric.

[0110] Alternatively or additionally, the neural network can also be trained to estimate the expected improved gains of the shared recommender model when using client data, this estimation procedure being performed and can be performed only on the client side, thus preserving data privacy.

[0111] Tests have shown that application of a recommender system according to any of the described embodiments can reduce error rates in designs and reduce the time required to complete a design.

[0112] In the context of the present application, the designs generated by using the recommender system are applied to manufacture, for example, new hybrid vehicles, electronic components, etc., or parts thereof, if they meet the requirements of the respective product, for example in terms of functionality. Thus, the efforts in manufacturing can be reduced, since the designs obtained by the engineering tools can be pre-analyzed in relevant aspects.

[0113] The term "recommendation device" may refer to a computer capable of executing instructions. The term "computer" may refer to a distributed set of processing units or services rented from a cloud provider, as well as a local processing unit where a client uses engineering tools for design purposes. Thus, the term "computer" encompasses any electronic device with data processing properties, such as a personal computer, a server, a client, an embedded system, a programmable logic controller (PLC), a handheld computer system, a pocket PC device, a mobile wireless device, a smart phone, a device, or any other communication device capable of processing data using computer support, processors, and other electronic devices for data processing. A computer may comprise one or more processors and memory units and may be part of a computer system. Furthermore, the term computer system includes general purpose or dedicated data processing machines, routers, bridges, switches, etc., which may be standalone, assisted, or embedded.

[0114] The term "user" may refer specifically to an individual, a group of individuals, or a company.

[0115] In the above description, various aspects of the present invention have been described. However, it will be appreciated by those of ordinary skill in the art that the present invention may be practiced with only some or all of the aspects of the present invention. For purposes of explanation, specific configurations have been described to provide a thorough understanding of the present invention. However, it will be apparent to those of ordinary skill in the art that the present invention may be practiced without these specific details. Some of the description has been presented in terms of operations performed by a computer system using terms such as data, status, links, faults, packets, and the like, consistent with the manner in which those of ordinary skill in the art commonly use to convey the substance of their work to others of ordinary skill in the art. As is well understood by those of ordinary skill in the art, these quantities take the form of electrical, magnetic, or optical signals that can be stored, transferred, combined, and otherwise manipulated through the mechanical and electrical components of a computer system.

[0116] Moreover, various operations are described sequentially as multiple discrete steps in a manner that is helpful in understanding the invention. However, the order of description should not be construed to imply that these operations are necessarily order dependent, particularly in terms of the order of their presentation.

[0117] Reference herein to "one embodiment" or "an embodiment" means that the particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

Claims

1. 1. A computer-implemented method for providing a recommender system to a complex system design process, comprising: The recommender system (SRS) is shared by multiple users (U1C1, U2C1, U1C2), said complex system comprising a plurality of connectable components is designed in a design process by a series of design steps (DS1, DS2), in each of which a partial design (PD) is created until a complete design (CD) is obtained; The part design (PD) of one step and the part design (PD) of a subsequent step differ by a design difference (DELTA) that reflects a difference in at least one element including components and / or connections between components; The shared recommender system (SRS) provides a prediction of the subsequent design difference (DELTA) at each of the design steps (DS1, DS2); The method comprises: a) providing, at a central server (CS), the shared recommender system (SRS) that encodes the partial designs (PD) and provides predictions of subsequent design differences (DELTA), the shared recommender system (SRS) being trained with shared training data (T / ED); b) sending parameters of the shared recommender system (SRS) from the central server (CS) to a number of users (U1C1, U2C1, U1C2) to initialize their versions of the shared recommender system (SRS); c) receiving, at the central server (CS), from a subset of users (U1C1, U1C2), gradient information obtained by a user-specific training (PTP) of a user's version of the shared recommender system (SRS) using user-specific training data (UD) to obtain a personalized recommender system (PRS), the gradient information being indicative of the evolution of the error of the prediction depending on the parameter (w); d) updating, at the central server (CS), at least one parameter of the shared recommender system (SRS) using the received gradient information; A method comprising:

2. The method of claim 1 , further comprising: e) transmitting updated parameters of the shared recommender system (SRS) from the central server (CS) to a plurality of users (U1C1, U2C1, U1C2).

3. 3. The method according to claim 1 or 2, wherein the shared training data (T / ED) used in the central server (CS) can be shared among individual users (U1C1, U2C1, U1C2) and / or the user-specific training data (UD) cannot be shared among all individual users (U1C1, U2C1, U1C2).

4. The shared recommender system (SRS) includes an encoder network (EN) and a decoder network (DN), the encoder network (EN) encodes information about components and connections of the complex system; The decoder network (DN) extracts from the information the probability that a particular design difference (DELTA) is selected at a particular design step (DS1, DS2, DS...), The method according to any one of claims 1 to 3, wherein the training in the central server (CS) comprises training the encoder network (EN) and the decoder network (DN).

5. The method of claim 4, wherein in the user-specific training (PTP), only parameters of the decoder network (DN) are trained, from which the gradient information is derived.

6. 6. The method according to claim 4 or 5, wherein in the updating step d) only the parameters of the decoder network (DN) are updated, the parameters of the encoder network (EN) formed in particular by a graph neural network are not changed.

7. b1) receiving performance metrics from users (U1C1, U1C2) representing information regarding usage of the personalized recommender system (PRS) by a particular user; A subset (U1C1, U1C2) of users receiving the gradient information in step c) Differences in the design of the complex system between individual users within a group of users (U1C1, U2C1); The number of training samples used by users (U1C1, U2C, U1C2), The accuracy of predictions of the shared recommender system (SRS) after initialization and before a personalized training procedure (PTP); - the accuracy of predictions of the personalized recommender system (PRS) after the personalized training procedure (PTP); The method of any one of claims 1 to 6, wherein the performance metric is determined based on a performance metric that depends on at least one of:

8. The method of claim 7 , wherein a reinforcement learning agent is trained to select a subset of users to which the gradient information is sent, and a reward in that training procedure is based on the performance metric.

9. The gradient information is calculated for each loss function L i The loss function L is calculated based on the sum of where n is the number of training data sets and L i (w, x i , y i ) is the training data x i , y i is the loss function for a particular set of x i is a particular partial design (PD), and y i The method according to any one of claims 1 to 8, wherein x is the design delta (DELTA) of prediction for this partial design (PD) and w is the weight used.

10. The update of the parameters of the shared recommender system (SRS), specifically the weights, is performed using gradient descent and is defined as follows: During the ceremony, W t+1 and W t 10. The method of claim 9, wherein: t is the weight at training steps t+1 and t, γ is a parameter representing the learning rate, and ∇L is the gradient of the loss function.

11. The parameters of the shared recommender system (SRS) are updated by averaging over a subset of users that receive the gradient information; In the formula, N c is the number of training samples for a particular user, and L c 11. The method of claim 9 or 10, wherein: denotes a loss function for a particular user, and N is the total number of training samples.

12. 12. The method of any one of claims 1 to 11, wherein the gradient information is formed by ∇L, the loss function L being determined by use of binary cross entropy and the loss function L being determinable as a function of a loss function for each training set i or / and for each user c.

13. A method according to any one of claims 1 to 12, wherein a component catalogue (CC) containing components and connections is located on the central server and transmitted to a number of users.

14. A computer program comprising program instructions which, when the computer program is run on a computer, cause the computer to carry out the method according to any one of claims 1 to 13.

15. Recommendation device storing and / or providing a computer program according to claim 14 and having a communication interface of an engineering tool for the design of complex systems.

Citation Information

Patent Citations

  • Automatic design processing method and device therefor

    JP1994168287A

  • Context-based recommendations for robotic process automation design

    US20210342124A1

  • Distributed deep learning system

    WO2021240636A1