Optimization based on transfer learning

Abstract

A system (100) and method for implementing transfer learning based optimization is disclosed. The method includes receiving, by a processing unit from a client device (110), an input indicating a target optimization task, wherein the target optimization task is associated with an unsolved optimization problem. Furthermore, a similarity between the target optimization task and the at least one source optimization task is determined by applying a predefined similarity check logic. The source optimization task is associated with the solved optimization problem and its complete optimization history, and furthermore, a complexity score is calculated for the source optimization task and the target optimization task from the output of the application subject to the similarity check logic to determine whether the source optimization task is more complex than the target optimization task. If so, transfer learning is initiated by adapting a target optimizer based on an initial population and one or more model parameters associated with a source optimizer employed in the source optimization task to solve the unsolved optimization problem.

Description

Technical Field

[0001] The present invention relates to optimization technology, and more particularly, to a system and method for implementing transfer learning-based optimization. Background Art

[0002] Typically, optimization tasks within the engineering field may include repeated formulations of optimization problems. Optimization practitioners rerun optimization studies that often make minor modifications, such as changing the boundary conditions of the underlying problem, and there is no mechanism to leverage learning from past optimization studies to accelerate searches in related problems. In other words, existing optimizers do not take into account "learning" from previous optimization problems when solving related optimization problems. This results in repetition of search patterns and, therefore, longer lead times for searching for optimal solutions, thereby increasing computational power measured by footprint. Furthermore, optimization problems based on real-world applications require highly reliable datasets that are computationally expensive to generate and are often subject to statistical epistemic uncertainty caused by a lack of data points.

[0003] Transferring learning from a solved optimization problem to a related unsolved optimization problem can mitigate the above disadvantages. Therefore, a mechanism is needed to identify optimization problems that are related to the optimization problem at hand. The prior art fails to disclose techniques for identifying related or similar optimization problems. Instead, the prior art relies on independently and artificially constructing simpler problems, or assuming that the two optimization problems are related to each other. Such assumptions may lead to cases of negative transfer, where the transfer of learning hinders the performance of the optimizer instead of improving it.

[0004] There is an opportunity where optimization problems can be categorized into groups or families, where optimization problems that exhibit common characteristics can be grouped together into a single family. Non-limiting examples of such characteristics include the nature of the design variables, boundary conditions, assumptions, physical properties of the problem, simulation fidelity, responses, etc. Grouping similar optimization problems into families opens up the possibility of leveraging learning within the family of optimization problems and achieving optimization strategies through transfer learning.

[0005] In view of the above, there is a need for a mechanism to perform transfer learning from a solved optimization problem to an unsolved optimization problem by eliminating the possibility of negative transfer of learning. Therefore, an object of the present invention is to provide a system and method for implementing optimization based on transfer learning. Summary of the invention

[0006] The objectives of the present invention are achieved through the system, device and method disclosed herein for implementing optimization based on transfer learning.

[0007] The method includes receiving, by a processing unit, an input from a client device indicating at least a target optimization task. The target optimization task is associated with an unsolved optimization problem.

[0008] The method also includes determining the similarity between the target optimization task and at least one source optimization task by applying a predefined similarity check logic. The source optimization task is associated with a solved optimization problem and its complete optimization history. In one embodiment, determining the similarity between the target optimization task and at least one source optimization task by applying a predefined similarity check logic includes performing a low-level check to determine the similarity based on at least one of the metadata and features presented in the source optimization task and the target optimization task. In addition, if the low-level check indicates the similarity between the source optimization task and the target optimization task, a high-level check is performed to calculate a plurality of similarity indexes. In addition, an output is determined based on the values ​​of the plurality of similarity indexes, wherein the output indicates one of the similarity and dissimilarity between the source optimization task and the target optimization task. In one embodiment, the high-level check is based on one or more of an interpretable self-organizing map, an image comparison, a Pearson similarity index, a cosine similarity index, a Jaccard similarity index, a mean square, a structural similarity index, and visualization.

[0009] The method also includes generating an output on a user interface of the client device indicating an outcome of applying the similarity checking logic.

[0010] In a further embodiment, the method further includes, subject to the output of applying the similarity check logic, calculating a complexity score associated with each of the source optimization task and the target optimization task using predefined logic. In addition, comparing the estimated complexity scores associated with each of the source optimization task and the target optimization task to determine whether the complexity of the source optimization task is greater than the complexity of the target optimization task.

[0011] In one embodiment, the method further includes initiating transfer learning from the source optimization task to the target optimization task if the complexity of the source optimization task is greater than the complexity of the target optimization task. In one embodiment, initiating transfer learning from the source optimization task to the target optimization task includes selecting a target optimizer from a plurality of optimizers based on the optimization history of the solved optimization problem. In a further embodiment, the method includes first identifying an initial population from the source optimization task and one or more model parameters associated with the source optimizer adopted in the source optimization task. The one or more model parameters are associated with the minimum model error in the source optimizer. In addition, the target optimizer is adapted based on the identified initial population and the one or more model parameters. The adapted target optimizer is also used to generate a solution to an unsolved optimization problem. In one embodiment, each of the source optimizer and the target optimizer is a meta-heuristic optimization algorithm. In a further embodiment, the meta-heuristic optimization algorithm is one of a genetic algorithm and a neural network algorithm.

[0012] In one embodiment, the method further comprises generating, on a user interface of the client device, a notification indicating at least a solution to the unresolved optimization problem.

[0013] The purpose of the present invention is achieved by a device for implementing optimization based on transfer learning. The device includes one or more processing units, and a memory unit communicatively coupled to the one or more processing units. The memory unit includes a comparison module stored in the form of machine-readable instructions that can be executed by one or more processing units. The comparison module is configured to perform the above method steps. A coprocessor such as a graphics processing unit (GPU), a field programmable gate array (FPGA) or a neural processing / computing engine can also be used to perform the operation of the comparison module. In addition, the memory unit can also include a database.

[0014] In addition, the object of the present invention is achieved by a system for implementing optimization based on transfer learning. The system includes one or more client devices, and a device communicatively coupled to the one or more client devices, the device being configured to implement the optimization problem based on transfer learning based on input received from the one or more client devices according to any of the method steps described above.

[0015] The object of the present invention is also achieved by a computer program product. The computer program product may be, for example, a computer program or may include another element in addition to the computer program. The other element may be hardware, such as a memory device storing the computer program, a hardware key for using the computer program, etc., and / or software, such as a document or software key for using the computer program.

[0016] The object of the present invention is also achieved by a computer-readable medium, on which a program code segment of a computer program is stored. When the program code segment is executed, the program code segment can be loaded into a processor that executes the above method and / or can be executed by the processor.

[0017] Implementing the present invention by means of a computer program product and / or a non-transitory computer-readable storage medium has the following advantages: a computer system can be easily adapted to work as proposed by the present invention by installing the computer program.

[0018] The computer program product may be, for example, a computer program or may include another element in addition to the computer program. The other element may be hardware, such as a memory device storing the computer program, a hardware key for using the computer program, etc., and / or software, such as a document or software key for using the computer program.

[0019] The nature, features and advantages of the present invention and the manner of achieving them will become more apparent and easier to understand (clear) with the following description of the embodiments of the present invention in conjunction with the corresponding drawings. The illustrated embodiments are intended to illustrate, rather than limit, the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the illustrative embodiments shown in the accompanying drawings, in which:

[0021] Figure 1A A block diagram of a system for implementing optimization based on transfer learning according to an embodiment of the present invention is shown;

[0022] Figure 1B A block diagram of a system for implementing optimization based on transfer learning according to an embodiment of the present invention is shown;

[0023] Figure 2 A flowchart of a method 200 for managing an optimization problem according to an embodiment of the present invention is shown;

[0024] Figure 3 A detailed flow chart of an exemplary method 300 for performing a similarity check according to an embodiment of the present invention is shown;

[0025] Figure 4 A flow chart showing a method for achieving optimization of an unsolved optimization problem based on transfer learning from a solved optimization problem according to an exemplary embodiment of the present invention;

[0026] Figure 5 An exemplary workflow of transfer learning-based optimization using a genetic algorithm according to an exemplary embodiment of the present invention is shown;

[0027] Fig. 6AAn exemplary workflow of optimization using a neural network algorithm (source optimizer) according to an embodiment of the present invention is shown;

[0028] Figure 6B An exemplary workflow for performing transfer learning-based optimization using a neural network algorithm (target optimizer) according to an embodiment of the present invention is shown;

[0029] Figure 7 shows graphs indicating convergence of optimization of two similar functions, a 2D spherical function and a 2D Bohachevsky function, using transfer learning based on a neural network algorithm and a genetic algorithm and without using transfer learning;

[0030] Figure 8 shows graphs indicating convergence of optimization of two dissimilar functions, a 2D spherical function and a 2D Matyas function, using transfer learning based on a neural network algorithm and a genetic algorithm and without using transfer learning;

[0031] Fig. 9 is a graph showing a comparison between the convergence of optimizations with and without the transfer of learning from the optimization problem of IPMSMX to the optimization problem of IPMSMA;

[0032] Fig.10 shows a graph showing a comparison between the convergence of optimizations with and without transfer of learning from an optimization problem of IPMSMX to an optimization problem of SCIM6; and

[0033] Fig.11 A graph showing a comparison between the convergence of optimizations with and without transfer learning from a knowledge-poor source problem to a knowledge-rich target problem. DETAILED DESCRIPTION

[0034] Hereinafter, embodiments for implementing the present invention are described in detail. Various embodiments are described with reference to the accompanying drawings, wherein the same reference numerals are used to refer to the same elements throughout. In the following description, for the purpose of explanation, many specific details are set forth to provide a thorough understanding of one or more embodiments. Obviously, such embodiments can be implemented without these specific details.

[0035] Figure 1A A block diagram of a system 100 for implementing transfer learning-based optimization according to an embodiment of the present invention is shown.

[0036] System 100 includes an apparatus 105 and a plurality of client devices 110A-N (hereinafter collectively referred to as client devices 110). Each of client devices 110A-N is connected to apparatus 105 via a network 112, such as a local area network (LAN), a wide area network (WAN), Wi-Fi, etc. In one embodiment, apparatus 105 is a computing system implemented in a cloud computing environment. As used herein, a "cloud computing environment" refers to a processing environment that includes configurable computing physical and logical resources (e.g., networks, servers, storage devices, applications, services, etc.) and data distributed over a network (e.g., the Internet). A cloud computing environment provides on-demand network access to a shared pool of configurable computing physical and logical resources.

[0037] like Figure 1B As shown, the device 105 includes a processing unit 115, a memory 120, a storage unit 125, a communication module 130, a network interface 135, an input unit 140, an output unit 145, and a standard interface or bus 150. The device 105 can be a (personal) computer, a workstation, a virtual machine running on host hardware, a microcontroller or an integrated circuit. As an alternative, the device 105 can be a real or virtual computer group (the technical term for a real computer group is a "cluster" and the technical term for a virtual computer group is a "cloud"). The term "processing unit" used here means any type of computing circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing microprocessor, a reduced instruction set computing microprocessor, a very long instruction word microprocessor, an explicit parallel instruction computing microprocessor, a graphics processor, a digital signal processor, or any other type of processing circuit. The processing unit 115 can also include an embedded controller, such as a general or programmable logic device or array, an application-specific integrated circuit, a single-chip computer, etc. Generally, the processing unit 115 can include hardware elements and software elements. Processing unit 115 may be configured for multithreading, ie, processing unit 115 may host different computing processes simultaneously, executing in parallel or switching between active and passive computing processes.

[0038] The memory 120 may include one or more of a volatile memory and a non-volatile memory. The memory 120 may be coupled for communication with the processing unit 115. The processing unit 115 may execute instructions and / or codes stored in the memory 120. Various computer-readable storage media may be stored in the memory 120 and may be accessed from the memory 120. The memory 120 may include any suitable element for storing data and machine-readable instructions, such as a read-only memory, a random access memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a hard drive, a removable media drive for processing optical disks, digital video disks, magnetic disks, cassettes, memory cards, etc. The memory 120 includes a comparison module 155. The comparison module 155 may be stored in the memory 120 in the form of machine-readable instructions and may be executed by the processing unit 115. When executed by the processing unit 115, these machine-readable instructions enable the processing unit 115 to implement optimization based on transfer learning. The comparison module 155 includes a metadata extractor module 160, a similarity verifier module 165, a transfer learning orchestration module 170, and Figure 1B shown.

[0039] The similarity verifier module 165 is configured to receive an input indicating a target optimization task from the client device 110, wherein the target optimization task is associated with an unsolved optimization problem. The target optimization task may not include an optimization history or only include a portion of the optimization history. The terms "optimization history" or "optimization run history" may be used interchangeably in the present disclosure to indicate optimization data, iterations, stopping criteria, features, metadata, etc. associated with the optimization problem. The term "iteration number" as used herein refers to the count of intermediate "correction" steps or iterations that the optimizer undergoes until an optimal solution (optima) is identified.

[0040] The similarity validator module 165 is also configured to determine the similarity between the target optimization task and at least one source optimization task by applying a predefined similarity check logic. The source optimization task is associated with a solved optimization problem and its complete optimization history. The similarity index indicates the degree of similarity between the source optimization task and the target optimization task. The source optimization task is a first graph data structure that includes metadata and a trajectory or complete history of optimization runs associated with the solution of the source problem. In one example, the source optimization task can be stored in a JSON file. The target optimization task is a second graph data structure that includes at least metadata associated with the target problem.

[0041] The similarity verifier module 165 is also configured to generate output on a user interface of the client device 110 indicating the outcome of the application of the similarity checking logic.

[0042] The transfer learning orchestration module 170 is configured to obey the output of the application of the similarity check logic and use the predefined logic to calculate the complexity score associated with each of the source optimization task and the target optimization task. The transfer learning orchestration module 170 is also configured to determine whether the complexity of the source optimization task is greater than the complexity of the target optimization task by comparing the estimated complexity scores associated with each of the source optimization task and the target optimization task. The transfer learning orchestration module 170 is also configured to initiate transfer learning from the source optimization task to the target optimization task if the complexity of the source optimization task is greater than the complexity of the target optimization task.

[0043] The apparatus 105 may include a network interface 135 for communicating with the client device 110 via the network 112. Each of the client devices 110A-N includes a user interface (not shown) that enables a user to upload information related to performing a transfer learning-based optimization for an optimization task. The user interface may also be configured by the apparatus 105 to generate an output indicating a solution to the optimization problem when performing a transfer learning-based optimization. In addition, the user interface may also be configured by the apparatus 105 to generate an output indicating, for example, a similarity index, a complexity score, a number of iterations, or a result of other intermediate processes associated with a given optimization technique. In one embodiment, the user interface of the client device 110 may be provided by a network-based or client-based application on the client device 110.

[0044] The storage unit 125 includes a non-volatile memory including a database 175. The database 175 stores one or more source optimization tasks. The input unit 140 may include an input device capable of receiving an input signal, such as a keypad, a touch-sensitive display, a camera, etc. The bus 150 serves as an interconnection between the processing unit 115, the memory 120, the storage unit 125, and the network interface 135. The communication module 130 enables the device 105 to communicate with the client devices 110A-N. The communication module 130 can support different standard communication protocols, such as Transmission Control Protocol / Internet Protocol (TCP / IP), Profinet, Profibus, and Internet Protocol version (IPv4).

[0045] The apparatus 105 according to an embodiment of the present disclosure includes an operating system that employs a graphical user interface. The operating system allows for the simultaneous presentation of multiple display windows in the graphical user interface, wherein each display window provides an interface to a different application or a different instance of the same application. A user can manipulate a cursor in the graphical user interface via a pointing device. The position of the cursor can be changed and / or an event such as a click of a mouse button can be generated to initiate a desired response.

[0046] Any of a variety of commercial operating systems may be used, if appropriately modified, such as Microsoft Windows 10, a product of Microsoft Corporation located in Redmond, Washington. TM As described, an operating system is modified or created according to the present disclosure.

[0047] Those of ordinary skill in the art will understand that Figure 1A , 1B The hardware depicted in the 1C may vary for different implementations. For example, other peripheral devices, such as optical drives, local area network (LAN) / wide area network (WAN) / wireless (e.g., Wi-Fi) adapters, graphics adapters, disk controllers, input / output (I / O) adapters, network connection devices, may also be used in addition to or in place of the described hardware. The described examples are provided for the purpose of explanation only and are not meant to imply architectural limitations to the present disclosure.

[0048] The present invention is not limited to a specific computer system platform, processing unit, operating system or network. One or more aspects of the present invention may be distributed in one or more computer systems, for example, a server configured to provide one or more services to one or more client computers or to perform a complete task in a distributed system. For example, according to various embodiments, one or more aspects of the present invention may be performed on a client-server system including components distributed in one or more server systems that perform multiple functions. These components include, for example, executable code, intermediate code or interpreted code that communicates over a network using a communication protocol. The present invention is not limited to being executable on any particular system or system group, and is not limited to any particular distributed architecture, network or communication protocol.

[0049] Figure 2 A flow chart of a method 200 for managing an optimization problem according to an embodiment of the present invention is shown. The method may be implemented on the device 105 in the form of machine-readable instructions.

[0050] At step 205, input indicating at least a target optimization task is received by the processing unit 115 from the client device 110. The target optimization task is associated with an unsolved optimization problem.

[0051] In one embodiment, the input may be received in the form of a configuration file. The configuration file may include at least one indication of a target optimization task. In order to identify the target optimization task from the configuration file, the metadata extraction module 170 may parse the received configuration file to extract metadata corresponding to the target problem. The metadata is further stored in a graph data structure within the database 175 together with a unique ID. This graph data structure forms the target optimization task. In one implementation, if available, the configuration file may also include a partial history of optimizations associated with an unsolved optimization problem. Alternatively, the configuration file may include a location within the database 175 where a partial optimization history is stored. During optimization based on transfer learning, a partial optimization history of an unsolved optimization problem may help converge to a solution more quickly.

[0052] In another implementation, the configuration file may also include an indication of a source optimization task. A source optimization task is associated with a solved optimization problem and its complete optimization history. If the configuration file includes an indication of a source optimization task and its complete optimization history, the metadata extractor module 160 dynamically extracts metadata associated with the source optimization task by parsing the configuration file to identify the source optimization task, similar to the process of generating the target optimization task. This is helpful when a user is interested in determining whether transfer learning from a source optimization task of interest to a target optimization task is possible. Alternatively, the configuration file may simply indicate the location in the database 175 where the source optimization task is located. For example, the source optimization task may be associated with the optimization of a 2D spherical function, while the target optimization task may be the optimization of a 2D Bohachevsky function.

[0053] At step 210 , the similarity between the target optimization task and at least one source optimization task is determined by applying a predefined similarity check logic.

[0054] If the configuration file does not include any indication of a source optimization task, the source optimization task may be selected from a plurality of source optimization tasks stored in the database 175 based on a predetermined order.

[0055] The similarity check logic helps determine whether the solved optimization problem and the unsolved optimization problem belong to the same "class" of optimization problems. This is because effective transfer learning from the source optimization task to the target optimization task can only occur if the solved optimization problem and the unsolved optimization problem belong to the same class of optimization problems.

[0056] The similarity check logic may perform similarity checks at two levels, a primary low-level check and an advanced high-level check. The low-level check is performed to determine similarity based on at least one of metadata and features present in the source optimization task and the target optimization task. This is because, in the case of optimization problems belonging to different classes, the metadata or features may be different. The term "metadata" as used herein refers to a series of data sets that represent the impact on the output or input of the optimization problem. Non-limiting examples of metadata may include values ​​and ranges of parameters, variables, and responses. For example, the presence of quadratic terms in the optimization function results in a convex response.

[0057] The term "feature" as used herein refers to a property that indicates how an optimization problem behaves in a search space. Non-limiting examples of features include the nature of the response space, complexity, nonlinearity, type of Pareto front, type of design space, physics type, and multi-physics type. In one example, if the number of features, type of features, and count of boundary conditions are the same, the source optimization task and the target optimization task can be determined to be similar at a low-level check. The low-level check establishes dissimilarity between the source optimization task and the target optimization task based on metadata or features of the source optimization task and the target optimization task, thereby eliminating the need for a high-level check. In the case of such dissimilarity, the method ends at step 215. In addition, the target optimization task can be stored as a new entry in the database 175.

[0058] If the output of the low-level check indicates similarity between the source optimization task and the target optimization task, a high-level check is performed to calculate multiple similarity indices. The similarity indices are calculated based on optimization data, such as distributions, means, and standard deviations of variables and responses. If at least a predefined number of similarity indices among the multiple similarity indices indicate similarity, the output of the application of the similarity check logic can indicate the similarity between the source optimization task and the target optimization task. The use of two-level similarity checks and the calculation of multiple similarity indices help to improve the credibility of the output of the application of the similarity check logic. Refer to later Figure 3 Explain an exemplary high-level check.

[0059] In the event that the input received from the client device 110 does not include an indication of a source optimization task, step 210 may be repeated for each source optimization task present in the database 175 until a source optimization task similar to the target optimization task is identified.

[0060] Advantageously, the use of low-level checks and high-level checks helps optimize the use of computing resources when evaluating the similarity between the source optimization task and the target optimization task. Once the low-level check is successful, the high-level check performed helps determine the similarity between the source optimization task and the target optimization task with greater confidence. In one embodiment, the user may be provided with an option to decide whether to perform the high-level check based on the results of the low-level check. For example, if the low-level check determines that the source optimization task and the target optimization task are similar, the user may be provided with the option to select "yes" or "no" in response to a text prompt such as "Do you want to continue with the high-level check?"

[0061] At step 215 , output is generated on a user interface of the client device 110 indicating the outcome of the application of the similarity checking logic.

[0062] Figure 3 A detailed flow chart of an exemplary method 300 for performing advanced inspections according to an embodiment of the present invention is shown.

[0063] In this embodiment, the source optimization task 305 and the target optimization task 310 are first compared using an interpretable self-organizing map (iSOM) similarity check (as shown in step 315), an image similarity check (as shown in step 320), and a visualization method (as shown in step 325).

[0064] Here, the iSOM similarity check is an algorithm that uses a modified version of the traditional self-organizing map (SOM). SOM is an artificial neural network (ANN) based on unsupervised learning techniques, which is used as a visualization tool for high-dimensional data. The input of SOM includes high-dimensional data, and the output of SOM includes a "map" or visual representation of the high-dimensional data on a two-dimensional space. The architecture of SOM includes two layers, an input layer and an output layer. Unlike other artificial neural networks, SOM lacks an activation layer. Therefore, the weights are passed directly to the neurons in the output layer. Each neuron in the output layer is assigned a weight vector corresponding to the dimension of the input data set (hereinafter referred to as the "input space").

[0065] Advantageously, SOMs produce visual representations that preserve the topology of the high-dimensional input space, unlike other visualization tools. In other words, the mapping of high-dimensional data to low-dimensional data preserves the relative distances between input data points.

[0066] Unlike artificial neural networks that use error correction-based learning techniques (such as back propagation with gradient descent), SOM uses competitive learning, which includes three processes: competition, cooperation, and adaptation.

[0067] Each neuron in the output layer carries a weight vector of the same dimension as the input space. Therefore, if the input space includes "n" variables, each neuron in the output layer carries an n-dimensional weight vector. In the competition phase, the distance between each neuron in the output layer and the input layer is measured using the Euclidean metric. Based on the measured distance, the output neuron closest to the input layer, i.e., the output neuron at the lowest distance, is identified as the winning neuron in the competition phase.

[0068] During the collaboration phase, the weight vector of each neighbor vector of the winning neuron is updated. Subsequently, during each training cycle, the neighborhood kernel function is used to select neighbor neurons for updating the weight vector. Advantageously, the neighborhood kernel function preserves the topological properties of the n-dimensional input space. The neighborhood kernel function relies on two factors, namely time and the distance between the winning neuron and each neighbor neuron, to select neighbor neurons to update the corresponding weight vector.

[0069] During the adaptation phase, i.e. after deciding the winner neuron and the neighboring neurons, the map generated by the output layer is rearranged to take on a shape corresponding to the dimensions of the input space.

[0070] The weight vectors of the neighboring neurons of the winner neuron are updated using the following equation:

[0071] w k =w k -η(t),h tk (t), (x η w k ) (1)

[0072] The learning rate used for ANN is an exponential decay function as follows:

[0073]

[0074] As time “t” increases, the learning rate converges to zero. Therefore, once the learning rate converges to zero, the winner neuron will not be updated. This also serves as the termination criterion for the iteration of the ANN.

[0075] In conventional SOMs, the winning neuron identified during the initial iteration may be changed in subsequent iterations due to the neighborhood kernel function. Iterations are repeated until at least one predefined termination criterion is met. The predefined termination criterion may correspond to a specified error convergence or exhaustion of allowable function evaluations. However, during each iteration, there is a possibility that the map generated by the output layer may collapse and self-intersect. This may affect the visualization and interpretation of the input data.

[0076] Therefore, the iSOM algorithm is used instead of the traditional SOM algorithm to evaluate similarity. In iSOM, the winning neuron is selected by using data from the input space (unlike the traditional SOM which uses both input and output values), and the weights of the corresponding node and the nodes in its neighborhood are updated as the difference between the data point and the corresponding response.

[0077] In the present invention, the iSOM algorithm is used to obtain a two-dimensional visual representation of metadata or features associated with each of the source optimization task and the target optimization task. In one example, the visual representation includes a graph formed by a plurality of hexagonal regions. Each hexagon in the figure represents a neuron in the output layer of the ANN. The weight vectors of the neurons in the output layer of the ANN of both the source optimization task and the target optimization task can be compared with each other to identify similarities. The weight vector associated with each neuron contains information about the position of the neuron in the input space, which in turn depicts the topology of the optimization problem. Therefore, comparing the weight vectors of the neurons in the output layer also means comparing the structural similarities between the unsolved optimization problem and the solved optimization problem.

[0078] In step 320, the graph or visual representation obtained in step 315 is further converted into an image object and provided to a comparison engine to perform an image similarity check. The image similarity check involves comparing the images corresponding to the source optimization task and the TOT. Here, the comparison engine compares each pixel in the image of the source optimization task with the corresponding pixel in the image of the target optimization task. More specifically, the comparison engine identifies the color of the pixel in the graph of the source optimization task and compares the identified color with the color of the corresponding pixel in the image of the TOT. If the colors are consistent, the comparison engine determines that the images are the same. In addition, the comparison engine generates a value that quantifies the visual similarity between the images as output.

[0079] The performance of the comparison engine is associated with a plurality of predetermined parameters. In one example, the user can specify the parameters in a configuration file. In another example, the parameters can take default values. Each parameter implements additional processing of the bitmap image, which makes the comparison more flexible. Non-limiting examples of such parameters include pixel tolerance, color tolerance, transparent color, and comparison mask. Here, the parameter "pixel tolerance" defines the allowed number of dissimilar pixels. If the number of dissimilar pixels is less than or equal to the pixel tolerance, the comparison engine identifies the images as identical. The parameter "color tolerance" specifies the acceptable color difference that two pixels can be considered the same. In one example, the color difference is expressed as an integer value in the range of 0 to 255, which specifies the acceptable difference in the intensity of each color component (red, green, and blue) of the compared pixels. If the difference between the intensity of each of the color components of two pixels does not exceed the color tolerance, the two pixels are considered to be the same.

[0080] The parameter "Transparent Color" when enabled causes the comparison engine to treat the color of the top left pixel of the image as the transparent color and not compare all pixels in the compared images that have the same coordinates as transparent pixels in the images, similar to how transparency is implemented in .ICO and .GIF formats.

[0081] The parameter "comparemask" specifies the area of ​​the image to be compared. A comparemask is another image whose pixels can be either black or white. If a comparemask is specified, white pixels within the comparemask are considered during the comparison, while black pixels are ignored. Unlike the case of defining a transparent color, the image does not need to be modified when a comparemask is used.

[0082] Those skilled in the art must understand that, in addition to the above techniques, various other techniques may be followed for image comparison. In this embodiment, image comparison techniques are used as another method to identify structural similarities between source optimization tasks and target optimization tasks.

[0083] In addition, a similarity index is calculated to obtain a quantitative value of the similarity between the source optimization task and the target optimization task. In the present exemplary embodiment, the similarity index is based on the calculation of the cosine similarity index (at step 330), the Pearson similarity index (at step 335), the mean square error (MSE) (at step 340), the structural similarity index (SSI) (at step 345), and the Jaccard similarity index (at step 350).

[0084] Each of the cosine similarity index and the Pearson similarity index measures the similarity between two vectors in the inner product space, measured by the cosine of the angle between the two vectors. In the present embodiment, the calculation of the cosine similarity index and the Pearson similarity index is performed in combination with the iSOM technique and the active subspace and manifold technique explained previously. In the case of the iSOM technique, the proximity of the weight vectors of the neurons in the output layer of the ANN corresponding to the source optimization task and the target optimization task is used to calculate the cosine similarity index and the Pearson similarity index. In the case of the active subspace and manifold techniques, a one-dimensional direction vector is used to calculate the cosine and Pearson similarity values.

[0085] Advantageously, the cosine similarity index and the Pearson similarity index are size independent and can therefore be used to find similarity even if the weight vectors are far apart from each other in the vector space. The formula for calculating similarity is listed below:

[0086] If M = [v1, v2, ..., v N ] and L=[u1,u2,...,u N ] are two vectors, then the cosine similarity index of M and L is calculated as follows:

[0087]

[0088] in<M,L> =M T L=(v1u1)+…+(v n u n ) (4)

[0089]

[0090] The Pearson similarity index is calculated as follows:

[0091]

[0092] Those skilled in the art will appreciate that image comparison may be limited to specific regions of each image.Image similarity indices, such as MSE, SSI and Jaccard similarity values, provide a quantitative assessment of the similarity between two image objects or two image regions within an image object.

[0093] Here, MSE is the sum of square differences between intensity values ​​of pixels in the compared images. It can be understood that the calculation of MSE requires that the intensity values ​​in the image objects are in the same range.

[0094]

[0095] in,

[0096] Y - Reference image object generated from the iSOM map corresponding to the source optimization task.

[0097] Y 1 - Image objects generated from the iSOM map corresponding to the target optimization task.

[0098] SSI is calculated on each window of the image. For two windows x and y of common size N×N, SSI is calculated as follows:

[0099]

[0100] in,

[0101] μ x is the mean value of x;

[0102] μ y is the mean value of y;

[0103] is the variance of x;

[0104] is the variance of y;

[0105] σxy is the covariance of x and y; and

[0106] c1 and c2 are variables of pixel value to stabilize the division with weak denominator.

[0107] The Jaccard similarity value, also known as the Jaccard similarity coefficient, is a statistic used to assess the similarity and difference between two image objects.

[0108]

[0109] in,

[0110] Y is the reference image

[0111] Y 1 are the images being compared.

[0112] The visualization technique performed at step 325 involves manual intervention, wherein the similarity between the source optimization task and the target optimization task is determined by the user based on visual similarity. In the present embodiment, the visualization includes displaying to the user an iSOM map corresponding to the source optimization task and the target optimization task. In addition, a plurality of predefined category labels may be provided to the user to select from. For example, the category labels may include "very similar", "similar", "somewhat similar", "dissimilar", and "very dissimilar". Each category is weighted to a weight ranging from 0.2 to 1, where 1 represents the maximum similarity and 0.2 represents the minimum similarity.

[0113] As previously described, if at least a predefined number of similarity indices among the plurality of similarity indices indicate similarity, the output of the application of the similarity check logic may indicate the similarity between the source optimization task and the target optimization task. For example, if the similarity index is cosine similarity, a value greater than 0, such as 0.9, may indicate similarity, if the similarity index is mean square error (MSE), a value less than 0.10 may indicate similarity, and so on. In one example, a total of six different types of similarity indices may be calculated. If, for example, four of the six similarity indices indicate similarity, the output of the application of the similarity check logic may indicate that the solved optimization problem and the unsolved optimization problem are determined to be from the same class of optimization problems, as shown in step 355. Therefore, transfer learning may be used to transfer learning from the source optimization task to the target optimization task. Otherwise, the unsolved optimization problem is determined to be dissimilar, as shown in step 360. Therefore, transferring learning from the source optimization task to the target optimization task may be ineffective in solving the unsolved optimization problem. In the following, “transfer of learning” from one optimization problem to another should be interpreted as the transfer of learning between the individual optimization tasks of an optimization problem, i.e., from the source optimization task to the target optimization task.

[0114] Table 1 below shows experimental results of similarity indexes calculated for two structurally similar functions, a 2D spherical function and a 2D Bohachevsky function:

[0115]

[0116] Table I: Experimental results of calculating the similarity index of 2D spherical function and 2D Bohachevsky function.

[0117] Table II shows the similarity index of two dissimilar functions, 2D spherical function and 2D Matyas function:

[0118]

[0119] Table II: Experimental results of similarity index of 2D spherical function and 2D Matyas function.

[0120] Figure 4 A flow chart of a method 400 for optimizing an unsolved optimization problem based on transfer learning according to an exemplary embodiment of the present invention is shown. Hereinafter, the term "target problem" may be used to refer to an unsolved optimization problem, and the term "source problem" may be used to refer to a solved optimization problem, and its complete optimization history is readily available. In the present embodiment, the optimization of both the source problem and the target problem is based on a meta-heuristic optimization algorithm. The present disclosure is explained by using two meta-heuristic optimization algorithms, examples of a neural network algorithm and a genetic algorithm. However, it will be appreciated by those skilled in the art that the teachings of the present disclosure may be extended to other types of meta-heuristic algorithms, such as particle swarm optimization, artificial bee colony optimization, taboo search, etc. In general, the teachings of the present invention may be applied to any optimization algorithm (i.e., optimizer) independent of dimension. Hereinafter, the term "source optimizer" is used to refer to an optimizer used in a source optimization task to solve a source problem. Similarly, the term "target optimizer" is used to refer to an optimizer used in a target optimization task to solve a target problem.

[0121] At step 405, input indicating at least a target optimization task is received by the processing unit 115 from the client device 110. Furthermore, as explained previously with reference to steps 205 and 210 of the method 200, a source optimization task and a target optimization task are generated based on the input.

[0122] At step 410, similarity checking logic is used to check the similarity between the source optimization task and the target optimization task, as previously described with reference to steps 210 and 211 of method 200. Figure 3 If the outcome of the application of the similarity check logic indicates that the source optimization task and the target optimization task are not similar, then the method ends at 415. Otherwise, step 420 is performed.

[0123] In step 420, using predefined logic, a complexity score associated with each of the source problem and the target problem is calculated. The complexity score is a numerical value indicating the complexity associated with the optimization problem. Only when the complexity associated with the source optimization task is greater than the complexity of the target optimization task, the migration of learning from the source optimization task to the target optimization task can occur. In one implementation, the complexity score can be calculated for a given optimization problem based on the number of variables or features in the corresponding optimization task using predefined logic. In addition, the estimated complexity scores are compared to determine whether the complexity of the source optimization task is greater than the complexity of the target optimization task. If not, the method ends in step 425. Otherwise, the transfer learning process is initiated in steps 430, 435 and 440.

[0124] At step 430, a target optimizer is selected from a plurality of optimizers based on the optimization history of the source problem. In the present embodiment, the plurality of optimizers include a neural network algorithm and a genetic algorithm. More specifically, if optimization data corresponding to the optimization based on the genetic algorithm is available in the source optimization task, the genetic algorithm is selected as the target optimizer, as shown in step 440. Otherwise, as shown in step 435, the neural network algorithm is selected as the target optimizer.

[0125] After the target optimizer is selected, transfer learning from the source optimization task to the target optimization task is initiated. In this embodiment, the transfer learning includes identifying an initial population and one or more model parameters associated with the source optimizer. The one or more model parameters correspond to the minimum model error in the source optimizer. In addition, the target optimizer is adapted based on the identified initial population and the one or more model parameters. The adapted target optimizer is further used to generate a solution to the target problem.

[0126] Here, the term "model parameters" may refer to one or more parameters associated with an optimizer used to solve an optimization problem. For example, if the optimizer is based on a genetic algorithm, the model parameters may include statistical parameters such as, but not limited to, an initial population, a mean, a standard deviation, and an average distance between potential solutions in a population. The term "population" as used herein refers to a set of potential solutions to an optimization problem. Similarly, if the optimizer is a neural network algorithm, the model parameters may include an initial population, tuning parameters, and weights of neurons associated with the neural network algorithm.

[0127] Refer to Figure 5 and Fig. 6A -B Explain in detail the process of transfer learning from the source optimization task to the target optimization task using each of the genetic algorithm and the neural network algorithm.

[0128] Figure 5An exemplary workflow 500 of transfer learning based optimization using a genetic algorithm according to an exemplary embodiment of the present invention is shown.

[0129] In step 505 , the initial population of the target optimization task is set to the initial population of the source optimization task.

[0130] In step 510, the minimum fitness value associated with the initial population is calculated. To calculate the minimum fitness value, a fitness function is used to evaluate the fitness of each potential solution in the initial population. In addition, the minimum fitness value is identified based on the value corresponding to the fitness of each potential solution. In addition, the mean of the population is set to the identified minimum fitness value.

[0131] At step 515, a new population of potential samples is generated based on the minimum fitness value of the initial population and the model parameters 520 associated with the source optimizer. Here, the model parameters 520 include the standard deviation and the average distance between potential samples, as determined from the source optimization task. In particular, the model parameters 520 are used to initialize the target optimizer, i.e., the genetic algorithm for solving the target problem. In this embodiment, the standard deviation corresponding to the source problem is used as the standard deviation of the population of step 505 corresponding to the target problem, and the average distance corresponding to the source problem is used as the range of the population.

[0132] In step 520, it is determined whether a predetermined condition is satisfied. In this embodiment, the predefined condition includes that the maximum number of iterations or the minimum fitness value of steps 510 and 515 is less than 10. -6 , whichever is earlier. If the predefined condition is met, step 525 is performed. Otherwise, steps 510 and 515 are iterated for the new population by replacing the initial population with the new population.

[0133] At step 525, an optimal solution to the target problem is identified from the new population. Here, the optimal solution is the potential solution that provides the minimum fitness value.

[0134] Fig. 6A An exemplary workflow 600 of optimization using a neural network algorithm (source optimizer) according to an embodiment of the present invention is shown.

[0135] Neural network algorithms are meta-heuristic optimization algorithms known in the art (Sadolla et al.) for solving optimization problems. For the sake of brevity, it is assumed that the reader is familiar with the relevant prior art literature and techniques, and such features will only be briefly mentioned or hinted at when the context requires.

[0136] The neural network algorithm operates on a population of prediction solutions called pattern solutions. The best pattern solution for a given target problem is identified as the target solution (optimal solution). The neural network algorithm aims to reduce the error between the target solution and other prediction solutions by adjusting the weight values ​​and pattern solutions. The neural network algorithm is similar to an artificial neural network (ANN) in that it uses N input data with D dimensions and one target data (response). The target data includes the target solution X Goal and the corresponding target weight W Goal . Corresponding to the target solution X Goal The target weight W Goal Select from the population of weights in the weight matrix W. The weight matrix W is a square matrix (N×N) including random numbers uniformly between 0 and 1.

[0137] Method 600 begins at step 602 .

[0138] At step 604, the user selects a population size and the number of fitness evaluations (NFEs) to generate an initial population.

[0139] At step 606, a population of random pattern solutions is generated and stored in a solution matrix X1. The population can be represented as [X11, X12...X1N], where each of X11, X12...X1N is a pattern solution.

[0140] At step 608, the cost of each pattern solution in the solution matrix X1 is calculated.

[0141] At step 610, a target solution X is determined based on the calculated cost. Goal1 and the target weight W Goal1 In particular, the target solution X Goal1 and the target weight W Goal1 corresponds to the minimum value of the cost.

[0142] In step 612, based on the constraint that the sum of the weights of the pattern solutions does not exceed 1, the weights in the weight matrix W1 are randomly updated.

[0143] At step 614, the new pattern solution X is calculated using an equation inspired by the weight summation technique used in ANNs. New1 In addition, the population Xi updated the new mode solution X New1 .

[0144] At step 616, a predefined logic is used to calculate the target weight W. Goal1 Update the weight matrix W1 with the value of

[0145] In step 618, the correction factor β is used to determine the percentage of pattern solutions to be changed using the bias operator or the migration function operator. Here, the value of the correction factor is initialized to 1 in the first iteration. A correction factor of 1 indicates that there is a 100% chance of modifying each pattern solution in the population. In addition, for each iteration, the correction factor is adaptively reduced in each iteration using a predefined reduction formula. If the random variable is less than the value of the correction factor in the iteration, the bias operator is applied to the population as shown in step 620. Otherwise, the migration function operator is applied to the population as shown in step 622. The combination of the bias operator and the migration function operator in the neural network algorithm plays a vital role in generating solutions to the target problem. In particular, the bias operator is responsible for generating new pattern solutions during the early iterations (rand<=β), while the migration function operator plays a more important role in the final iteration (rand>β).

[0146] At step 620, the bias operator modifies a certain percentage of the pattern solutions in the new population of pattern solutions generated at step 614 and the updated weight matrix W1 generated at step 616. The bias operator acts as noise to explore the population and generate solutions closer to the target solution X Goa11 In addition, step 624 is performed on the updated population.

[0147] At step 622, the migration function operator migrates each of the new pattern solutions from the current position to a new position in the population to bring the new pattern solutions closer to the target solution X. Goal1 Additionally, step 624 is performed on the updated population.

[0148] At step 624, the cost associated with each new pattern solution is calculated. If the cost is less than the target cost, step 626 is performed.

[0149] In step 626, the target solution X is updated with the pattern solution in the set of pattern solutions corresponding to the lowest cost value obtained in step 624. Goal In addition, the solution corresponding to the target X Goal1 The weight is used to update the target weight W Goal1 .

[0150] In step 628, it is determined whether at least a predetermined stopping criterion is met. If so, step 630 is performed. Otherwise, steps 614 to 628 are repeated until the stopping criterion is met. Each such repetition is an iteration of the optimization run.

[0151] At step 630, generate an indication target solution X Goal1 and the target weight W Goal1 Output.

[0152] The method ends at step 632 .

[0153] Figure 6B An exemplary workflow 634 for performing transfer learning based optimization using a neural network algorithm (objective optimizer) according to an embodiment of the present invention is shown.

[0154] The method starts at step 636 .

[0155] At step 638, a population of pattern solutions is generated by seeding the initial population of source optimizers obtained at step 606 of method 600. In addition, the generated pattern solutions are stored in the solution matrix X2. The population can be represented as [X21, X22...X2N], where X21, X22...X2N are pattern solutions.

[0156] At step 640, the cost of each pattern solution in the solution matrix X2 is calculated.

[0157] At step 642, a target solution X is determined based on the calculated cost. Goal2 and the target weight W Goal2 In particular, the target solution X Goal2 and the target weight W Goal1 corresponds to the minimum value of the cost.

[0158] At step 644, the weights in weight matrix W2 are set to the target weights W obtained at step 612 in the final iteration of method 600. Goal1 . This serves as the first level of transfer learning from the source optimizer.

[0159] At step 646, the new pattern solution X is calculated using an equation inspired by the weight summation technique used in ANNs. New2 In addition, with the new model solution X New2 Update pattern solution group.

[0160] At step 648, a predefined logic is used to calculate the target weight W. Goal2 Update the weight matrix W2 with the value of

[0161] At step 650, similar to step 618 of method 600, a correction factor β is used to determine the percentage of pattern solutions to be changed using a bias operator or a migration function operator. If the random variable is less than the value of the correction factor in an iteration, the bias operator is applied to the population as shown in step 652. Otherwise, the migration function operator is applied to the population as shown in step 654. In particular, the bias operator generates new pattern solutions during early iterations (rand<=P), while the migration function operator plays a more important role in the final iterations (rand>P).

[0162] At step 652, the bias operator modifies a certain percentage of the pattern solutions in the population of new pattern solutions generated at step 646 and the updated weight matrix W2 generated at step 648. The bias operator acts as noise to explore the population and generate solutions closer to the target solution X Goal2 Additionally, step 656 is performed on the updated population.

[0163] At step 654, the migration function operator migrates each new mode solution from its current position to a new position in the population to bring the new mode solution closer to the target solution X. Goal2 Additionally, step 656 is performed on the updated population.

[0164] At step 656, the cost associated with each new pattern solution is calculated. If the cost is less than the target cost, step 658 is performed.

[0165] In step 658, the target solution X is updated with the pattern solution corresponding to the lowest cost value obtained in step 656 among the set of pattern solutions. Goal2 In addition, the solution corresponding to the target X Goal2 The weight is used to update the target weight W Goal2 . Target Solutions X Goal2 and the target weight W Goal2 It is further stored in a new memory location within the memory 120 .

[0166] At step 660, a target solution X is determined based on the value stored in the memory location. Goal2 and the target weight W Goal2 remains the same for three consecutive iterations. If so, then the target solution X Goal2 and the target weight W Goal2 are updated to the target solution X obtained in step 626 in the final iteration of method 600. Goal1 and the target weight W Goal1 . This serves as a second level of transfer learning from the source optimizer.

[0167] Advantageously, the transfer in both stages (step 638 and step 644) helps the target optimizer to retrace the desired path in the population to converge to the global optimal solution more quickly, even in the event of divergence during later iterations. In addition, the second stage of transfer learning is triggered when the target weights and target solution remain unchanged, rather than only at one critical iteration.

[0168] In step 662, it is determined whether at least a predetermined stopping criterion is met. If so, step 664 is performed. Otherwise, steps 646 to 664 are repeated until at least one stopping criterion is met. Each such repetition is an iteration of the optimization run.

[0169] At step 664, generate an indication of the target solution X Goal2 and the target weight W Goal2 Output.

[0170] Figure 7 A graph showing the convergence of optimization of two similar functions, a 2D spherical function (source problem) and a 2D Bohachevsky function (target problem), using transfer learning based on a neural network algorithm and a genetic algorithm and without using transfer learning.

[0171] Table III shows a comparison of the convergence of the optimization using the neural network algorithm with and without transfer learning from the 2D spherical function to the 2D Bohachevsky function:

[0172]

[0173] Table III: Convergence of optimization using neural network algorithm.

[0174] The symbol x* represents the target solution, while f(x*) represents the cost function associated with the neural network algorithm. The true value indicates the actual solution to the optimization problem.

[0175] refer to Figure 7 From Table III, it can be seen that without transfer learning from the source optimizer, the target optimizer based on the neural network algorithm took 81 iterations to converge and reach the near-optimal solution of the 2D Bohachevsky function. However, after the transfer learning-based optimization using the neural network algorithm, the target optimizer reached the optimal solution within 32 iterations. This example illustrates the positive transfer of learning from the source optimization task to the target optimization task, as the transfer has led to faster convergence to the global optimal solution.

[0176] Table IV shows a comparison of the convergence of the optimization using the genetic algorithm with and without transfer learning from the 2D spherical function to the 2D Bohachevsky function:

[0177]

[0178] Table IV: Convergence of optimization using genetic algorithm.

[0179] In the example of Table IV, before transfer learning, the genetic algorithm-based target optimizer took 72 iterations to converge and reach the optimal solution of the 2D Bohachevsky function, and after transfer learning it took 25 iterations. Therefore, the genetic algorithm-based optimization also leads to positive transfer of learning from the source optimization task to the target optimization task.

[0180] Figure 8 Graphs indicating the convergence of optimization of two dissimilar functions, a 2D spherical function and a 2D Matyas function, using transfer learning based on a neural network algorithm and a genetic algorithm and without using transfer learning are shown.

[0181] Table V shows the convergence of the optimization using the neural network algorithm with and without the learning transfer from the 2D ball function to the 2D Matyas function:

[0182]

[0183] Table V: Convergence of optimization using neural network algorithm.

[0184] refer to Figure 8 From Table V, it can be seen that without transfer learning from the source optimizer, the target optimizer based on the neural network algorithm took 63 iterations to converge and reach the near-optimal solution of the 2D Matyas function. However, after the transfer learning-based optimization using the neural network algorithm, the target optimizer reached the optimal solution at 100 iterations, which indicates a slower convergence after transfer learning.

[0185] Table VI shows the convergence of the optimization using the genetic algorithm with and without the learning transfer from the 2D ball function to the 2DMatyas function:

[0186]

[0187] Table VI: Convergence of optimization using genetic algorithm.

[0188] In the example of Table VI, before transfer learning, the genetic algorithm-based objective optimizer took 23 iterations to converge and reach the optimal solution for the 2D Matyas function, and after transfer learning, it took 33 iterations. Figure 8 ,Table V and Table VI, it can be seen that, regardless of the type of optimizer, performing transfer learning between dissimilar functions will lead to negative transfer of learning.

[0189] In addition to the above, the concept of transfer learning based optimization can be further illustrated with optimization problems associated with three different types of motors, namely, frameless interior permanent magnet synchronous motor (IPMSMA), framed interior permanent magnet synchronous motor (IPMSMX), and squirrel cage induction motor (SCIM6). Here, IPMSMA and IPMSMX belong to the same motor family and operate based on the same principle. On the other hand, SCIM6 belongs to a different motor family and operates based on a different working principle from IPMSMX and IP-MSMA.

[0190] In this example, the stator slot opening and the magnet width can be considered as design variables for each motor. In addition, an optimization problem for minimizing torque ripple in each motor can be formulated based on the corresponding design variables.

[0191] IPMSMX:

[0192]

[0193] IPMSMA:

[0194]

[0195] From the above equation, we can see that the optimization problem of IPMSMX is a "poly22" function with six terms, while the optimization task of IPMSMA is a "poly21" function with five terms. Therefore, due to the higher number of terms in the optimization problem, the complexity score corresponding to the optimization problem associated with IPMSMX is higher than that of IPMSMA.

[0196] Tables 5A-6B list the optimization results based on transfer learning for different scenarios.

[0197] Fig. 9 is a graph showing a comparison between the convergence of optimization with and without transfer of learning from the optimization problem of IPMSMX to the optimization problem of IPMSMA. Table VII below shows the calculated similarity index for the optimization tasks corresponding to IPMSMX and IPMSMA.

[0198]

[0199] Table VII: Experimental results of similarity indices calculated for optimization tasks corresponding to IPMSMX and IPMSMA.

[0200] It can be understood that since both IPMSMX and IPMSMA belong to the same type of motor, the similarity index indicates a high similarity between the optimization problems.

[0201] Table VIII shows the convergence of the optimization using the neural network algorithm with and without the transfer of learning from the optimization problem of IPMSMX to the optimization problem of IPMSMA.

[0202]

[0203] Table VIII: Convergence of optimization using neural network algorithm.

[0204] It can be understood from the number of iterations shown in Table 5B that the migration of learning from the optimization problem of IPMSMX to the optimization problem of IPMSMA leads to a faster convergence speed.

[0205] Fig.10 A graph is shown showing a comparison between the convergence of optimization with and without transfer of learning from the optimization problem of IPMSMX to the optimization problem of SCIM6.

[0206] Table IX below shows the similarity indices calculated for the optimization problems corresponding to IPMSMX and SCIM6.

[0207]

[0208] Table IX: Similarity indices corresponding to the optimization problems of IPMSMX and SCIM6.

[0209] As can be seen from Table 6A, the similarity index indicates that the optimization tasks are dissimilar. Therefore, the transfer of learning from the optimization task of IPMSMA to the optimization task of SCIM6 results in negative transfer learning, as shown in Table 6B.

[0210] Table X shows the convergence of optimization using the neural network algorithm with and without transfer of learning from the optimization problem of SCIM6 to the optimization problem of IPMSMX.

[0211]

[0212] Table X: Convergence of optimization using the neural network algorithm.

[0213] Therefore, it can be understood that determining the similarity between two optimization problems before initiating transfer learning helps to avoid negative transfer learning.

[0214] In addition to the above, the transfer of learning from low-complexity optimization problems (knowledge-poor source problems) to high-complexity optimization tasks (knowledge-rich target problems) can also lead to negative transfer, as shown in the test results of Table 7A.

[0215] Table XI shows the results of the transfer of learning from the optimization problem of IPMSMA (the knowledge-poor source problem) to the optimization problem of IPMSMX (the knowledge-rich target problem):

[0216]

[0217] Table XI: Convergence of optimization using neural network algorithm.

[0218] Fig.11 A Plough comparison between the convergence of the optimization with and without transfer learning from a knowledge-poor source problem to a knowledge-rich target problem is shown. From Table 7A and Fig.11 It can be seen that when the source problem is less complex compared to the target problem, the convergence rate decreases after transfer learning, resulting in negative transfer.

[0219] Therefore, for similar optimization problems, it is also necessary to ensure that learning transfer is performed from higher complexity optimization tasks to lower complexity optimization problems to avoid negative transfer.

[0220] Advantageously, the present invention enables determining whether positive transfer learning is possible between two optimization problems in two stages. The first stage is associated with determining the similarity between the optimization tasks, while the second stage is associated with comparing the complexity levels of the optimization problems. This helps researchers make careful decisions about transfer learning to avoid negative transfer learning, thereby reducing computational complexity and unnecessary costs.

[0221] Although the present invention has been described in detail with reference to the preferred embodiments, the present invention is not limited to the disclosed examples. Other variations may be derived by those skilled in the art without departing from the scope of protection of the present invention.

[0222] Reference numerals list

[0223] 100 Systems for implementing transfer learning-based optimization

[0224] 105 Installation

[0225] 110A-N Client Equipment

[0226] 112 Network

[0227] 115 Processing Units

[0228] 120 Memory

[0229] 125 storage units

[0230] 130 Communication module

[0231] 135 Network Interface

[0232] 140 Input unit

[0233] 145 Output Unit

[0234] 150 Standard interface or bus

[0235] 155 Comparison Module

[0236] 160 Metadata Extractor Module

[0237] 165 Similarity Validator Module

[0238] 170 Transfer Learning Orchestration Module

[0239] 175 databases.

Claims

1. A computer-implemented method comprising: Receiving, by a processing unit (115), an input from a client device (110) indicating a target optimization task, wherein the target optimization task is associated with an unsolved optimization problem; determining similarity between a target optimization task and at least one source optimization task by applying a predefined similarity check logic, wherein the source optimization task is associated with a solved optimization problem and its complete optimization history; as well as Output is generated on a user interface of the client device (110) indicating the application results of the similarity checking logic.

2. The method of claim 1, wherein determining the similarity between the target optimization task and the at least one source optimization task by applying a predefined similarity check logic comprises: performing a low-level check to determine similarity based on at least one of metadata and features present in the source optimization task and the target optimization task; If the low-level check indicates similarity between the source optimization task and the target optimization task, a high-level check is performed to calculate a plurality of similarity indices; as well as An outcome is determined based on values ​​of the plurality of similarity indexes, wherein the outcome indicates one of a similarity and a dissimilarity between the source optimization task and the target optimization task.

3. The method according to claims 1 and 2, wherein the high-level inspection is based on one or more of interpretable self-organizing maps, image comparison, Pearson coefficient, cosine similarity, Jaccard similarity, mean square error, structural similarity index and visualization.

4. The method according to any one of claims 1 to 3, further comprising: subject to an output of the application of the similarity checking logic, a complexity score associated with each of the source optimization task and the target optimization task is calculated using predefined logic; as well as By comparing the estimated complexity scores associated with each of the source optimization task and the target optimization task, it is determined whether the complexity of the source optimization task is greater than the complexity of the target optimization task.

5. The method according to any one of claims 1 to 4, further comprising: If the complexity of the source optimization task is greater than the complexity of the target optimization task, Then initiate transfer learning from the source optimization task to the target optimization task.

6. The method according to claims 1 and 5, wherein initiating transfer learning from a source optimization task to a target optimization task comprises: A target optimizer is selected from a plurality of optimizers based on the optimization history of the solved optimization problem.

7. The method according to claims 2 and 6, wherein initiating transfer learning from a source optimization task to a target optimization task further comprises: identifying, from the source optimization task, an initial population and one or more model parameters associated with a source optimizer employed in the source optimization task, wherein the one or more model parameters are associated with a minimum model error in the source optimizer; adapting a target optimizer based on the identified initial population and one or more model parameters; as well as Generate solutions to unsolved optimization problems using an adapted objective optimizer.

8. The method according to claims 1 and 7, wherein each of the source optimizer and the target optimizer is a meta-heuristic optimization algorithm.

9. The method according to claims 1 and 7, wherein the meta-heuristic optimization algorithm is one of a genetic algorithm and a neural network algorithm.

10. The method according to claims 1 and 7, further comprising: A notification is generated on a user interface of the client device (110) indicating at least a solution to the unresolved optimization problem.

11. An apparatus (105) comprising: one or more processing units (115); as well as A memory unit (120) communicatively coupled to the one or more processing units (115), wherein the memory unit (120) comprises a comparison module (155) stored in the form of machine-readable instructions executable by the one or more processing units (115), wherein the comparison module (155) is configured to perform the method steps according to any one of claims 1 to 10.

12. A system (100) comprising: one or more client devices (110); as well as The apparatus (105) of claim 11, being communicatively coupled to the one or more client devices (110), wherein the apparatus (105) is configured to implement transfer learning based optimization based on input received from the one or more client devices (110) according to any one of the method claims 1 to 10.

13. A computer program product having machine-readable instructions stored therein, which, when executed by one or more processing units (115), cause the processing unit (155) to perform the method according to any one of claims 1 to 10.

Images