Combinatorial optimization system, control method thereof, and training method of combinatorial optimization system

Abstract

The present invention relates to a combinatorial optimization system, a control method thereof, and a training method of a combinatorial optimization system. More specifically, the training method of the combinatorial optimization system according to the present invention may comprise the steps of: specifying a training data set including an instance for a combinatorial optimization problem and an optimal solution for the instance; performing supervised learning on a combinatorial optimization model by using the training data set; acquiring the combinatorial optimization model on which the supervised learning has been performed using the training data; and performing reinforcement learning on the combinatorial optimization model on which the supervised learning has been performed.

Description

Combinatorial optimization system and its control method, and learning method of the combinatorial optimization system

[0001] The present invention relates to a combinatorial optimization system and a control method thereof, and a learning method of the combinatorial optimization system.

[0002] Combinatorial optimization (CO) problems play a key role in diverse fields, including operations research, computer science, logistics, and circuit design (e.g., PCB design). However, most combinatorial optimization problems are classified as NP-hard, requiring significant computational resources to derive solutions due to the size of the solution space and the complexity of constraints.

[0003] Traditionally, combinatorial optimization has primarily relied on algorithms that don't utilize machine learning. For example, dynamic programming, greedy algorithms, and branch-and-bound methods are widely used. However, these algorithms are either specialized for specific combinatorial optimization problems or suffer from significant speed limitations.

[0004] Accordingly, research utilizing machine learning (ML) to solve combinatorial optimization problems has been actively underway recently. This research demonstrates the potential of solving combinatorial optimization problems through data-driven approaches. These machine learning-based combinatorial optimization solutions can be categorized into supervised learning (SL) and reinforcement learning (RL). The difference between supervised learning and reinforcement learning lies in the presence of a training dataset containing labeled solutions to instances of the combinatorial optimization problem.

[0005] For example, supervised learning mimics (or replicates) a training dataset containing high-quality solutions. Recently, research has been published on applying generative models, such as the diffusion model, which has proven successful in image and natural language processing, to combinatorial optimization problems. However, because conventional diffusion models do not consider cost information during the learning process, even if the quality of the generated solutions appears similar, there can be significant discrepancies in actual cost.

[0006] Meanwhile, reinforcement learning has the advantage of complementing these problems because it aims for cost optimization, but it has the limitation of being difficult to learn in large-scale problems where rewards are rare and have large variance.

[0007] Therefore, there is still a need for methods to solve combinatorial optimization problems.

[0008] The present invention provides a combinatorial optimization system capable of producing an optimal solution for a combinatorial optimization problem (CO), a control method thereof, and a learning method of the combinatorial optimization system.

[0009] More specifically, the present invention provides a combinatorial optimization model capable of producing an optimal solution that minimizes cost while satisfying constraints of a combinatorial optimization problem instance.

[0010] Furthermore, the present invention aims to provide a combinatorial optimization model that can be universally utilized in combinatorial optimization problems of various scales.

[0011] In addition, the present invention aims to solve the problem of lack of high-quality learning data and provide a learning method of a combinatorial optimization model that can generate high-quality solutions.

[0012] Furthermore, the present invention provides a learning method for a combinatorial optimization model that can generate high-quality solutions while satisfying constraints.

[0013] In order to solve the problem discussed above, a learning method of a computerized combinatorial optimization system according to the present invention, including the following, may include a step of specifying a learning data set including an instance for a combinatorial optimization problem and an optimal solution for the instance, a step of performing supervised learning on a combinatorial optimization model using the learning data set, a step of obtaining a combinatorial optimization model supervised by the learning data, and a step of performing reinforcement learning on the combinatorial optimization model supervised by the learning data.

[0014] In an embodiment, the parameters of the combinatorial optimization model are learned based on the supervised learning so as to approximate the conditional distribution of the optimal solution for the instances included in the learning data set, and the combinatorial optimization model can be reinforced learned using the parameters of the supervised learned combinatorial optimization model.

[0015] In an embodiment, the parameters of the combinatorial optimization model can be learned so that the probability distribution of the solutions sampled from the combinatorial optimization model approximates the conditional distribution of the optimal solution.

[0016] In the embodiment, in the step of learning the map, the combination optimization model is learned through a diffusion process, and the diffusion process may include a forward noising process and a backward denoising process.

[0017] In an embodiment, the combinatorial optimization model may be trained to gradually add noise to the optimal solution through the forward noise addition process to generate solutions with added noise, and to gradually remove noise included in the solutions with added noise through the backward restoration process to restore a solution close to the optimal solution.

[0018] In an embodiment, the combinatorial optimization model can be trained to approximate the conditional distribution of the optimal solution from the solution to which the noise has been added.

[0019] In an embodiment, the parameters of the combinatorial optimization model can be learned by optimizing the first objective function to approximate the conditional distribution of the optimal solution from the solution to which the noise has been added.

[0020] In an embodiment, as the restoration result of the combination optimization model, a restored solution that approximates the optimal solution can be obtained, and using a decoder, the restored solution can be converted into a solution that satisfies preset constraints for the instance.

[0021] In the embodiment, in the reinforcement learning step, the supervised learning combinatorial optimization model can be reinforced learned using a solution that satisfies the constraints.

[0022] In an embodiment, the reinforcement learning step may include a step of calculating a cost for a solution satisfying the constraint using a cost function, a step of calculating a reward for the cost using a reward function, and a step of providing the reward to the supervised learning combinatorial optimization model.

[0023] In the embodiment, in the step of providing the compensation, the compensation can be provided for the restored solution obtained as the restoration result of the combination optimization model.

[0024] In the embodiment, in the reinforcement learning step, the supervised learning combinatorial optimization model can be reinforced by considering the solution satisfying the constraint and the cost.

[0025] In an embodiment, the supervised learning combinatorial optimization model can be trained to maximize compensation for the cost.

[0026] In an embodiment, the parameters of the supervised learning combinatorial optimization model can be learned by optimizing the second objective function so that the cost for a solution satisfying the above constraints is minimized.

[0027] In the embodiment, in the reinforcement learning step, reinforcement learning-based fine-tuning (RL fine-tuning) may be performed on the parameters of the supervised learning combinatorial optimization model to optimize the second objective function.

[0028] In the embodiment, in the reinforcement learning step, a learning instance may be newly generated from a distribution of a predetermined instance, or a learning instance may be sampled from the instance included in the learning data set.

[0029] A control method of a combinatorial optimization system according to the present invention may include a step of receiving an instance for a combinatorial optimization problem from a user terminal, a step of processing the instance as an input of a combinatorial optimization model learned through supervised learning and reinforcement learning, a step of obtaining an optimal solution for the instance from the combinatorial optimization model, and a step of providing the optimal solution to the user terminal.

[0030] A combinatorial optimization system according to the present invention, comprising a memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, specifies an instance for a combinatorial optimization problem and a learning data set including an optimal solution for the instance, performs supervised learning of a combinatorial optimization model using the learning data set, obtains a combinatorial optimization model supervised by the learning data, and performs reinforcement learning of the supervised-learned combinatorial optimization model.

[0031] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, and may include instructions for performing a step of specifying a learning data set including an instance for a combinatorial optimization problem and an optimal solution for the instance, a step of performing supervised learning of a combinatorial optimization model using the learning data set, a step of obtaining a combinatorial optimization model supervised by the learning data, and a step of performing reinforcement learning of the combinatorial optimization model supervised by the learning data.

[0032] As discussed above, the combinatorial optimization system and its control method according to the present invention, as well as the learning method of the combinatorial optimization system, can train a combinatorial optimization model using supervised learning and reinforcement learning. Through this, the combinatorial optimization model can generate high-quality solutions that minimize cost while satisfying constraints for various combinatorial optimization problem instances. In other words, the combinatorial optimization model according to the present invention can save computational resources and generate optimal solutions suitable for the problem, even for high-dimensional problems with complex solution space sizes and constraints.

[0033] Furthermore, as discussed above, the combinatorial optimization system and its control method according to the present invention, as well as the learning method of the combinatorial optimization system, can solve combinatorial optimization problems by reinforcing the combinatorial optimization model by reflecting the cost information of solutions that satisfy constraints, thereby constructing a combinatorial optimization model with high prediction accuracy and cost optimization performance. Through this, regardless of data quality and problem size, an optimal solution with high quality can be generated while minimizing costs through high optimization performance. In other words, the combinatorial optimization model according to the present invention can be universally utilized for various problem types and scales.

[0034] Furthermore, according to the combinatorial optimization system and its control method according to the present invention, and the learning method of the combinatorial optimization system, a supervised learned combinatorial optimization model can be transferred based on reinforcement learning, thereby achieving robust performance for new instances with different problem scales without separate additional labeling. Accordingly, the combinatorial optimization model according to the present invention can effectively respond to new problem instances even if training data containing high-quality optimal solutions does not exist, and can simultaneously secure versatility and scalability for various combinatorial optimization problems. In other words, the present invention improves computational efficiency and saves memory resources, while maintaining high performance without degradation of the model. Through this, the present invention can be applied and utilized in various fields and services such as operations research, logistics optimization, manufacturing and production planning, semiconductor and chip design automation (e.g., PCB design, circuit design, etc.), communications and network design, financial services, games, elevators, security (or patrol), hospitals, etc.

[0035] Figure 1 is a conceptual diagram for explaining a combination optimization system according to the present invention.

[0036] Figures 2a and 2b are conceptual diagrams for explaining a learning data set according to the present invention.

[0037] Figures 3a and 3b are flowcharts for explaining a learning method of a combinatorial optimization system according to the present invention.

[0038] FIG. 4, FIG. 5, FIG. 6, FIG. 7, FIG. 8, FIG. 9, FIG. 10, FIG. 11, FIG. 12, FIG. 13, and FIG. 14 are conceptual diagrams for explaining a learning method of a combination optimization system according to the present invention.

[0039] Figure 15 is a flowchart for explaining a control method of a combination optimization system according to the present invention.

[0040] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0041] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0042] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0043] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0044] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0045] The present invention provides a combinatorial optimization system capable of producing an optimal solution to a combinatorial optimization (CO) problem, a control method thereof, and a learning method for the combinatorial optimization system. More specifically, the present invention provides a combinatorial optimization model capable of producing an optimal solution that minimizes cost while satisfying constraints of a combinatorial optimization problem instance.

[0046] Combinatorial optimization is a branch of optimization that seeks to find a solution that satisfies an optimal objective value from a finite set of discrete solutions.

[0047] These combinatorial optimization problems have various characteristics. For example, the solution set of combinatorial optimization problems can be discrete (e.g., paths, sets, placements, matching, etc.). Furthermore, as the problem size increases, the solution space can grow exponentially. Furthermore, most combinatorial optimization problems can be classified as NP-hard or NP-complete. Furthermore, because the search target is not the entire solution space, but rather a subspace satisfying constraints, finding a valid solution can be challenging.

[0048] As an example, a combinatorial optimization problem can be understood as one that satisfies various conditions. The solution space may be discrete, and nodes and edges may be the basic units. Furthermore, constraints may be set (or provided) for problem instances, and the optimization of the objective function may be directed toward satisfying the constraints. Furthermore, the objective function may be quantified as a cost or value, and the goal may be minimization or maximization. Furthermore, the problem's solution space may be so large that brute-force search is practically impossible.

[0049] In combinatorial optimization problems, the solution space can be the set of all possible candidate solutions. When constraints are set, the actual search target may be limited to a subset of valid solutions that satisfy the constraints, rather than the entire solution space. The combinatorial optimization system according to the present invention can search for an optimal solution that achieves objective function optimization within the valid solution space that reflects the constraints.

[0050] These combinatorial optimization problems, which seek optimal solutions within discrete solution spaces and constraints, can encompass various types of problems, such as path optimization, set selection, and graph structure optimization. For example, the traveling salesman problem (JSP), a combinatorial optimization problem, can be defined as the problem of determining the order of cities visited by a traveling salesman, given a set of cities, so that each city is visited exactly once while achieving the shortest total travel distance. In the JSP, instances can represent the n cities to be visited. The solution for each instance can be represented as a matrix (binary matrix), and each element of the matrix can indicate whether or not a trip is made between specific cities. The valid solution space within the entire solution space can be the set of valid TSP paths that visit each city exactly once. In this case, the objective function represents the total length of the given path, which must be minimized. In other words, the target value of the JSP problem can be the total travel distance between cities.

[0051] For another example, in the Maximum Independent Set (MIS) problem, instances represent graphs, which can include a set of vertices (or nodes) and a set of edges (or edges). The solution space indicates whether each vertex is included in the solution set. To satisfy the independence condition, solutions must not simultaneously include vertices connected by edges. In this case, the objective function represents the total number of selected vertices, which must be maximized.

[0052] To this end, the present invention provides a method and system for effectively searching for an optimal solution that optimizes an objective function while satisfying constraints, using a combinatorial optimization model learned through supervised learning and reinforcement learning for such combinatorial optimization problems.

[0053] Meanwhile, the combinatorial optimization system and its control method, and the learning method of the combinatorial optimization system according to the present invention can be applied to various industries and services. For example, the present invention can be applied to and usefully utilized in a system (or application, software, website, program, etc.) based on at least one of an artificial neural network, a generative AI model (e.g., a diffusion model), and an artificial intelligence algorithm (e.g., a shortest path search algorithm, an algorithm related to combinatorial optimization, etc.).

[0054] The industries and services to which the present invention can be applied are diverse. The present invention can be applied and utilized in research to solve traveling salesman problems, vehicle routing problems, and maximum independent set problems. For example, the present invention can be applied and utilized in various fields and services, including operations research and logistics optimization, manufacturing and production planning, semiconductor and chip design automation (e.g., PCB design, circuit design, etc.), communications and network design, financial services, gaming, elevators, security (or patrol), and hospitals.

[0055] In this regard, industries and services to which the combinatorial optimization method and system according to the present invention can be applied will be briefly reviewed with reference to FIG. 1. The combinatorial optimization system according to FIG. 1 may include a combinatorial optimization model (or generative model), and by using the same, may generate an optimal solution (i.e., an optimal wiring path) that satisfies various constraints required for PCB design (e.g., a no-wiring zone, wiring width and spacing, wiring angle, multi-layer wiring, etc.), or may search for the shortest path for wiring terminals at the minimum cost (or expense) based on PCB data input by the user.

[0056] In one embodiment, the control unit (180) may receive PCB data (1010) including a net list (1011), terminal information (1012), and constraint information (1013) based on selection of a graphic object (1020) linked to a user input receiving function of a service page (1000) from a user terminal (10).

[0057] A net is a path that must be connected to transmit signals or supply power in a circuit. It can refer to a group of terminals (electrical contacts) that require electrical connection or exchange of signals. For example, if terminal 1 and terminal 2 are connected by wiring, they can be understood as belonging to the same net.

[0058] The net list (1011) may include a list (or set) of terminals that must be electrically connected. More specifically, the net list (1011) may include information defining the connection relationship between the terminals that must be electrically connected. The terminals defined in the net list (1011) may be core nodes that must be electrically connected in the PCB. For example, a user (or engineer) may group terminals that must be electrically connected among multiple terminals into a single net and input the grouped net group information into the net list (1011). In this case, “net 1: terminal 1, terminal 2” input in the net list (1011) may be interpreted to mean “the first terminal must be connected to the second terminal.”

[0059] Accordingly, the net list (1011) included in the PCB data (1010) may include net group information that groups terminals that must be electrically connected among a plurality of terminals and groups them into each net. That is, the net group information included in the net list (1011) may be understood as information that includes which terminals are grouped into one net and which connections must be made between the terminals included in each net.

[0060] In addition, the plurality of terminals included in the terminal information (1012) may actually mean electrical connection points (pins or nodes of electronic components) where wiring should be performed. Here, the term “electrical connection point” refers to an electrical contact point (start point and end point) where wiring begins or ends, and a terminal may refer to an electrical contact point where wiring begins or ends. An electrical contact point may serve to form a path for an electrical signal to travel to another component or circuit within a circuit. For example, it may i) allow current to be transmitted to an electronic device through the contact point, ii) allow power to be supplied from the contact point of a battery terminal (positive and negative), iii) provide a physical connection between electronic components to allow signals or power to flow, or iv) serve to connect pins of an IC chip with traces in a PCB design.

[0061] These terminals are points where electronic components such as resistors or transistors are mounted on the PCB, and the terminals can be expressed as component pins or electrical nodes on the PCB. For example, a specific pin of an integrated circuit (IC) chip, a contact point of a connector, an end point of a PCB (or circuit) trace, etc. can correspond to a terminal. That is, a plurality of terminals can correspond to electrical contact points where wiring starts or ends on the PCB defined by the PCB data (1010). In this case, the PCB data (1010) can include coordinate information for each of the plurality of terminals. At this time, when the PCB to be designed has a multi-layer structure, the PCB data (1010) can further include information about the layer on which each terminal is located.

[0062] Furthermore, the wiring constraints (1013) are conditions that must be observed in PCB wiring, and may include various conditions essential to ensure circuit performance and reliability. For example, the wiring constraints may include at least one of i) wiring width, ii) wiring spacing, iii) wiring prohibition areas (or zones), iv) 45-degree wiring, and v) wiring length constraints.

[0063] Line width can refer to the thickness (or thickness) of a wire. For example, there is a need to maintain a consistent line width to ensure safe current flow on a PCB.

[0064] Line spacing can refer to the minimum distance that must be maintained between different nets (wires). For example, if wires are too close together, signal interference or short circuits can occur, so maintaining a safe distance (or minimum distance) between different wires is necessary to prevent this.

[0065] A no-wiring zone may refer to an area where wiring cannot be placed (or connected) (i.e., an area where wiring cannot be drawn). For example, if a specific area is intended for electronic components, or is subject to electromagnetic interference (EMI) shielding or other significant restrictions, the specific area may be designated as a no-wiring zone.

[0066] A 45-degree wiring angle can refer to a restriction that requires wiring to be connected at a 45-degree angle rather than a right angle (90 degrees). For example, 45-degree wiring may be designed (or connected) at a 45-degree angle to reduce signal reflections and ensure smooth signal transmission, particularly for high-speed or sensitive signals.

[0067] Wire length constraints can refer to restrictions on the maximum length of the path connecting terminals. This is crucial for high-frequency devices, as signal amplitude can be reduced and noise amplified when propagating along long wires, potentially degrading circuit performance.

[0068] This PCB data (1010) may be data related to an instance of a combinatorial optimization problem. The goal in PCB design is to connect all terminals included in the netlist (1011) while satisfying all the constraints discussed above.

[0069] In this case, the control unit (180) can use the learned combinatorial optimization model (170) to conduct inference for PCB design, which is one of the combinatorial optimization problems. The learned combinatorial optimization model (170) can generate an optimal solution (or optimal path) that satisfies constraints and routes terminals (nodes) defined in the net list at the minimum cost, based on an instance input by the user, and provide the generated optimal solution to the user terminal (10).

[0070] Meanwhile, the combinatorial optimization system according to the present invention includes a combinatorial optimization model, and the present invention aims to provide a combinatorial optimization model that can be universally utilized in combinatorial optimization problems of various sizes.

[0071] Hereinafter, the present invention will be described in more detail with reference to the attached drawings. Fig. 1 is a conceptual diagram for explaining a combinatorial optimization system according to the present invention, and Figs. 2a and 2b are conceptual diagrams for explaining a learning data set according to the present invention. Figs. 3a and 3b are flowcharts for explaining a learning method of the combinatorial optimization system according to the present invention, and Figs. 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, and 14 are conceptual diagrams for explaining a learning method of the combinatorial optimization system according to the present invention. Furthermore, Fig. 15 is a flowchart for explaining a control method of the combinatorial optimization system according to the present invention.

[0072] Meanwhile, as illustrated in FIG. 1, the combination optimization system (100) according to the present invention may include at least one of an input unit (110), an output unit (120), a communication unit (130), a storage unit (140), a data generation unit (150), an optimal solution generation unit (160), a combination optimization model (170), and a control unit (180).

[0073] The combination optimization system (100) according to the present invention may include at least one processor and at least one memory containing computer program code. In this case, the memory may function as the storage unit (140) described above. In the present invention, the memory and the program code may cooperate with the processor to perform a series of processes described below.

[0074] Although not shown, the combinatorial optimization system (100) according to the present invention may include one or more processors, which may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., a digital signal processor, a tensor processing unit (TPU), a graphics processing unit (GPU), a neural network processing unit (NPU), an application-specific integrated circuit, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a quantum processing unit (or quantum processor, QPU), etc.). The one or more processors may be configured to execute instructions stored (or included) in the storage unit (140), computer-readable instructions, and / or other instructions described herein. The combinatorial optimization method and system according to the present invention may enable the memory and at least one processor to cooperate to perform data processing as described below. The processor may perform a series of operations and data processing using data and information stored in the memory. At this time, the memory may be a component of the storage unit (140).

[0075] In addition, the combinatorial optimization system (100) according to the present invention can perform data processing and calculation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into account implementation in a quantum computer environment. For example, the present invention can perform parallel operations based on qubits, and such quantum operations can operate complementarily with existing classical computers.

[0076] These quantum computers may include qubit-based parallel computing and high-speed data processing devices leveraging quantum entanglement, enabling hardware-based computational optimization using FPGAs and ASICs. Furthermore, quantum computers can utilize quantum processors capable of qubit-based parallel computing, and data processing efficiency can be improved through hybrid architectures with existing classical computers.

[0077] Meanwhile, the input unit (110) may be configured as a means for data input and may be configured in various types. For example, the input unit (110) may be configured to receive user input. The input unit (110) may be configured to receive user input from a user terminal (10). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on the input being made by the user through the input unit configuration provided in the user terminal (10).

[0078] In addition, in the present invention, the input unit (110) does not necessarily mean a hardware means, and can be understood as a passage for receiving input from a user.

[0079] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, computer mouse, keyboard, keypad, touch pad, trackball, joystick, voice recognition module, or other similar device. However, the present invention does not limit the type of input unit (110).

[0080] Here, the user input may include documents, text, images (or videos), voice, etc. In this case, the combination optimization system (100) may further include a module that converts voice into text.

[0081] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal (10) linked with the combination optimization system (100) according to the present invention. For example, the output unit (120) can output a page (or service page, 1000) linked with the combination optimization system (100) according to the present invention to the display unit of the user terminal (10). In addition, the output unit (120) does not necessarily mean a hardware means, and can be understood as a passage for outputting results to the user.

[0082] Next, the communication unit (130) may be connected to a user terminal (10), a server (e.g., a central server, an external server, etc.), a device, and at least one network through a wireless or wired network, and may be configured to receive or transmit overall data and information necessary for the operation of the combination optimization system (100) according to the present invention.

[0083] Here, the user terminal (10) may include at least one of a mobile phone, a smart phone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head mounted display (HMD)).

[0084] Furthermore, the communication unit (130) can support various communication methods according to the communication standards of the communicating device.

[0085] For example, the communication unit (130) may be configured to communicate with a communication target using at least one of WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0086] Next, the storage unit (140, or memory) serves to store various data related to the present invention, and may include one or more non-transitory computer-readable storage media that can be read and / or accessed by at least one of the one or more processors.

[0087] The one or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage (140) may be implemented using a single physical device (e.g., a single optical, magnetic, organic, or other memory or disk storage device), while in other examples, the storage (140) may be implemented using two or more physical devices.

[0088] The storage unit (140) may include computer-readable instructions and additional data. The storage unit (140) may include storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the devices and networks.

[0089] Furthermore, at least a portion of the storage unit (140) may be a cloud storage or a cloud server. The storage unit (140) may store at least a portion of data corresponding to user input received from the input unit (110) and learning data (or learning data set (200)).

[0090] That is, it can be understood that the storage unit (140) is sufficient as a space in which information necessary for the operation of the combination optimization system (100) according to the present invention is stored, and there are no restrictions on the physical space.

[0091] Furthermore, the storage unit (140) may store a computer program including computer program instructions. Furthermore, the storage unit (140) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (180) when loaded onto the processor of the system (100).

[0092] Next, the data generation unit (150) may be configured to generate data (instances) required for learning the combinatorial optimization model (170) according to the present invention. The data generation unit (150) may be configured to generate various types (or types) of graphs that include (or are configured, made up of) nodes and edges. To this end, the data generation unit (150) may include at least one model that generates different types of graphs composed of nodes and edges. For example, the data generation unit (150) may include at least one of the first model (151), the second model (152), the third model (153), and the fourth model (154). In the present invention, the data generation unit (150) may also be referred to as a “graph generation unit.”

[0093] The first model (151) may also be called an “ER (Erdos-Renyi) model” and may be a model that generates a first type of graph among a plurality of preset graph types. For example, as illustrated in (a) of FIG. 2B, the first model (151) may generate a first type of graph (221) by connecting edges with the same probability for all pairs of nodes in the graph. The first type of graph (221) may include an ER graph. This first type of graph (221) is a type of random graph, and the probability that an edge exists between any pair of nodes may be “r”. In order to generate the first type of graph (221), the data generation unit (150) may sample the edge generation probability from a uniform distribution and use the sampled value as a parameter of the first model (151).

[0094] The second model (152) may also be called a “Random-Regular (RR) model” and may be a model that generates a second type of graph among a plurality of preset graph types. For example, as illustrated in (b) of FIG. 2, the second model (152) may generate a second type of graph (222) in which all nodes of the graph have the same degree. The second type of graph (222) may include a random regular (RR) graph. This second type of graph (222) is a type of random graph and may have the characteristic that all nodes have the same degree. The data generation unit (150) sets all nodes to have the same number of neighbors in order to generate a second type of graph (222). At this time, the number of neighbors is sampled from “Uniform({3,4,5})”, which is a uniform (or equal) distribution of the degree of the graph, and the sampled degree value can be used as a parameter of the second model (152).

[0095] The third model (153) may also be called the “WS (Watts-Strogatz) model” and may be a model that generates a third type of graph among a plurality of preset graph types. For example, as illustrated in (c) of FIG. 2, the third model (153) may generate a third type of graph (223) having small-world properties including both minimum path length and high clustering. The third type of graph (223) may include a WS graph. This third type of graph (223) is a graph generated to simulate (or imitate) small-world networks, and provides a balance between regular connections and random connections, thereby reproducing the small-world effect and clustering observed in the real world. The third model (153) can sample the average node degree from the uniform distribution “Uniform({3,4,5,6})” and the rewiring probability from “Uniform(0, 1)” to generate the third type of graph (223).

[0096] The fourth model (154) may also be called a “grid model” and may be a model that generates a fourth type of graph among a plurality of preset graph types. For example, as illustrated in (d) of FIG. 2, the fourth model (224) may generate a fourth type of graph (224) in which all nodes are arranged in a grid shape. The fourth type of graph (224) may include a grid or a lattice graph. This fourth type of graph (224) may be understood as a graph having a structure in which nodes and edges are arranged in a grid shape. When generating the fourth type of graph (224), the fourth model (154) may sample a combination of width and height for the input number of nodes (n) and set it to be a product of n. For example, when n = 20, the fourth model (154) can generate a graph by sampling from combinations such as 4X5 or 5X4, and ensuring that one dimension is greater than a preset value (e.g., 4). In this case, unlike other graph types, for the graph-based instance of the fourth type, the edge cost can be set to a specific fixed value (e.g., 1). That is, the fourth type graph (224) generated from the fourth model (154) among multiple graph types (221, 222, 223, 224) can be generated with the cost of the edge fixed. This can be understood to be to ensure that the instance has properties similar to those searched (or discovered) in a Euclidean space.

[0097] However, in the present invention, the models included in the data generation unit (150, or combinatorial optimization system (100)) for generating multiple graph types are not necessarily limited to the mentioned models, and may further include other models in addition to the first model (151), second model (152), third model (153), and fourth model (154) above. It goes without saying that the models applied in the present invention may be variously changed and / or set to one or more depending on the purpose of use or situation (or case).

[0098] Next, the optimal solution generation unit (160, or optimal solution generation unit, optimal solution calculation unit, optimal solution calculation unit, etc. 160) may be configured to perform the role of generating (or calculating, calculating, etc.) an optimal solution (or optimal solution) of an instance for a Steiner tree problem (or combinatorial optimization problem).

[0099] The optimal solution generation unit (160) may include a mixed-integer linear programming (MILP)-based SCIP-Jack solver for calculating an optimal solution. Mixed-integer linear programming is an extension of linear programming and can be understood as a method of modeling and solving a problem by combining integer variables and real variables. The optimal solution generation unit (160) may calculate the optimal cost of an instance for a combinatorial optimization problem generated based on a graph, and may produce an optimal solution based on the calculated result. In addition, the optimal solution generation unit (160) may include at least one of a Concorde solver (e.g., a Concorde TSP solver) and an LKH-3 heuristic solver.

[0100] At least one of the optimal solutions for the learning instances included in the learning data set (200) according to the present invention may include an optimal solution generated from the optimal solution generation unit (160).

[0101] In one embodiment, as illustrated in FIG. 2A, a plurality of optimal solutions (201, 202, 203, 204) for a plurality of instances included in a learning data set (200) may be optimal solutions generated from an optimal solution generation unit (160). The optimal solution generation unit (160) may generate optimal solutions (201, 202, 203, 204) for each of the plurality of instances using at least one of a Concorde solver and an LKH-3 heuristic solver.

[0102] In another embodiment, the multiple optimal solutions (211, 212, 213, 214) for the multiple instances included in the learning data set (200) may be optimal solutions generated from the optimal solution generation unit (160). At this time, the multiple instances may be related to instances generated through multiple graphs (221, 222, 223, 224) having different types generated from the data generation unit (150). In this case, the optimal solution generation unit (160) may generate the optimal solutions (211, 212, 213, 214) for each of the instances generated using the multiple graphs (221, 222, 223, 224) using the SCIP-Jack solver.

[0103] Meanwhile, the combinatorial optimization model (170) can perform the role of producing an optimal solution for instances and instances for a combinatorial optimization problem.

[0104] In the present invention, the combinatorial optimization model (170) is a diffusion-based generative model (e.g., diffusion model), which learns the distribution of high-quality solutions in the solution space of a combinatorial optimization problem and can generate solutions for new problem instances based on the learned distribution. In this case, the combinatorial optimization model (170) can also be understood as performing the role of a solver that solves the combinatorial optimization problem. In the present invention, the combinatorial optimization model (170) can also be referred to as a “generative model,” a “DIFUSCO model,” a “CADO model,” a “diffusion (or diffusion) model,” a “combinatory optimization solver,” etc.

[0105] In the present invention, an anisotropic graph neural network (GNN) with edge gating applied can be used as the backbone network of the combinatorial optimization model (170). The combinatorial optimization model (170) considers the characteristics of nodes and edges in each layer (see (a) to (c) of FIG. 13). In addition, the sine wave-based time series characteristics corresponding to the restoration time step can be represented as in (d) of FIG. 13. Thereafter, the characteristics are transferred to the next layer through an anisotropic message transfer mechanism. This can be represented as in (e) of FIG. 13.

[0106] Here, learnable parameters in each layer can be represented as in (f) of Fig. 13, and the ReLU activation function can be represented as in (g) of Fig. 13. In addition, batch normalization can be represented as in (h) of Fig. 13, and the aggregation function implemented through SUM polling can be represented as in (i) of Fig. 13. In addition, the sigmoid function can be represented as in (a) of Fig. 14, and the Hadamard product can be represented as in (b) of Fig. 14. Furthermore, the set of neighboring nodes of node (i) can be represented as in (c) of Fig. 14, and the multilayer perceptron can be represented as in (d) of Fig. 14.

[0107] In one embodiment, for the traveling salesman problem, the initial edge features are derived from the state at time t (see (e) and (f) of Figure 14), and the initial node features can be initialized with the sinusoidal features of the nodes (see (g) of Figure 14). Alternatively, for the maximum independent set problem, the edge features can be initialized to 0, and the node features can be set to binary values ​​corresponding to the state at time t. Then, a classification or regression head is applied, and for classification, two neurons can be used for each node and edge, and for regression, one neuron. Finally, the final embedding for the state at time t (see (h) of Figure 14) can be used according to the discrete and continuous diffusion models, respectively.

[0108] This combinatorial optimization model (170) can be supervised learning (SL) to generate solutions of a similar type to the learning data set (200) by using a learning data set (200) containing solutions to combinatorial optimization problems. In this case, in the present invention, the supervised learning combinatorial optimization model can be additionally trained through reinforcement learning (RL).

[0109] More specific details about learning the combinatorial optimization model (170) will be described later.

[0110] Next, the control unit (180) may perform a role of controlling the overall operation of the combination optimization system (100) related to the present invention. The control unit (180) may process signals, data, information, etc. input or output through the components of the combination optimization system (100) discussed above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (180) may be physically implemented by the processor discussed above.

[0111] Meanwhile, the present invention provides a combinatorial optimization system capable of producing an optimal solution for a combinatorial optimization (CO) problem, a control method thereof, and a learning method for the combinatorial optimization system. More specifically, the present invention provides a combinatorial optimization model capable of producing an optimal solution that minimizes cost while satisfying constraints of a combinatorial optimization problem instance. In the following, the learning method for the combinatorial optimization model will be examined in more detail.

[0112] Meanwhile, in the present invention, a learning data set including an instance for a combinatorial optimization problem and an optimal solution for the instance is specified (S310), and a process of supervised learning of a combinatorial optimization model using the learning data set can be performed (S320, see FIG. 3).

[0113] The control unit (180) can specify a learning data set to be used for training the combination optimization model (170). In the present invention, the criteria (or method) for specifying the learning data set may vary. For example, the control unit (180) can specify a learning data set (200) stored in the storage unit (140, or memory) as the learning data set (200) to be used for training the combination optimization model (170), or can specify data according to user input input from the user terminal (10) as the learning data.

[0114] In one embodiment, the control unit (180) may specify a learning data set (200) stored in the storage unit (140, or memory) as learning data to be used for learning a combinatorial optimization model. As described above, the learning data set (200) may be configured to include at least one instance of a combinatorial optimization problem and an optimal solution for the instance.

[0115] For example, the learning data set (200) may include multiple instances and optimal solutions (201, 202, 203, 204) for each of the multiple instances. In this case, the multiple instances and optimal solutions included in the learning data set (200) may be collected from various sources (e.g., web crawling, a server linked to the combinatorial optimization system (100), an external server, etc.) (see FIG. 2a).

[0116] For another example, at least some of the multiple instances and multiple optimal solutions included in the learning data set (200) may include optimal solutions generated from an instance optimal solution generation unit (160) generated through a data generation unit (150). At this time, the instances generated through the data generation unit (150) may be related to multiple graphs (221, 222, 223, 224) having different types. In this case, the optimal solution generation unit (160) may generate optimal solutions (211, 212, 213, 214) for each of the instances generated using the multiple graphs (221, 222, 223, 224) (see FIGS. 2a and 2b).

[0117] Meanwhile, the control unit (180) can supervise learning of a combination optimization model (170) using a specific learning data set (200).

[0118] In relation to the combinatorial optimization problem, the set of all instances included in the learning data set (200) may be represented as in (a) of FIG. 6, and one specific instance may be represented as in (b) of FIG. 6. Each instance (g) included in the learning data set (200) includes a discrete solution space and an objective function (see (b) of FIG. 6), and for each solution (see (c) of FIG. 6), it may be defined as in (d) of FIG. 6. Here, cost represents a cost value to be optimized (see (e) of FIG. 6), and valid may be a function indicating whether a constraint is satisfied (see (f) of FIG. 6). That is, if the solution (x) belongs to the valid solution space, the value of the function indicating whether the constraint is satisfied is 0, and if the solution does not belong to the valid solution space, the value of the function may be infinite.

[0119] The goal of combinatorial optimization is to derive an optimal solution for an input (or given) instance (g) of a combinatorial optimization model (170). The optimal solution for a combinatorial optimization problem instance can be represented as shown in (g) of Fig. 6. In the present invention, the optimal solution may also be referred to as an "optimal solution," "correct solution," "high-quality solution," "correct solution," or "high-quality solution."

[0120] For supervised learning of the combinatorial optimization model (170), the control unit (180) can input the learning data set (200) into the combinatorial optimization model (170) (S351, see FIG. 3b). The control unit (180) can process at least one instance included in the learning data set (200) and an optimal solution for the instance as inputs to the combinatorial optimization model (170). In this case, the instance input to the combinatorial optimization model (170) can be input in the form of a binary matrix (see FIG. 4). This binary matrix can be converted to correspond to each of a plurality of instances and included in the learning data set (200), or the control unit (180) itself can convert a specific instance into a binary matrix and input it into the combinatorial optimization model (170).

[0121] In the supervised learning process (or step), the combinatorial optimization model (170) can assume the availability of an optimal solution for each instance (or learning instance, see (a) of FIG. 7) included in the learning data set (200). At this time, the distribution of the combinatorial optimization instances can be represented as in (b) of FIG. 7.

[0122] The goal of supervised learning is to specify (or determine) parameters that simulate (or approximate) the conditional distribution of optimal solutions for a learning instance using a combinatorial optimization model (170). The combinatorial optimization model (170) maximizes likelihood (or likelihood) through an objective function, which can be represented as shown in (d) of FIG. 7 .

[0123] Here, the conditional distribution may refer to a distribution (or probability distribution) from which an optimal solution is generated (or sampled, output, etc.) when an instance related to a combinatorial optimization problem is given. This may mean learning the parameters of the combinatorial optimization model (170) so that the distribution (or probability distribution, generation distribution, sampling distribution, prediction distribution, etc.) of the combinatorial optimization model (170) becomes close to (or similar to) the actual data distribution, i.e., the distribution of the optimal solution corresponding to the correct answer.

[0124] That is, the control unit (180) can maximize the probability distribution of the optimal solution being generated from the combinatorial optimization model (170). As another example, the control unit (180) can minimize the difference between the probability distribution of the optimal solution being generated from the combinatorial optimization model (170) and the distribution of the optimal solution included in the learning data set (200).

[0125] The control unit (180) can supervise learning of a combinatorial optimization model to approximate (or simulate) the conditional distribution of optimal solutions for instances included in the learning data set (200) (S353, see FIG. 3b). More specifically, the control unit (180) can learn (or adjust) the parameters of the combinatorial optimization model to approximate the conditional distribution (or conditional probability distribution) of optimal solutions for instances into which the combinatorial optimization model (170) is input.

[0126] As discussed above, the combinatorial optimization model (170) according to the present invention may include a diffusion-based generative model. The control unit (180) may learn the combinatorial optimization model through a diffusion process. The optimal solution may be a discrete space (Fig.

[0127] This diffusion process may include a forward noise addition process and a backward denoising process. In the diffusion process, the control unit (180) may learn the parameters of the combinatorial optimization model (170) so that the probability distribution of the solutions sampled (or generated, predicted) by the model approximates the conditional distribution of the optimal solution. The parameters of the combinatorial optimization model (170) may be adjusted so as to well approximate the distribution of the optimal solution. In the diffusion process (e.g., the backward denoising process), the control unit (180) may train the combinatorial optimization model (170) so that the combinatorial optimization model (170) well approximates the distribution of the optimal solution at each time step. That is, in the diffusion process, the combinatorial optimization model (170) can be understood as gradually performing restoration from a solution with added noise, and adjusting the probability distribution of each stage to be close to the distribution of the optimal solution (i.e., minimizing the difference between the distribution of the optimal solution and the distribution of the solution restored by the model) in order to ultimately generate a solution similar to the optimal solution.

[0128] Below, we will examine the diffusion process that occurs during supervised learning in more detail.

[0129] As illustrated in FIGS. 4 and 5, a specific instance can be input into the combinatorial optimization model (170) in the form of a binary matrix. In the binary matrix, each row and column represents a node, and the value can indicate whether or not it is connected (e.g., 1 = connected, 0 = not connected).

[0130] Since the optimal solution (410) is contained in a discrete space (see (a) of Fig. 8), the forward noise addition process and the backward restoration process can be performed on the same ideal space. Here, the forward noise addition process can be represented as in (h) of Fig. 7, and the backward restoration process can be represented as in (i) of Fig. 7.

[0131] The combinatorial optimization model (170) can generate (or sample) solutions with added noise by incrementally adding noise to the optimal solution through a forward noise addition process. More specifically, the combinatorial optimization model (170) can generate a sequence of latent variables (e.g., 401, 402, 403, etc.) by incrementally adding noise to the initial solution. The initial solution can be represented as in (b) of FIG. 8, and the sequence of latent variables can be represented as in (c) of FIG. 8. In combinatorial optimization, the initial solution follows the optimal solution for a given instance, which can be represented as in (d) of FIG.

[0132] These sequences are a succession of solutions to which noise has been progressively added, and for example, the second sequence (402, or the second solution to which noise has been added) may contain more noise than the first sequence (403, or the first solution to which noise has been added), and the third sequence (401, or the third solution to which noise has been added) may contain more noise than the first sequence (403) and the second sequence (402).

[0133] Additionally, the fully noise-added solution (401) at the final (or last) time point (T) of the forward noise addition process becomes a Bernoulli random variable and can follow the probability of a specific value (see (e) of Fig. 8). At this time, each variable is independent, and this can be represented as in (f) of Fig. 8.

[0134] The forward noise addition process discussed above can be represented as shown in (g) of Fig. 8. Here, the initial state is sampled from a probability distribution defined according to the problem instance, and the probability distribution of the latent variable sequence (or noise sequence) generated from the initial solution can be represented as the product of the probabilities (transition probabilities) of generating the next state from the previous state at each time point (t).

[0135] Furthermore, the combinatorial optimization model (170) can be trained to restore the solutions (401, 402, 403) with added noise to a solution approximate to the optimal solution (410) by gradually removing the noise contained in the solutions (401, 402, 403) through a backward restoration process. The combinatorial optimization model (170) can gradually restore the original solution while predicting the previous step in each state. That is, the combinatorial optimization model (170) can learn the process of gradually removing the noise contained in the solutions (401, 402, 403) with added noise to restore the solution approximate to the optimal solution (410). This backward restoration process can be treated in the same way as action selection in reinforcement learning, which will be described later.

[0136] The reverse restoration process discussed above can be represented as shown in (h) of Fig. 8. The control unit (180) can adjust the parameters of the combinatorial optimization model (170) during the supervised learning process so that the combinatorial optimization model approximates the conditional distribution of the optimal solution from the solution with added noise.

[0137] More specifically, the control unit (180) can optimize the parameters of the combinatorial optimization model (170) through a diffusion process so that the combinatorial optimization model (170) can gradually add noise to the optimal solution and restore the solution with added noise to a solution that well approximates the distribution of the optimal solution.

[0138] The control unit (180) can train the combinatorial optimization model (170) in a direction in which the probability distribution (see (a) of FIG. 9) sampled (or generated) from the combinatorial optimization model (170) closely approximates the distribution of the optimal solution (see (b) of FIG. 9).

[0139] In this case, the control unit (180) can adjust the parameters of the combinatorial optimization model (170) by optimizing the first objective function for supervised learning to minimize the distribution difference between the solution (404) restored from the combinatorial optimization model (170) and the optimal solution (410). For example, the first objective function can be optimized by minimizing the upper bound of the variation of the negative log likelihood. This first objective function can be represented as in (c) of FIG. 9.

[0140] Meanwhile, in combinatorial optimization, each item of the optimal solution indicates whether to select a node or an edge, and when modeled with a Bernoulli distribution, each item can also be represented as a one-hot vector (see (e) of Fig. 11). Therefore, in the diffusion process, the optimal solution is converted into N one-hot vectors (see (f) of Fig. 11), and thereafter, a diffusion-based combinatorial optimization model (170) can be used.

[0141] Specifically, the transition process at each time (or time) step (t) can be represented as in (a) of Fig. 12. Here, the discrete category distribution for the probability vector (p) can be represented as in (b) of Fig. 12, and the transition probability matrix can be represented as in (c) of Fig. 12. In addition, the noise level at the time step (t) can be represented as in (d) of Fig. 12, and the margin distribution of the time step can be represented as in (e) of Fig. 12. Here, the cumulative transition matrix is ​​represented as in (f) of Fig. 12, and Bayes' theorem is applied to obtain a conditional probability distribution in the backward restoration process. This can be represented as in (g) of Fig. 12.

[0142] Furthermore, the combinatorial optimization model (170) trained to predict the optimal solution included in the learning data set (200) can calculate the posterior distribution using the predicted values ​​as surrogate values ​​for the optimal solution in the backward restoration process (see (h) of FIG. 12). This calculation can be represented as (i) of FIG. 12.

[0143] Meanwhile, in the present invention, the reverse restoration process of a supervised learning combinatorial optimization model (170) can be modeled as a Markov Decision Process (MDP) by reflecting a decoder. This Markov Decision Process can be defined as a tuple (see (a) of FIG. 10).

[0144] Here, the state within the state space includes the combinatorial optimization instance, the current step (or type step), and the current state, and can be represented as in (b) of Fig. 10. The action within the action space can be represented as selecting the state of the previous step, as in (c) of Fig. 10. The state transition probability is represented as in (d) of Fig. 10, and the initial state distribution can be represented as a state containing completely noisy, as in (e) of Fig. 10. The reward function can be represented as a cost criterion of a solution satisfying the constraints processed by the decoder, as in (f) of Fig. 10.

[0145] The goal of reinforcement learning may be to learn a policy to maximize the cumulative reward (see (h) of Figure 10). The policy is a probability distribution of a combinatorial optimization model (170), and the probability distribution may be determined by the model's parameters. For example, learning a policy can also be understood as learning the model's parameters. Maximizing the cumulative reward can be represented as shown in (i) of Figure 10. Here, the state-action sequence generated by the policy within the MDP can be represented as shown in (j) of Figure 10.

[0146] In the present invention, the backward restoration process in the diffusion process for combinatorial optimization can be modeled (or formulated) as a Markov decision process. This can be represented as in (a) of Fig. 11. Here, Bern(p) samples the initial noise with a Bernoulli distribution in which each element independently follows probability (p) (see (b) of Fig. 11), and the Dirac delta distribution can be a Dirac delta distribution with a non-zero density only in y (see (c) of Fig. 11).

[0147] Additionally, the present invention can apply a policy gradient algorithm (e.g., policy slope, policy gradient, etc.) to optimize the iterative backward restoration process under a cost function. This can be represented as shown in (d) of Fig. 11. If the agent learns the proposed MDP in the correct direction, the combinatorial optimization model (170) can consider the impact of the solution post-processed by the decoder (e.g., a solution satisfying constraints) on the objective function. This will be described in more detail later.

[0148] Meanwhile, in the present invention, a supervised learning combinatorial optimization model may be obtained using learning data (S330), and a process of reinforcement learning the supervised learning combinatorial optimization model may be performed (S340, see FIG. 3).

[0149] In the invention, the control unit (180) can acquire a combinatorial optimization model (170) that has been supervised-learned using learning data (S355, see FIG. 3b). The control unit (180) can learn (or adjust) the parameters of the combinatorial optimization model (170) based on supervised learning to approximate the conditional distribution of the optimal solution for an instance, and can acquire the learned parameters as a result of the supervised learning. The parameters acquired through supervised learning can also be understood as optimal parameters, and the control unit (180) can specify (or determine) these optimal parameters as parameters to be used in reinforcement learning and set them as initial values.

[0150] In this regard, the control unit (180) can obtain a restored solution that approximates the optimal solution as a result of supervised learning of the combinatorial optimization model (170). For example, as illustrated in FIGS. 4 and 5 , the control unit (180) can obtain a restored solution (404) that approximates the optimal solution as a result of supervised learning through the diffusion process of the combinatorial optimization model. The restored solution can be represented as shown in (d) of FIG. 9 and can also be understood as a solution finally sampled in the diffusion process.

[0151] As discussed above, the valid solution space can be a much smaller subset of the entire solution space (see (e) of Figure 9). Here, the valid solution space is the set of solutions that satisfy constraints for a given instance, and these constraints can vary depending on the type of combinatorial optimization problem (e.g., traveling salesman problem, maximum independent set problem, etc.).

[0152] The control unit (180) can transform (or modify) the restored solution (404) into a solution (or valid solution) that satisfies preset constraints so that the restored solution (404) is included (or belongs) in the valid solution space. The control unit (180) can use a decoder to transform the restored solution (404) into a solution that satisfies preset constraints for the instance. In this case, the preset constraints can be set differently depending on the type (or definition) of the instance. For example, in the case of an instance related to the traveling salesman tour problem, the constraints can be set to visit each city exactly once and return to the starting city after the tour. As another example, in the case of an instance related to the maximum independent set problem, the constraints can be set to prevent selected pairs of vertices from being connected by edges (independent set condition).

[0153] The control unit (180) can process the restored solution (404) as input to the decoder. The decoder can transform the restored solution (404) into a solution (e.g., a feasible solution) that satisfies preset constraints so as to approximate the optimal solution. The decoder can transform the restored solution (404) into a solution that satisfies constraints within a range close to the optimal solution. This can be understood as transforming the restored solution (404) into a solution that satisfies constraints while maintaining the structure (or form) of the restored solution (404) as much as possible. Such a decoder is illustrated as in (f) of FIG. 9, and the solution that satisfies constraints transformed from the decoder can be illustrated as in (g) of FIG. 9.

[0154] And, the control unit (180) can obtain a solution that satisfies the constraints from the decoder, and use the solution that satisfies the constraints to reinforce the supervised learning combinatorial optimization model (170) (S357, see FIG. 3b).

[0155] The control unit (180) can calculate the cost for a solution that satisfies constraints using a cost function, and can calculate a reward for the cost using a reward function. The control unit (180) can provide (or grant) the reward calculated based on the cost for a solution that satisfies constraints to a supervised learning combinatorial optimization model (170).

[0156] In this case, a reward may be provided for a restored solution (404) obtained as a result of restoration of the supervised combinatorial optimization model (170). The control unit (180) may provide a reward only for the final time point corresponding to the time point at which a solution close to the optimal solution is restored during the backward restoration process of the supervised combinatorial optimization model (170). At this time, the parameter update of the supervised combinatorial optimization model (170) may be performed for each step of the backward restoration process including the final time point. The control unit (180) may update the parameters of the combinatorial optimization model (170) by multiplying the calculated reward value by the gradient (or slope, inclination) of each step of the backward restoration process. That is, the gradients of all steps of the backward restoration process are multiplied by the same reward value, so that the policy (probability distribution) of all steps can be updated. Therefore, the reward is applied equally to the policy of all steps of the backward restoration process, so that reinforcement learning of the supervised combinatorial optimization model (170) can be performed.

[0157] Furthermore, the control unit (180) can perform reinforcement learning on a supervised combinatorial optimization model by simultaneously considering solutions that satisfy constraints and costs. The supervised combinatorial optimization model (170) can be reinforced learned to maximize rewards for costs. The supervised combinatorial optimization model (170) can be reinforced learned to minimize costs for solutions that satisfy constraints while maximizing rewards.

[0158] The control unit (180) can learn the parameters of the supervised learning combinatorial optimization model (170) by optimizing the second objective function so that the cost for a solution satisfying constraints is minimized. In this case, the parameters of the supervised learning combinatorial optimization model (170) can be adjusted by maximizing the objective function so that the cost for a solution satisfying constraints is minimized while the reward for the cost is maximized.

[0159] In the reinforcement learning stage, parameter adjustment can be achieved through reinforcement learning-based fine-tuning (RL fine-tuning). That is, the control unit (180) can optimize the second objective function by performing reinforcement learning-based fine-tuning (RL fine-tuning) on ​​the parameters of the supervised-learned combinatorial optimization model. As discussed above, the objective in the combinatorial optimization problem can also be changed from minimizing the cost of the solution directly generated by the model (see (h) of FIG. 9) and the value indicating whether the constraints are satisfied (see (i) of FIG. 9) to minimizing the cost of the solution that satisfies the constraints converted from the decoder. This second objective function can be represented as shown in (i) of FIG. 9.

[0160] That is, in the past, if the solution generated by the model did not satisfy either the cost or constraints, learning itself was invalidated, making learning difficult. However, in the present invention, by converting the solution into one that satisfies the constraints through a decoder and minimizing the cost thereof, the model is trained in a good direction.

[0161] Meanwhile, in the reinforcement learning-based fine-tuning process according to the present invention, the combinatorial optimization system (100) can newly generate learning instances from a distribution of pre-specified instances or sample learning instances from instances included in a learning data set (200).

[0162] For example, a distribution of a given (or defined) instance is a set of probabilistic rules or generation conditions for generating problem instances of a specific combinatorial optimization problem, and such information may be stored and maintained in a storage unit (140, or memory). This distribution is used to control the structural characteristics, data format, size, and difficulty of the problem, and can serve as the basis for dynamically sampling training instances so that the model can generalize across various situations.

[0163] That is, in the present invention, by creating new instances that do not exist in the learning data set (200), the generalization performance of the combinatorial optimization model (170) is improved so that it can be well adapted to various problem types.

[0164] In addition, in the reinforcement learning-based fine-tuning process according to the present invention, the control unit (180) can apply various learning techniques (or methods) for efficient learning of the supervised learning combinatorial optimization model (170).

[0165] The control unit (180) can fix the first plural layers (11) in the architecture of the graph neural network, which is the backbone network of the supervised learning combinatorial optimization model (170), and update the parameters of only a specific layer (e.g., the last layer) among the plural layers. Additionally, the control unit (180) can selectively apply LoRA (Low-Rank Adaptation) to the remaining layers, excluding the specific layer, to perform fine-tuning.

[0166] Here, LoRA is a technique that models the parameter changes of each linear weight of the model (e.g., the weights (or weight matrices) of the linear layer of the model) as the product of multiple (two) low-rank matrices, assuming that the parameter changes due to model adaptation are low-rank. In other words, LoRA can mean a technique that models the change in the weight matrix as the product of two low-rank matrices, and updates the model through low-rank changes instead of adjusting the entire weight.

[0167] This LoRA can be selectively applied depending on the situation (or occasion). This allows the present invention to improve model learning speed and reduce memory usage. In particular, by selectively applying LoRA depending on the situation, the present invention can improve the performance of supervised learning models.

[0168] Meanwhile, the combinatorial optimization method and system according to the present invention, as described above, can provide the optimal solution for the instance input by the user to the user terminal (S1640) through the process of receiving an instance for a combinatorial optimization problem from a user terminal (S1610), the process of processing the instance as an input of a combinatorial optimization model learned through supervised learning and reinforcement learning (S1620), and the process of obtaining the optimal solution for the instance from the combinatorial optimization model (S1630), in the inference step, as illustrated in FIG. 15.

[0169] In this case, the combinatorial optimization model finally learned in the present invention (i.e., the combinatorial optimization model utilized in the inference process) may also be named “CADO (Cost-Aware Diffusion Solvers).”

[0170] In this regard, the combinatorial optimization model (170) learned according to the learning method described above can be applied and utilized to various combinatorial optimization problems.

[0171] First, the combinatorial optimization model (170) can be applied to a PCB (Printed Circuit Board) design optimization problem.

[0172] Specifically, between given nodes (or terminals), there may be pairs that must be connected (required connection pairs) and pairs that must not be connected (forbidden connection pairs), and all connection paths must maintain the shortest possible distance.

[0173] Additionally, overlap between wires must be minimized, and constraints may be imposed to limit the number of overlaps below a certain number of layers. For example, while overlap between wires is prohibited in a single layer, a multilayer structure may require a per-layer overlap limit.

[0174] In this case, the combinatorial optimization model (170) according to the present invention can generate an optimal wiring plan (or optimal wiring path (i.e., optimal solution)) that i) connects required connection pairs with the shortest distance, ii) prevents forbidden connection pairs from being connected, iii) satisfies constraints such that the overlap for each layer is within an allowable limit, and iv) minimizes the sum of the total wiring length. In this case, the instance input may include both connection requirements between nodes and wiring overlap constraints for each layer. In addition, it is possible to determine whether there is an intersection between wiring paths, and if the allowable range is exceeded, readjust the path to generate a final PCB wiring path that satisfies the constraints.

[0175] Furthermore, in relation to PCB design, the present invention can design a PCB by performing wiring in a direction similar to (or close to) human wiring through an image-based approach using a diffusion-based combinatorial optimization model (170), taking into account the high computational burden of the process of converting data into a graph.

[0176] Next, the combinatorial optimization model (170) can be applied to solve the traveling salesman problem (TSP) with multiple constraints.

[0177] As discussed above, the traveling salesman tour problem can be defined as the problem of determining the order of cities that a traveling salesman must visit, given a set of cities to be visited. In the traveling salesman tour problem, an instance can represent the n cities to be visited. The solution for an instance can be represented as a matrix (a binary matrix), with each element of the matrix indicating whether a trip is made between specific cities. The valid solution space within the entire solution space can be the set of valid TSP paths that visit every city exactly once. In this case, the objective function represents the total length of the given path, which must be minimized. In other words, the target value of the traveling salesman tour problem can be the total distance traveled between cities.

[0178] At this time, additional conditions may be set, including at least one of the following: when each city has a specific visitable date (for example, the first city (City A) can only be visited on Mondays or Wednesdays), and when certain pairs of cities cannot be visited consecutively (for example, a third city (City C) cannot be visited immediately after visiting a second city (City B).

[0179] Under these multiple constraints, the combinatorial optimization model (170) according to the present invention can produce (or generate) an optimal route (i.e., an optimal solution) that i) visits every city exactly once, ii) minimizes the total travel distance, iii) satisfies the constraint on the number of visitable days for each city in the visiting order, and iv) does not violate the constraint on prohibiting consecutive visits.

[0180] To this end, the combinatorial optimization system (100) according to the present invention can process as input additional constraint data, including information on the dates available for each city, in addition to a matrix indicating the availability and cost of travel between cities. Furthermore, the decoder can verify whether the generated route violates any constraints. If a violation is discovered through the verification results, the decoder can perform postprocessing to apply a possible modified route to generate a final solution that satisfies the constraints.

[0181] Furthermore, the combinatorial optimization model (170) learned in the present invention can be utilized in various operating environments.

[0182] For example, a learned combinatorial optimization model (170) can be stored on a server (central server) linked to a combinatorial optimization system (100). A user can transmit an instance to the server via a user terminal, and the server can calculate an optimal solution and provide it to the user terminal (10). The server can process large-scale problems at high speed using high-performance computing devices such as a GPU, TPU, or NPU.

[0183] For another example, the learned combinatorial optimization model (170) can be lightweight and directly embedded in smartphones, tablets, edge devices, and the like. Users can perform the combinatorial optimization process in real time on their local devices without a network connection. For device-specific optimization, the learned combinatorial optimization model (170) can apply pruning, quantization, and other techniques, taking into account memory capacity and computational performance.

[0184] For another example, a learned combinatorial optimization model (170) can be distributed via a third-party platform. The combinatorial optimization system (100) loads the model onto platforms such as AWS SageMaker, Azure ML, and Google Vertex AI, and enables real-time utilization of optimization results in various external systems via API calls.

[0185] In this way, the combination optimization system according to the present invention can be flexibly applied to various industrial fields and service environments.

[0186] Meanwhile, as discussed above, the combinatorial optimization model (170) may be configured to include an anisotropic graph neural network as a backbone network. Furthermore, the combinatorial optimization model (170) may include at least one of a node feature embedding layer, an edge feature embedding layer, a diffusion layer (or a diffusion process layer), and a decoder.

[0187] The node feature embedding layer and edge feature embedding layer each receive node and edge features and generate initial embeddings. Each node's features may include node type, coordinates, and connectivity, while edge features may include distance and connection weights.

[0188] Additionally, the diffusion process layer can play a role in gradually adding noise to the initial solution during the forward process and restoring the optimal solution from the state with added noise during the backward restoration process.

[0189] Additionally, anisotropic graph neural networks can model directed interactions between nodes by applying message passing techniques. In layers of anisotropic graph neural networks, node embeddings and edge embeddings can be updated simultaneously.

[0190] Additionally, the decoder can verify and correct the solutions generated for various combinatorial optimization problems according to their respective constraints (visitable dates, connection restrictions, overlap restrictions, etc.).

[0191] Furthermore, in the supervised learning (SL) phase, the combinatorial optimization model (170) can be trained to minimize the negative log-likelihood-based loss so as to approximate a conditional distribution similar to the optimal solution as much as possible. Subsequently, in the reinforcement learning (RL) phase, the combinatorial optimization model (170) can be trained to update parameters to maximize the cumulative reward through a policy gradient based on a reward function.

[0192] The learning framework of this combinatorial optimization model (170) can be performed based on PyTorch, TensorFlow, or JAX, and can also be extended to a quantum computing environment (QPU-based).

[0193] As discussed above, the combinatorial optimization system and its control method according to the present invention, as well as the learning method of the combinatorial optimization system, can train a combinatorial optimization model using supervised learning and reinforcement learning. Through this, the combinatorial optimization model can generate high-quality solutions that minimize cost while satisfying constraints for various combinatorial optimization problem instances. In other words, the combinatorial optimization model according to the present invention can save computational resources and generate optimal solutions suitable for the problem, even for high-dimensional problems with complex solution space sizes and constraints.

[0194] Furthermore, as discussed above, the combinatorial optimization system and its control method according to the present invention, as well as the learning method of the combinatorial optimization system, can solve combinatorial optimization problems by reinforcing the combinatorial optimization model by reflecting the cost information of solutions that satisfy constraints, thereby constructing a combinatorial optimization model with high prediction accuracy and cost optimization performance. Through this, regardless of data quality and problem size, an optimal solution with high quality can be generated while minimizing costs through high optimization performance. In other words, the combinatorial optimization model according to the present invention can be universally utilized for various problem types and scales.

[0195] Furthermore, according to the combinatorial optimization system and its control method according to the present invention, and the learning method of the combinatorial optimization system, a supervised learned combinatorial optimization model can be transferred based on reinforcement learning, thereby achieving robust performance for new instances with different problem scales without separate additional labeling. Accordingly, the combinatorial optimization model according to the present invention can effectively respond to new problem instances even if training data containing high-quality optimal solutions does not exist, and can simultaneously secure versatility and scalability for various combinatorial optimization problems. In other words, the present invention improves computational efficiency and saves memory resources, while maintaining high performance without degradation of the model. Through this, the present invention can be applied and utilized in various industries and services, such as operations research, logistics optimization, manufacturing and production planning, semiconductor and chip design automation (e.g., PCB design, circuit design, etc.), communications and network design, and financial services.

[0196] Furthermore, according to the combinatorial optimization system and its control method according to the present invention, and the learning method of the combinatorial optimization system, the supervised learning combinatorial optimization model can be transferred to various problem sizes based on reinforcement learning, so that it can be applied to new instances with different problem sizes without separate labeling. Accordingly, it can effectively respond to new problem instances without additional high-quality learning data, and simultaneously secure versatility and scalability for combinatorial optimization problems. Through this, the present invention can be applied and utilized in various fields and services such as operations research, logistics optimization, manufacturing and production planning, semiconductor and chip design automation (e.g., PCB design, circuit design, etc.), communications and network design, financial services, games, elevators, security (or patrol), hospitals, etc.

[0197] Meanwhile, the present invention discussed above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a quantum processor and quantum memory based on qubits, and may include software and hardware interfaces optimized for quantum computing.

[0198] Quantum processors in quantum computers utilize qubits (qubits) to efficiently perform complex calculations, such as parallel computation, quantum entanglement, and quantum superposition, that classical computers' binary bits cannot perform. Quantum processors process data using quantum gates, and can provide exponential speedups for certain problems.

[0199] Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a medium (or recording medium) that can be read by the computer.

[0200] Furthermore, the present invention discussed above can be implemented as computer-readable code or instructions on a program-recorded medium. In other words, the present invention can be provided in the form of a program.

[0201] Meanwhile, computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0202] Furthermore, the computer-readable medium may include a storage device and may be a server or cloud storage device accessible via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage device via wired or wireless communication.

[0203] A computer program can reach the system (100) via a variety of suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product tangibly embodying a computer program. The transmission mechanism may be a signal configured to reliably transmit a computer program over air or through an electrical connection. The system (100) may propagate or transmit a computer program as a computer data signal.

[0204] Furthermore, references to 'computer-readable storage media', 'computer program product', 'a computer program tangibly embodied', or the like, or to 'controller', 'computer', 'processor', etc., should be understood to include computers having various architectures, such as single / multiprocessor architectures and sequential (Von Neumann) / parallel architectures, as well as specialized circuits such as field-programmable gate arrays (FPGAs), application-specific circuits (ASICs), signal processing devices, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as the programmable contents of hardware devices, whether instructions for a processor or configuration settings for fixed-function devices, gate arrays, or programmable logic devices.

[0205] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, i.e., a CPU (Central Processing Unit), and there is no particular limitation on its type.

[0206] Meanwhile, the above detailed description should not be construed as limiting in any respect and should be considered illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. A computerized method comprising: A step of specifying a training data set including instances for a combinatorial optimization problem and optimal solutions for the instances; A step of supervised learning of a combinatorial optimization model using the above learning data set; A step of obtaining a combinatorial optimization model learned using the above learning data; and A learning method of a combinatorial optimization system, characterized in that it includes a step of performing reinforcement learning on the above-mentioned supervised learned combinatorial optimization model.

2. In paragraph 1, Learning the parameters of the combinatorial optimization model based on the supervised learning so as to approximate the conditional distribution of the optimal solution for the instances included in the learning data set, A learning method of a combinatorial optimization system characterized in that the combinatorial optimization model is reinforced by using the parameters of the combinatorial optimization model learned above.

3. In paragraph 2, A learning method for a combinatorial optimization system, characterized in that the parameters of the combinatorial optimization model are learned so that the probability distribution of solutions sampled from the combinatorial optimization model approximates the conditional distribution of the optimal solution.

4. In paragraph 1, In the above map learning step, The above combinatorial optimization model is learned through the diffusion process, The above diffusion process is, A learning method for a combinatorial optimization system, characterized by including a forward noise addition process and a backward denoising process.

5. In paragraph 4, The above combinatorial optimization model is, Through the above forward noise addition process, noise is gradually added to the optimal solution to generate solutions with added noise, A learning method of a combinatorial optimization system characterized in that, through the above backward restoration process, noise included in the solutions to which noise has been added is gradually removed to restore the solution to a solution close to the optimal solution.

6. In paragraph 5, The above combinatorial optimization model is, A learning method for a combinatorial optimization system characterized in that learning is performed to approximate the conditional distribution of the optimal solution from the solution to which the noise has been added.

7. In paragraph 6, A learning method of a combinatorial optimization system characterized in that the parameters of the combinatorial optimization model are learned by optimizing the first objective function to approximate the conditional distribution of the optimal solution from the solution to which the noise has been added.

8. In paragraph 5, As the restoration result of the above combinatorial optimization model, a restored solution is obtained that approximates the optimal solution, A learning method of a combinatorial optimization system characterized by using a decoder to convert the restored solution into a solution that satisfies preset constraints for the instance.

9. In paragraph 8, In the above reinforcement learning step, A learning method of a combinatorial optimization system characterized by performing reinforcement learning on the supervised learned combinatorial optimization model using a solution that satisfies the above constraints.

10. In paragraph 9, The above reinforcement learning step is, A step of calculating the cost for a solution that satisfies the above constraints using a cost function; A step of calculating a reward for the cost using a reward function; and A learning method of a combinatorial optimization system, characterized in that it comprises a step of providing the reward to the combinatorial optimization model learned above.

11. In paragraph 10, In the step of providing the above compensation, A learning method for a combinatorial optimization system, characterized in that the compensation is provided for the restored solution obtained as the restoration result of the combinatorial optimization model.

12. In paragraph 10, In the above reinforcement learning step, A learning method for a combinatorial optimization system characterized by performing reinforcement learning on the supervised learned combinatorial optimization model by considering the solution satisfying the above constraints and the above cost.

13. In paragraph 12, A learning method of a combinatorial optimization system, characterized in that the above-mentioned supervised learning combinatorial optimization model is learned to maximize compensation for the cost.

14. In paragraph 12, A learning method of a combinatorial optimization system characterized in that the parameters of the supervised learning combinatorial optimization model are learned by optimizing the second objective function so that the cost for a solution satisfying the above constraints is minimized.

15. In paragraph 14, In the above reinforcement learning step, A learning method of a combinatorial optimization system characterized in that the second objective function is optimized by performing reinforcement learning-based fine-tuning (RL fine-tuning) on ​​the parameters of the above-mentioned supervised learning combinatorial optimization model.

16. In paragraph 1, In the above reinforcement learning step, A learning method for a combinatorial optimization system characterized by newly generating learning instances from a distribution of previously specified instances or sampling learning instances from the instances included in the learning data set.

17. A step of receiving an instance for a combinatorial optimization problem from a user terminal; A step of processing the above instance as input to a combinatorial optimization model learned through supervised learning and reinforcement learning; A step of obtaining an optimal solution for the instance from the above combinatorial optimization model; and A control method of a combinatorial optimization system, characterized in that it includes a step of providing the above optimal solution to the user terminal.

18. A system comprising a memory configured to store executable instructions, and one or more processors configured to perform operations by executing one or more instructions, The above system, Specify a training data set containing instances of a combinatorial optimization problem and optimal solutions for said instances, Using the above learning data set, a combinatorial optimization model is supervised learning, Obtain a combinatorial optimization model that has been supervisedly learned using the above learning data, A combinatorial optimization system characterized by performing reinforcement learning on the above-mentioned supervised combinatorial optimization model.

19. A program executed by one or more processes in an electronic device and stored in a computer-readable recording medium, The above program is, A step of specifying a training data set including instances for a combinatorial optimization problem and optimal solutions for the instances; A step of supervised learning of a combinatorial optimization model using the above learning data set; A step of obtaining a combinatorial optimization model learned using the above learning data; and A program stored on a computer-readable recording medium, characterized in that it includes commands for performing a step of reinforcement learning of the above-mentioned supervised learning combinatorial optimization model.

Images