Graph neural network embedded management device, graph neural network embedded management method, and graph neural network embedded management system

Abstract

To enable utilizing highly accurate scheduling of GNN in an operating environment with limited time.SOLUTION: A GNN-embedded management apparatus includes: a data input unit for receiving an input dataset for defining a first event sequence and a second event sequence; a graph generation unit which generates an initial selection graph for defining a first node set for the first event sequence, a second node set for the second event sequence, and a first neighborhood information set; a graph correction unit for generating a revised selection graph by replacing the second node set with a first composite node when the second node set satisfies a structural similarity criterion; and a node embedding unit for generating an embedded graph in which the first neighborhood information set is embedded in each of the nodes.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates generally to managing information embedding for graph neural networks (GNNs), and more particularly to using graph neural networks for scheduling problems. [Background technology]

[0002] In recent years, reinforcement learning (RL) has become an important tool for solving scheduling problems of high complexity, and has proven to be superior to traditional approaches based on priority dispatching rules for scheduling problems.

[0003] For example, P. Junyoung et al. (Non-Patent Document 1) describe a technique in which RL is applied to job shop problems. More specifically, Non-Patent Document 1 states, "We propose a framework for learning to schedule job shop problems (JSSPs) using graph neural networks (GNNs) and reinforcement learning (RL). As a sequential decision-making problem, we formulate a scheduling process for JSSPs using a graph representation of states that considers the structure of the JSSPs. In solving the formulated problem, the proposed framework uses GNNs to learn node features that embed the spatial structure of the JSSPs represented as a graph (representation learning), and derive optimal scheduling policies that map the embedded node features to the best scheduling actions (policy learning). We train the above two modules end-to-end using a proximity policy optimization (PPO)-based RL strategy. We empirically demonstrate that the GNN scheduler outperforms execution-preferred dispatching rules and RL-based schedulers on various benchmark JSSPs due to its excellent generalization capability. We also confirm that the proposed framework learns transferable scheduling policies that can be used to schedule completely novel JSSPs (in terms of size and parameters) without further training." [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] P. Junyoung, C. Jaehyeong, K. Sang Hun, K. Youngkook, and P. Jinkyoo, "Learning to schedule job-shop problems: Representation and policy," CoRR, vol. abs / 2106.01086, 2021. Summary of the Invention [Problem to be solved by the invention]

[0005] Generally, in RL-based approaches, a scheduler (often an RL agent) learns a dispatching policy that maps the current state of the system to scheduling actions while taking into account the sequential nature of the problem. To determine the optimal scheduling action, it is important to provide the scheduler with a state representation that captures the maximum amount of information about the current state of the system.

[0006] Graph representation has proven to be an effective method to capture the underlying structure of scheduling scenarios, as it preserves the state relationships of the scheduled system in the form of neighborhood information. Graph neural networks (GNNs) have been used to efficiently embed the above neighborhood information in a structured manner, allowing the state embedded vectors to reflect the neighborhood information.

[0007] However, this embedding procedure is time-consuming, which is an obstacle for tasks that require a solution within a limited time frame, such as train rescheduling due to disruptions in the original schedule. Therefore, the time-consuming nature of the embedding procedure has limited the application of GNNs to dynamically changing environments such as train rescheduling.

[0008] In the technique disclosed in Non-Patent Document 1, for example, graph representation and RL are applied to the scheduling problem, but the entire selection graph is used for embedding by GNN, which leads to a long calculation time as mentioned above. Therefore, the technique disclosed in Non-Patent Document 1 is not suitable for application to scheduling problems in dynamically changing environments with strict time constraints, such as train rescheduling.

[0009] Therefore, in view of the above challenges, an approach is needed that can leverage the high-precision scheduling of GNNs in time-constrained dynamic environments.

[0010] Therefore, an objective of the present disclosure is to provide a graph neural network embedding management technique that makes GNNs applicable in dynamic environments by reducing the computation time for embedding neighborhood information. [Means for solving the problem]

[0011] A representative example of the present disclosure relates to a graph neural network embedding management device, comprising: a data input unit for receiving an input data set including a first event cycle defining a first event sequence and a second event sequence; a graph generation unit for generating an initial selection graph to represent the first event cycle, the initial selection graph defining a first node set corresponding to a first event set of the first event sequence, a second node set corresponding to a second event set of the second event sequence, and a first neighborhood information set defining a relationship between each node of the first node set and the second node set; a graph modification unit for generating a revised selection graph in which the second node set is replaced by a first composite node aggregating the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; and a node embedding unit for generating an embedded graph in which the first neighborhood information set is embedded in each node of the first node set and the second node set based on the revised selection graph. [Effects of the Invention]

[0012] According to the present disclosure, it is possible to provide a graph neural network embedding management technique that enables GNNs to be applied in dynamic environments by reducing the computation time for embedding neighborhood information.

[0013] Problems, configurations, and advantages other than those mentioned above will become apparent from the following description of embodiments for carrying out the present invention. [Brief explanation of the drawings]

[0014] [Figure 1]FIG. 1 is a block diagram illustrating an exemplary computing architecture for implementing embodiments of the present disclosure. [Figure 2] FIG. 2 shows a selection graph for a job shop problem. [Figure 3] FIG. 3 shows the embedding layer of a GNN. [Figure 4] FIG. 4 shows an example of a rail network including two trains and four rail sections. [Figure 5] FIG. 5 illustrates a framework for training a GNN-RL scheduler. [Figure 6] FIG. 6 shows a selection graph for the train scheduling problem. [Figure 7] FIG. 7 illustrates the progression of a choice graph through multiple states in response to choices made by an agent. [Figure 8] Figure 8 shows the choice graph for two trains running in parallel and two trains running in opposite directions in a railway network. [Figure 9] FIG. 9 is a choice graph for two trains making multiple trips between two ends of a rail network. [Figure 10] FIG. 10 illustrates an example of the application of the graph modification process according to the present disclosure in relation to train scheduling. [Figure 11] FIG. 11 illustrates an exemplary configuration of a graph neural network embedded management system according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a diagram showing an example of a set of neighborhood information embedded in a node of a revised selection graph. [Figure 13] FIG. 13 illustrates an exemplary flow diagram of a graph neural network embedding management method according to an embodiment of the present disclosure. [Figure 14] FIG. 14 is a diagram illustrating an exemplary operation of a graph updater according to an embodiment of the present disclosure. [Figure 15]FIG. 15 illustrates the first part of an example of the progression of a choice graph for a train scheduling scenario through six stages. [Figure 16] FIG. 16 illustrates the second part of an example of the progression of a selection graph for a train scheduling scenario through six stages. [Figure 17] FIG. 17 is a first graph illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure (illustrated by the lower line) and existing techniques (illustrated by the upper line). [Figure 18] FIG. 18 is a second graph illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure (illustrated by the lower line) and existing techniques (illustrated by the upper line). [Figure 19] FIG. 19 is a third graph illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure (illustrated by the lower line) and existing techniques (illustrated by the upper line). DETAILED DESCRIPTION OF THE INVENTION

[0015]

[0023] Herein, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described herein are not intended to limit the present invention according to the claims, and it should be understood that each and every element and combination of elements described with respect to the embodiments is not strictly necessary to practice aspects of the present invention.

[0016] Various aspects are disclosed in the following description and related drawings. Alternative aspects may be devised without departing from the scope of the present disclosure. Additionally, well-known elements of the present disclosure will not be described in detail or will be omitted so as not to obscure the relevant details of the present disclosure.

[0017] The words "exemplary" and / or "example" are used herein to mean "serving as an example, instance, or illustrative example." Any aspect described herein as "exemplary" and / or "example" is not necessarily to be construed as preferred or advantageous over other aspects. Likewise, the phrase "aspects of the present disclosure" does not require that all aspects of the present disclosure include the discussed feature, advantage, or characteristic of operation.

[0018] Furthermore, many aspects are described in terms of sequences of actions to be performed by, for example, elements of a computing device. It will be appreciated that various actions described herein can be performed by specific circuitry (e.g., an application specific integrated circuit (ASIC)), by program instructions executed by one or more processors, or a combination of both.

[0019] Additionally, the sequences of actions described herein can be considered to be embodied generally in any form of computer-readable storage medium storing a corresponding set of computer instructions that, when executed, cause an associated processor to perform the functions described herein. As such, various aspects of the present disclosure may be embodied in many different forms, all of which are contemplated to be within the subject matter set forth in the claims.

[0020] Referring now to the drawings, Figure 1 is a schematic block diagram of a computer system 100, according to an embodiment, for implementing various embodiments of the present disclosure. The mechanisms and devices of the various embodiments disclosed herein are equally applicable to any suitable computing system. The major components of computer system 100 include one or more processors 102, memory 104, terminal interface 112, storage interface 113, I / O (input / output) device interface 114, and network interface 115, all of which are communicatively coupled, directly or indirectly, for inter-component communication via memory bus 106, I / O bus 108, bus interface unit 109, and I / O bus interface unit 110.

[0021] Computer system 100 may include one or more general-purpose programmable central processing units (CPUs) 102A and 102B, generally referred to herein as processors 102. In embodiments, computer system 100 may include multiple processors, although in particular embodiments, computer system 100 may alternatively be a single CPU system. Each processor 102 executes instructions stored in memory 104 and may include one or more levels of on-board cache.

[0022] In embodiments, memory 104 may include random-access semiconductor memory, storage devices, or storage media (either volatile or nonvolatile) for storing or encoding data and programs. In particular embodiments, memory 104 represents the entire virtual memory of computer system 100 and may also include virtual memory of other computer systems coupled to computer system 100 or connected via a network. While memory 104 can conceptually be viewed as a single monolithic entity, in other embodiments, memory 104 is a more complex device, such as a hierarchy of caches and other memory elements. For example, memory may exist in multiple levels of caches, which may be further divided by function, whereby one cache holds instructions and another cache holds non-instruction data used by the processor. Memory may also be distributed and associated with different CPUs or sets of CPUs, as known in any of a variety of so-called non-uniform memory access (NUMA) computer architectures.

[0023] Memory 104 may store all or part of the various programs, modules, and data structures for handling data transfers described herein. For example, memory 104 may store graph neural network embedded management application 150. In embodiments, graph neural network embedded management application 150 may include instructions or statements that execute on, or are interpreted by, processor 102 to perform functions as described further below.

[0024] In particular embodiments, graph neural network embedded management application 150 is implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices in place of or in addition to a processor-based system. In embodiments, graph neural network embedded management application 150 may include data in addition to instructions or statements. In particular embodiments, a camera, sensor, or other data input device (not shown) may be provided in direct communication with bus interface unit 109, processor 102, or other hardware of computer system 100. In such a configuration, the need for processor 102 to access memory 104 and graph neural network embedded management application 150 may be reduced.

[0025] Computer system 100 may include a bus interface unit 109 that handles communication between processor 102, memory 104, display system 124, and I / O bus interface unit 110. I / O bus interface unit 110 may be coupled to I / O bus 108 to transfer data to and from various I / O units. I / O bus interface unit 110 communicates via I / O bus 108 with multiple I / O interface units 112, 113, 114, and 115, also known as I / O processors (IOPs) or I / O adapters (IOAs). Display system 124 may include a display controller, display memory, or both. The display controller may provide video, audio, or both types of data to display device 126. Additionally, computer system 100 may include one or more sensors or other devices configured to collect and provide data to processor 102.

[0026] By way of example, computer system 100 may include biometric sensors (e.g., collecting heart rate data, stress level data), environmental sensors (e.g., collecting humidity data, temperature data, pressure data), motion sensors (e.g., collecting acceleration data, movement data), etc. Other types of sensors are possible. Display memory may be dedicated memory for buffering video data. Display system 124 may be coupled to a display device 126, such as a standalone display screen, a computer monitor, a television, a tablet, or the display of a handheld device.

[0027] In one embodiment, the display device 126 may include one or more speakers for rendering audio. Alternatively, the one or more speakers for rendering audio may be coupled to the I / O interface unit. In an alternative embodiment, one or more of the functions provided by the display system 124 may be implemented in an integrated circuit that also includes the processor 102. Furthermore, one or more of the functions provided by the bus interface unit 109 may be implemented in an integrated circuit that also includes the processor 102.

[0028] The I / O interface unit supports communication with various storage devices and I / O devices. For example, the terminal interface unit 112 supports connection of one or more user I / O devices 116, which may include user output devices (such as a video display device, speakers, and / or a television receiver) and user input devices (such as a keyboard, mouse, keypad, touchpad, trackball, buttons, light pen, or other pointing device). A user may use a user interface to operate the user input devices to provide input data and commands to the user I / O devices 116 and the computer system 100, and may also receive output data via the user output devices. For example, the user interface may be presented via the user I / O devices 116, such as displayed on a display device, played through speakers, or printed by a printer.

[0029] Storage interface 113 supports the connection of one or more disk drives or direct access storage devices 117 (typically rotating magnetic disk drive storage devices, but alternatively other storage devices including arrays of disk drives or solid-state drives such as flash memory that appear as a single mass storage device to a host computer). In some embodiments, storage device 117 may be implemented by any type of secondary storage device. The contents of memory 104, or any portion thereof, may be stored in storage device 117 and retrieved from storage device 117 as needed. I / O device interface 114 provides an interface to any of a variety of other I / O devices or other types of devices, such as printers or fax machines. Network interface 115 provides one or more communication paths from computer system 100 to other digital devices and computer systems; these communication paths may include, for example, one or more networks 130.

[0030] 1 illustrates a particular bus structure providing direct communication paths between processor 102, memory 104, bus interface 109, display system 124, and I / O bus interface unit 110, in alternative embodiments, computer system 100 may include different buses or communication paths that may be configured in any of a variety of forms, such as hierarchical, star, or web configurations, multiple hierarchical buses, parallel and redundant paths, or point-to-point links in any other suitable type of configuration. Furthermore, while I / O bus interface unit 110 and I / O bus 108 are shown as separate respective components, computer system 100 may actually include multiple I / O bus interface units 110 and / or multiple I / O buses 108. While multiple I / O interface units are shown isolating I / O bus 108 from the various communication paths running to the various I / O devices, in other embodiments, some or all of the I / O devices are directly connected to one or more system I / O buses.

[0031] In various embodiments, computer system 100 is a multi-user mainframe computer system, a single-user system or server computer, or similar device with little or no direct user interface, but which receives requests from other computer systems (clients). In other embodiments, computer system 100 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable type of electronic device.

[0032] Next, the background and the problem solved by the present disclosure will be described with reference to FIGS.

[0033] Rail is an important and common means of transporting people and goods. Demand for rail services is increasing due to urban population growth. Current trends indicate that passenger and freight activity will more than double by 2050. Most conventional rail networks are currently located in North America, Europe, China, Russia, India, and Japan. These regions account for approximately 90% of global passenger travel on conventional rail, with India leading at 39%, followed by China at 27%, Japan at 11%, and the European Union at 9%.

[0034] Between 2018 and 2020, the market size was estimated at an average of 114 billion euros. The market is projected to grow at a compound annual growth rate of 2.6 percent, reaching an average annual valuation of 125 billion euros between 2021 and 2023. Railway infrastructure development is increasing significantly to meet this demand. This leads to the need for dense rail networks of stations, tracks, and trains to operate at higher frequencies to accommodate this demand. Therefore, efficient scheduling of the large number of trains on these networks becomes a significant challenge. This efficiency also affects passenger commute times, goods transport times, waiting times at stations, fuel use, and more. Scheduling techniques have been extensively studied in recent years in diverse fields such as job shops, workforces, warehouses, and transportation. The aim of scheduling is to improve process efficiency by optimizing key performance indicators such as makespan, waiting time, and costs.

[0035] The most common scheduling problem is the Job Shop Scheduling Problem (JSSP), which can be extended to encompass other areas of research in industry scheduling. The Train Scheduling Problem (TSP) is an extension of the JSSP, with additional blocking and no-wait constraints. This makes the train scheduling problem more complex than the traditional JSSP. The TSP has been extensively studied, and there are exact, heuristic, and hybrid methods, as well as the proposed learning-based method. Exact methods cannot be used for large-scale scheduling problems due to their time complexity.

[0036] Heuristic and hybrid methods are based on specific rules defined based on domain knowledge and provide a faster approach to achieving results. These are called priority dispatching rules (PDR). However, PDR ignores the sequential nature of the scheduling problem. Learning-based methods have shown superior performance compared to PDR. In reinforcement learning (RL) approaches, a scheduler (often an RL agent) learns a dispatching policy that maps the current state of the system to scheduling actions while taking into account the sequential nature of the problem.

[0037] However, these RL methods typically require instance-by-instance training to derive a schedule, making it difficult to apply the trained policy to novel scheduling problems. Graph representations have also been used to solve scheduling problems, such as the traveling salesman problem. Problem instances are represented as graphs, and embedding algorithms, such as graph neural networks (GNNs), are used to convert the graphs into vectors that outline the structural information of the problem. RL algorithms therefore use node embeddings to generate scheduling actions. Details of graph representations and node embeddings are provided in greater detail later in this specification.

[0038] The JSSP is one of the most well-known problems in the field of combinatorial optimization. A traditional n / m JSSP can be described as scheduling n jobs on m machines, with the objective of minimizing the makespan of all jobs. Each of the n jobs consists of several operations on m dedicated machines and requires a specific sequential processing time. Furthermore, a job's operations can only be performed on one machine at a time, and each job must be processed exactly once on each machine. The JSSP can be represented as a choice graph to facilitate the description of constraints and relationships between different operations and machines. Such a choice graph can be used to apply metaheuristics due to the ease with which complex relationships can be modeled.

[0039] An example of a job shop problem with three jobs and three machines is described with reference to Table 1 and Figure 2. Table 1 shows an example of a job shop problem with three jobs and three machines in tabular form. Figure 2 shows an example of a job shop problem with three jobs and three machines as a selection graph. [Table 1]

[0040] In Figure 2, O ij represents the operation of job i on machine j, and p ij represents the processing time of job i on machine j. A solid edge between two nodes is a conjunction edge, indicating a given priority of the job's operations. A dashed edge between two nodes is a disjunction edge, representing a machine-sharing requirement between the two operations. Note that these dashed edges are bidirectional, meaning that the scheduling order of the nodes connected by these edges is not known in advance. These edges become unidirectional once the operations are scheduled. Therefore, scheduling can be understood as determining the direction of the disjunction edge. The two nodes at the beginning and end are dummy nodes representing the start and end of the entire process. As will be explained later, selection graphs can also be used to model train scheduling problems.

[0041] The information in a graph is represented as node and neighborhood information. These nodes indicate the status of an ongoing schedule, and edges indicate the neighborhood relationships between nodes. Graph embedding is the process of converting the graph data of a selection graph into a vector format that outlines the information in the graph. Graph Neural Networks (GNNs) are a tool for embedding graph representations of systems. Generally, a GNN contains a series of K embedding layers that utilize the neighborhood information of nodes to generate the corresponding embedded format.

[0042] FIG. 3 shows an example configuration of the embedding layer of a GNN. τ (0) represents the graph of the transition τ before passing through the first embedding layer. τ (K) represents the graph of transitions τ after passing through K embedding layers. Nodes in this embedded graph are embedded with neighborhood information that describes their relationships with other nodes in the graph. This neighborhood information embedding process needs to be performed each time a new state is provided to an agent in the RL process. The embedding time depends on the number of nodes and edges in the input graph. Therefore, when the number of nodes and edges is large, the computation time required to perform this embedding process increases, which becomes a challenge in large-scale networks. Therefore, as described later in this specification, aspects of the present disclosure relate to providing a graph neural network embedding management technique to reduce the computation time required to embed neighborhood information, enabling the application of GNNs in large-scale dynamic environments.

[0043] Next, we consider an example in which the graph representation technique and the GNN framework are applied to the train scheduling problem, with reference to Figs. 4 to 6 .

[0044] FIG. 4 is a diagram illustrating an example of a rail network 400. Generally, a rail network includes trains traveling on tracks connecting intermediate points, such as stations. These tracks may be divided into sections. When reinforcement learning is used to generate a schedule for a rail network, a scheduler determines which trains to schedule in a particular section at a given time. It is desirable for the scheduler to provide an efficient schedule that optimizes KPIs, such as passenger waiting time or total travel time. As shown in FIG. 4, the rail network 400 includes two trains, Train 1 and Train 2, traveling in opposite directions across four sections a, b, c, and d. Additionally, the processing time for each section (i.e., the time it takes for the train to travel a particular section) is also shown (e.g., 8 units for sections a, b, and c, and 10 units for section d).

[0045] Next, a training framework 500 for training a GNN-RL scheduler 530 for the train scheduling problem will be described with reference to Figure 5. As shown in Figure 5, the GNN-RL scheduler 530 inputs a Markov decision process (MDP) equation 520 in which a train scheduling problem instance 510 is represented as a choice graph, and a reinforcement learning method 540, such as proximity policy optimization, is used to train the GNN-RL scheduler 530 to learn a scheduling policy for train scheduling.

[0046] As described herein, the train scheduling problem extends the JSSP by adding other constraints, such as blocking conditions, which mean that a section of the rail network is blocked from new trains being scheduled until one of the trains currently stationed there has been scheduled onto a new available section (e.g., two trains cannot simultaneously occupy the same section of rail track). The choice graph for the JSSP can be modified to account for the additional blocking constraints described above.

[0047] FIG. 6 shows a choice graph 600 for the train scheduling problem. In FIG. 6, S0 represents the initial state of the choice graph. Here, node a1 indicates the possibility of train 1 entering section a. Solid edges represent potential connections, and dashed edges, as disjunction edges, represent blocking constraints of the railway scheduling problem. S1 represents the state of the choice graph when an agent selects node a1. Here, the agent's selection of node a1 means that the agent has chosen to assign train 1 to section a at a given time. The selected node is highlighted in the figure.

[0048] Figure 7 illustrates the progression of a choice graph through multiple states in response to selections made by an agent. In an embodiment, an agent may alternate node selections for different trains in a rail network. As shown in Figure 7, when the agent selects node a1, the choice graph progresses from initial state S0 to S1. The choice graph then progresses through states S2 and S3 in response to selections of nodes d2, b1, and c2, respectively, to reach state S4.

[0049] 6 and 7 show selection graphs for a simplified rail network with four sections and two trains, in reality a rail network will include many more sections and trains, and these trains will make multiple trips on those sections on a single day. Thus, at a given time, there will be a mix of trains running in parallel and counter directions.

[0050] FIG. 8 shows a choice graph 825 for two trains running in opposite directions on a rail network having eight sections, and a choice graph 850 for two trains running in parallel. Herein, as shown in FIG. 8, nodes are identified using a notation having the format "section, train number, service number." As an example, in FIG. 8, "a11" is used for the node representing train 1 occupying section a in service 1. Similarly, "c21" represents train 2 occupying section c in service 1, for example. Furthermore, a service is defined herein as the movement of a train from one end of the rail network to the other (e.g., from section a to section h, or from section h to section a). It can be seen from FIG. 8 that choice graphs 825 and 850 are significantly more complex than the choice graphs shown for the four sections in FIGS. 6 and 7.

[0051] Figure 9 illustrates a selection graph for two trains making multiple trips between two ends of a rail network. In Figure 9, the upper selection graph set 925 shows trips made by Train 1, and the lower selection graph set 950 shows trips made by Train 2. Train 1, in its first trip, starts at section a and travels to section h. Train 2, in its first trip, starts at section h and travels to section a. When either of these trains reaches the last section of the rail network, it begins its next trip along the reverse order of the sections. Note that this new trip (i.e., the reverse trip) is also disjunctively connected to all of the trips of each of the other trains. As the number of trains and the number of trips per train increase, the complexity of the selection graph increases, which in turn increases the computation time for embedding neighborhood information. This poses a challenge in train rescheduling scenarios, especially when new schedules need to be generated quickly by a trained GNN-RL scheduler based on the selection graph representing the state of the rail network.

[0052] Thus, as described herein, aspects of the present disclosure relate to graph neural network embedding management techniques that enable the application of GNNs in dynamic environments by reducing the computation time for embedding neighborhood information. Graph neural network embedding management techniques according to the present disclosure will now be described with reference to Figures 10 through 19.

[0053] 9, consecutive train trips form a unit that repeats throughout the remainder of the graph. For example, Train 1 Trip 3 is structurally similar (e.g., recursive) to Train 1 Trip 1 and Train 2 Trip 2. Thus, aspects of the present disclosure relate to leveraging the recursive structure present in some scheduling scenarios to reduce the number of nodes and the resulting computation time for the selection graph embedding.

[0054] More specifically, aspects of the present disclosure relate to compressing nodes of a selection graph that have a nearly repetitive structure with respect to other sections of the selection graph into a single composite node. Here, compressing refers to grouping multiple nodes together. When applying this approach to train scheduling, only ongoing trips maintain the original number of nodes connected by the original conjunction and disjunction edges. Because all other trips (future and past) can be expressed as repeating units of those ongoing trips, those other trips can be compressed into a single node representing each trip and expanded as needed for scheduling options. Here, expanding refers to splitting or dividing a composite node into the individual nodes it contains.

[0055] FIG. 10 illustrates an example in which the graph modification process according to the present disclosure is applied to train scheduling. As is apparent from comparing FIG. 9 and FIG. 10, the eight nodes of Train 2 Trip 2 in the lower selection graph set 950 shown in FIG. 9 have been replaced by a single composite node 1010 in FIG. 10. Similarly, the remaining trips that are not currently in progress or available for scheduling (Train 1 Trip 2, Train 1 Trip 3, and Train 2 Trip 3) are represented by a single composite node. In this way, the number of nodes and the resulting computation time for embedding the selection graph can be reduced.

[0056] Next, a graph neural network embedded management system according to an embodiment of the present disclosure will be described with reference to FIG.

[0057] 11 is a diagram illustrating an exemplary configuration of a graph neural network embedding management system 1100 according to an embodiment of the present disclosure. As shown in FIG. 11, the graph neural network embedding management system 1100 according to an embodiment of the present disclosure basically includes a user terminal 1120 and a graph neural network embedding management device 1150. The graph neural network embedding management device 1150 and the user terminal 1120 may be connected via a communication network (not shown in FIG. 11) such as the Internet.

[0058] User terminal 1120 is a computing device usable by a user of graph neural network embedded manager 1150. In an embodiment, user terminal 1120 is usable to submit an input dataset 1122 describing the current state of the system for which schedule generation is desired, and to subsequently receive a schedule 1172 generated based on input dataset 1122.

[0059] Graph neural network embedded management device 1150 is a computing device for generating schedule 1172 based on input data set 1122. As shown in FIG. 11 , graph neural network embedded management device 1150 basically includes a data input unit 1151, a graph generation unit 1152, a graph update unit 1154, a graph correction unit 1156, a scheduling unit 1158, and a storage unit 1161. In an embodiment, graph neural network embedded management device 1150 may be implemented using computer system 100 shown in FIG. 1 . In this case, data input unit 1151, graph generation unit 1152, graph update unit 1154, graph correction unit 1156, scheduling unit 1158, and storage unit 1161 of graph neural network embedded management device 1150 may be implemented as software modules of graph neural network embedded management application 150 of computer system 100.

[0060] While aspects of embodiments of the present disclosure will now be described with reference to the example of train scheduling, it should be noted that the graph neural network embedded control techniques described herein are not so limited and may be suitably applied to a variety of tasks. For example, the neural network embedded control techniques described herein may be applied to managing the production of products in a factory environment in addition to train scheduling.

[0061] The data input unit 1151 is a functional unit for receiving input of an input data set 1122. The input data set 1122 may include data in a tabular or matrix format. In an embodiment, the input data set 1122 may include a first event cycle defining a first event sequence and a second event sequence, where the first event sequence corresponds to a first time frame and the second event sequence corresponds to a second time frame subsequent to the first event sequence. Here, the first event cycle refers to a series of events. The first event sequence refers to a first subset of events in the first event cycle, and the second event sequence refers to a second subset of events in the first event cycle.

[0062] As an example, if the input dataset 1122 includes data defining a rail network along with data indicating the current location, configuration, route, and destination of each train on the rail network, the progression of the train through different waypoints (e.g., stations) in a first trip may correspond to a first set of events, and the progression of the train through waypoints in a second trip may correspond to a second set of events.

[0063] The graph generator 1152 is a functional unit for generating an initial selection graph (G0) based on the input dataset 1122. Here, the initial selection graph may be a data structure in which a set of nodes and a set of edges are used to represent the progression of an element through a first event sequence defined by the input dataset 1122. More specifically, the initial selection graph may define a first node set corresponding to a first event set in the first event sequence, a second node set corresponding to a second event set in the second event sequence, and a first set of neighborhood information between each node in the first node set and the second node set. Here, the first set of neighborhood information may include a predecessor node, a successor node, a disjunction node, a parallel node, a parallel directional disjunction complex node, and a backward directional disjunction complex node for each node in the initial selection graph.

[0064] The graph update unit 1154 updates the scheduling action (a τ ) based on the effective scheduling action (A τ ) is the updated selection graph (G τ The detailed processing of the graph update unit 1154 will be described later in this specification, and therefore the description thereof will be omitted here.

[0065] The graph corrector 1156 is a functional unit for correcting a selection graph (e.g., an initial selection graph generated by the graph generator 1152 or an updated selection graph generated by the graph updater 1154). In an embodiment, the graph corrector 1156 corrects a revised selection graph (g) in which a second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set. τ ), where the structural similarity criteria may include rules or standards that define the structural similarity between the first set of nodes and the second set of nodes.

[0066] In embodiments, the graph corrector 1156 may determine that the second node set satisfies the structural similarity criterion with respect to the first node set if the second node set is a repetition (e.g., the second node set is a repetition in the same order as the first node set) or a reverse repetition of the first node set (e.g., the second node set is a repetition in the reverse order of the first node set). Here, a "composite node" refers to a single node used to represent multiple nodes in the selection graph while maintaining neighborhood information between the aggregate node and other nodes in the selection graph.

[0067] The scheduling unit 1158 is a functional unit for learning a policy 1160 for predicting scheduling actions based on the embedded graph. More specifically, as shown in FIG. 11 , the scheduling unit 1158 embeds neighborhood information for each node of the revised selection graph to generate a policy 1160 for predicting scheduling actions based on the embedded graph (g τ (K) Based on the embedded graph, the scheduling unit 1158 determines a scheduling action (a) for the current transition (τ) of the embedded graph. τ ) is learned. By aggregating the scheduling actions predicted by the policy 1160 for multiple transitions of the embedded graph, it is possible to generate a schedule 1172 indicating recommended scheduling actions for the system defined by the input dataset 1122. This schedule 1172 may be temporarily stored in the storage unit 1161 and transmitted to the user terminal 1120 via a communication network such as the Internet.

[0068] The graph neural network embedding management system 1100 described above can reduce the computation time for embedding inter-node information, thereby enabling the application of GNNs to dynamic environments.

[0069] Next, an example of neighborhood information embedded in the nodes of the revised selection graph will be described with reference to FIG.

[0070] FIG. 12 illustrates an example of a neighborhood information set 1200 embedded in a node of a revised selection graph. As described herein, neighborhood information set 1200 is information defining relationships between nodes in the selection graph, and is embedded in each node of the selection graph by node embedding unit 1159 shown in FIG. 11 to facilitate schedule generation. These relationships may define constraints on the progression of elements (e.g., trains) through nodes in the selection graph. For example, a disjunction node may be used to express a blocking condition that prevents multiple elements from being scheduled to the same node at the same time.

[0071] As shown in FIG. 12, a neighborhood information set 1200 may define a predecessor node 1205, a successor node 1210, a disjunction node 1215, a parallel operation 1220, a parallel directional disjunction complex node 1225, and a backward disjunction complex node 1230 for each node of the revised selection graph.

[0072] As described herein, when this neighborhood information set 1200 is embedded into each individual node of the revised selection graph, if the number of nodes and edges is large (e.g., a train network has a large number of stations and trains), the computational time for performing this embedding process increases, becoming a challenge in large-scale networks. Therefore, as described herein, aspects of the present disclosure relate to providing graph neural network embedding management techniques for replacing multiple nodes (e.g., nodes that have structural similarities with other nodes in the selection graph) with a single composite node, thereby reducing the number of nodes and thereby reducing the computational time for embedding neighborhood information. In this way, it is possible to enable the application of GNNs in large-scale dynamic environments.

[0073] Next, with reference to FIG. 13, an exemplary flow of a graph neural network embedding management method according to an embodiment of the present disclosure will be described.

[0074] 13 is a diagram illustrating an example flow of a graph neural network embedding management method 1300 according to an embodiment of the present disclosure. The graph neural network embedding management method 1300 is a method for generating a schedule for a given scenario, and may be executed by various functional units of the graph neural network embedding manager 1150 shown in FIG. 11.

[0075] Initially, in step S1310, the data input unit 1151 receives an input of an input data set. As described herein, the input data set may include table or matrix data defining a current state of the system for which it is desired to generate a schedule. In an embodiment, the input data set may include a first event cycle defining a first event sequence and a second event sequence, the first event sequence corresponding to a first time frame and the second event sequence corresponding to a second time frame subsequent to the first event sequence. In an embodiment, this input data set may be received from a user terminal. As an example, the input data set may define the configuration of a rail network, along with the location, route, and destination of each train in the rail network.

[0076] Next, in step S1320, the graph generation unit 1152 generates an initial selection graph having a first node set and a second node set. More specifically, the graph generation unit 1152 may generate, as the initial selection graph, a selection graph that defines a first node set corresponding to a first event set in the first event sequence, a second node set corresponding to a second event set in the second event sequence, and a first set of neighborhood information between each node in the first node set and the second node set. Here, the first neighborhood information set may include a predecessor node, a successor node, a disjunction node, a parallel node, a parallel directional disjunction complex node, and a backward directional disjunction complex node for each node in the initial selection graph.

[0077] Next, in step S1330, the graph update unit 1154 generates an updated selection graph indicating a set of valid scheduling actions for a particular transition in the selection graph based on the scheduling actions proposed by the scheduling unit 1158. Note that during the first iteration of the graph neural network embedding management method 1300, no scheduling actions have yet been proposed by the scheduling unit 1158, so the graph update unit 1154 does not generate an updated graph but passes the initial selection graph to the graph modification unit 1156.

[0078] Next, in step S1340, the graph modifier 1156 generates a revised selection graph in which sets of nodes that satisfy a structural similarity criterion with respect to other sets of nodes in the revised selection graph are replaced with composite nodes. In an embodiment, the graph modifier 1156 may generate the revised selection graph by classifying nodes in the selection graph as belonging to either active event sequences (e.g., a moving train) or inactive event sequences (e.g., a train that is not moving), obtaining neighborhood information for nodes that belong to active event sequences, analyzing structural similarities between nodes that belong to active event sequences and nodes that belong to inactive event sequences, and replacing nodes that belong to inactive event sequences and satisfy the structural similarity criterion with composite nodes.

[0079] As described herein, in an embodiment, the graph corrector 1156 may determine that a second node set satisfies the structural similarity criteria with respect to a first node set if the second node set is a repetition (e.g., the second node set repeats in the same order as the first node set) or a reverse repetition of the first node set (e.g., the second node set repeats in the reverse order as the first node set). In an embodiment, the graph corrector 1156 may determine whether the second node set satisfies the structural similarity criteria with respect to the first node set by analyzing the graph using existing graph analysis tools.

[0080] Next, in step S1350, the graph correction unit 1156 defines a disjunction edge for the complex node in the revised selection graph based on the neighborhood information set. Here, defining a disjunction edge refers to assigning a disjunction relationship between the complex node and other nodes in the revised selection graph based on their directionality. In this way, even if a specific node set is replaced with a complex node, the disjunction relationship between elements of the graph can be maintained.

[0081] Next, in step S1360, the node embedding unit 1159 generates an embedded graph by embedding neighborhood information for each node of the revised selection graph generated in step S1340. Details of this node embedding process will be described later.

[0082] As described herein, this node embedding can be considered to be a feature vector that contains sufficient information about the target node and its relationship information in the graph. Here, an embedding technique such as a K-layer graph neural network (GNN) can be used. Such a GNN has K embedding layers through which the revised selection graph is sequentially passed. These embedding layers represent embedding functions that can be approximated using a multilayer perceptron (MLP). In an embodiment, the embedding function for each layer can be written as follows:

number

[0083] In the above equation, there are six differentiable functions that represent different aspects of the neighborhood of the node of interest v, where h v k denotes the k-th embedded form of node v. For each position, ReLu(x) = max(0,x), and || is the vector concatenation operator.

[0084] f pr (.;θ1) is the preceding node information Σ i∈Np(v) h i k-1is a predecessor node updater that takes input from p (v) is the list of predecessors of node v. f sc (.;θ2) is the successor node information Σ i∈Ns(v) h i k-1 A subsequent node updater that uses N s (v) is the list of successors of node v. f d (.;θ3) is the logical sum node information Σ i∈Nd(v) h i k-1 is a disjunction node updater that uses N s (v) is the list of successors of node v. f Tr,pl (.;θ4) is a complex node updater that uses information from complex nodes that represent event sequences with the same directionality as that represented by the target node v, and Σ i∈NTr,pl(v) h i k-1 N in Tr,pl (v) is a list of complex nodes in the parallel direction. f Tr,op (.;θ5) is a complex node updater that uses information from complex nodes that represent event sequences with the opposite directionality to that represented by the target node v, and Σ i∈NTr,op(v) h i k-1 N in Tr,op (v) is a list of complex nodes in the reverse direction. f t (.;θ6) is the target node updater. It uses the concatenated vector and returns a vector the size of the initial node information vector. h i k-1 represents the k-1th node embedding of the target node v. h v 0 represents the initial node embedding, which is the node information of the target node v before embedding.

[0085] Next, in step S1370, the scheduling unit 1158 determines a scheduling action for the current transition of the embedded graph. In an embodiment, as described herein, the scheduling action may include selecting a node from the embedded graph. Here, the scheduling action for a particular transition τ of the embedded graph is τ The embedded graph at a particular transition τ can be expressed as g τ (K) The set of executable actions (e.g., selectable nodes) may be represented by the updated selection graph generated in step S1330. Here, this set of executable actions may be represented by A gτ and A gτ :{at τ a τ v ;v∈τ is a processable node}. In addition, a τ v denotes the action of choosing node v at transition τ. Furthermore, an actor can be defined to generate a probability distribution over the available (feasible) actions at the τth transition.

number

[0086] where f l (h u (K) ;θ7) is a differentiable function that maps node embeddings to logit values ​​for each node. The actor generates a probability distribution over the available (feasible) actions to be taken by the target machine using a SoftMax function as shown in Equation 2 above. One of the feasible scheduling actions is then selected from the above probabilistic policy by the scheduling unit 1158. To train this scheduling policy, a policy gradient-based reinforcement learning algorithm such as Proximity Policy Optimization (PPO) can be used. Since the PPO algorithm uses a critic that approximates the state values ​​of the MDP, the critic V(g τ(K) ) can be defined as follows:

number

[0087] In response to determining a scheduling action for the current transition of the selection graph, the method returns to step S1330, where the graph update unit 1154 generates an updated selection graph indicating a set of valid scheduling actions for the next transition of the selection graph based on the scheduling actions proposed by the scheduling unit 1158 in step S1370. This process may be repeated until a scheduling action has been generated for each transition of the selection graph.

[0088] Next, in step S1380, once scheduling actions have been generated for each transition in the selection graph, the scheduling unit 1158 may aggregate the generated scheduling actions as a schedule for the scenario defined in the input data set and output the generated schedule to a user terminal.

[0089] While an example has been described in which the input data set includes a first event cycle having a first event sequence and a second event sequence, it should be noted that the present invention is not limited thereto, and a scenario in which the input data set includes multiple event cycles is also possible. In an embodiment, the event cycles may be temporally parallel to each other and may correspond to the progression of a different element (e.g., a train) than the first event sequence. By way of example, in addition to the first event cycle defining the first and second event sequences, the input data set may further include a second event cycle defining a third and fourth event sequence. The third event sequence may correspond to the same time frame as the first event sequence, and the fourth event sequence may correspond to the same time frame as the second event sequence. In this case, the processing steps proceed as described above, with the set of nodes that meet the structural similarity criterion being replaced with a composite node, and the embedding procedure being performed on the revised selection graph.

[0090] The graph neural network embedding management method 1300 described above can reduce the computation time for embedding neighborhood information, thereby enabling the application of GNNs to scheduling problems in dynamic environments.

[0091] Next, an exemplary operation of the graph updater according to an embodiment of the present disclosure will be described with reference to FIG.

[0092] 14 illustrates an example operation 1400 of the graph updater 1154 according to an embodiment of the present disclosure. As described herein, the graph updater 1154 may be configured to generate an updated selection graph that indicates a set of valid scheduling actions for a particular transition of the selection graph.

[0093] More specifically, the graph update unit 1154 updates the action a for the transition τ. τ Receives the state of the selected graph and sets it to G τ From G τ+1 and the effective scheduling action for the next transition is Aτ+1 If there is no valid action, the time t is incremented until a valid scheduling action is available. If there is at least one valid action, the next transition G τ+1 Updated choice graph and effective scheduling action for A τ+1 The set of graphs 1154 is output to the graph modifier 1156 and the scheduler 1158, respectively. An exemplary operation 1400 of the graph updater 1154 is described below.

[0094] First, in step S1410, the graph update unit 1154 updates the selected graph (G τ ) to identify nodes that belong to an active event sequence. In an embodiment, each node in the selection graph may be associated with a status indicator of

[0100] ,

[0010] , or

[0001] , where

[0100] indicates that the node is not yet occupied by an element (e.g., a train) and belongs to an inactive event sequence,

[0010] indicates that the node is currently occupied by an element and belongs to an active event sequence, and

[0001] indicates that the node was occupied by an element in the past but is not currently occupied and belongs to an inactive event sequence. In this case, the graph updater 1154 may identify all nodes that have a status indicator of

[0010] . As an example, the graph updater 1154 may identify node "a1" as belonging to an active event sequence.

[0095] Next, in step S1420, the graph update unit 1154 checks the remaining processing time of the node identified in step S1410 based on the neighborhood information. For example, referring to node a1 as shown in Figure 7, the processing time for section a is 8 units, so the graph update unit 1154 may determine that the remaining processing time is 8 units.

[0096] Next, in step S1430, if the remaining processing time of the node determined in step S1420 is 0 units, graph update unit 1154 extracts a successor node of the selected node. If the remaining processing time of the node is not 0, the method proceeds to step S1460. For example, the processing time for node a1 is 8 units, so no successor node is extracted.

[0097] Next, in step S1440, the graph update unit 1154 checks whether the section of the graph corresponding to the successor node extracted in step S1430 is occupied by an element (for example, a train).

[0098] Next, in step S1450, the graph update unit 1154 adds the nodes in the section of the graph that are not occupied by elements to a list of nodes that can be selected as valid scheduling actions for the next transition 1407. In this way, nodes that can serve as valid scheduling actions for the currently active event sequence can be determined.

[0099] Next, in step S1460, the graph update unit 1154 identifies nodes in the graph that have no predecessor nodes based on the neighborhood information. For example, the graph update unit 1154 may identify node "d2" as a node that has no predecessor nodes.

[0100] Next, in step S1470, the graph update unit 1154 adds the node identified in step S1460 to the list of valid scheduling actions for the next transition 1407. In this way, nodes that are not part of an active event sequence but are in a state where they can be scheduled are identified and can be included as candidates for the next scheduling action.

[0101] Next, in step S1480, the graph update unit 1154 increments the time to t+1 and updates the status of the node with a status indicator of

[0010] to

[0001] . In addition, the graph update unit 1154 sends the list of valid scheduling actions 1407 to the policy 1160 to predict the scheduling action for the next transition.

[0102] In this manner, the graph updater 1154 can generate an updated choice graph that indicates the set of valid scheduling actions for a particular transition in the choice graph.

[0103] Next, an example in which the graph neural network embedded management method according to an embodiment of the present disclosure is applied to a train scheduling problem will be described with reference to FIGS. 15 and 16. FIG.

[0104] Figures 15 and 16 show an example of the progression of a selection graph for a train scheduling scenario through six stages. The example in Figures 15 and 16 is shown for two trains, Train 1 and Train 2, performing three trips (denoted as "Trip" in Figures 15 and 16) across a rail network containing eight sections: a, b, c, d, e, f, g, and h.

[0105] 15, in a first stage 1510, train 1 begins its first trip (trip 11) on section a of the rail network, and train 2 begins its first trip (trip 21) on section h of the rail network. Note that at the start of first stage 1510, a set of nodes corresponding to the second trip of the first train (trip 12), the third trip of the first train (trip 13), the second trip of the second train (trip 22), and the third trip of the second train (trip 23) are represented using composite nodes. As described herein, this is because the trips are repeats or reverse repeats of the first trip of the first train or the first trip of the second train, thereby satisfying the structural similarity criteria described herein. Assume that in the first step 1510, the scheduler 1158 selected node a11 for placement of train 1.

[0106] Next, in a second stage 1520, node a11 is assigned for train 1, and the scheduling unit 1158 selects node h21 for placement of train 2. In a third stage 1530, node h21 is assigned for placement of train 2. Node selection and assignment continues in this manner, with the scheduling unit 1158 selecting nodes for placement of train 1 and train 2. For example, it may be assumed that the scheduling unit 1158 selects nodes b11, d11, and g11 for train 1, and nodes e21, d21, and c21 for train 2.

[0107] 16 , in a fourth step 1540, the scheduling unit 1158 selects node h11 for allocation of the first train. Because node h11 is a node included in the composite node for service 12, in a fifth step 1550, the scheduling unit 1158 expands the composite node for service 12, and at the same time, replaces the node set corresponding to service 11 with the composite node because the node corresponding to service 11 is no longer an active service (e.g., an active event cycle). In addition, the scheduling unit 1158 selects node a21 for allocation of the second train. Similarly, because node a21 is a node included in the composite node for service 22, in a sixth step 1560, the scheduling unit 1158 expands the composite node for service 22, and at the same time, replaces the node set corresponding to service 21 with the composite node because the node corresponding to service 21 is no longer part of an active service.

[0108] In this way, by replacing nodes that meet the structural similarity criteria corresponding to inactive event cycles (e.g., inactive train operations) with composite nodes and expanding those composite nodes when necessary, it is possible to reduce the number of nodes and edges in the selection graph, thereby reducing the computational time for the embedding procedure. As a result, GNNs may be applicable to scheduling problems in dynamic environments.

[0109] As described herein, conventional embedding methods perform embedding for each node of the selection graph. In contrast, the graph neural network embedding management method of the present disclosure modifies the selection graph to replace a set of nodes (e.g., a set of nodes that belong to inactive event sequences and have a repetitive structure) with composite nodes to reduce the total number of nodes and edges. For existing embedding methods and the graph neural network embedding management method of the present disclosure, the computation time for the embedding procedure across three embedding layers of the GNN was tested. These tests were performed using different parameters, such as the number of trains, the number of service trips, and the number of railway sections. The results of the above tests for a single embedding (across three embedding layers of the GNN) are shown in Figures 17 to 19.

[0110] Next, with reference to Figures 17 to 19, the computation time for embedding neighborhood information is compared between the graph neural network embedding management technique according to the embodiment of the present disclosure and the existing technique. Note that in Figures 17, 18, and 19, the upper line of the graph corresponds to the existing technique, and the lower line of the graph corresponds to the graph neural network embedding management technique according to the embodiment of the present disclosure.

[0111] FIG. 17 shows a graph 1700 illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure and existing techniques in a scenario where the number of trains varies from 2 to 10 in a trip of 4 trains per train across 8 rail sections.

[0112] FIG. 18 shows a graph 1800 illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure and existing techniques in a scenario where the number of trips changes from 8 to 24 with 10 trains and 10 trips per train.

[0113] FIG. 19 shows a graph 1900 illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure and existing techniques in a scenario where the number of rail sections varies from 8 to 24 with 10 trains and 10 trips per train.

[0114] As can be seen from graphs 1700, 1800, and 1900, the graph neural network embedding management technique according to embodiments of the present disclosure can reduce the computation time for the embedding process across a variety of scenarios. This reduction in computation time for embedding is achieved as a result of reducing the total number of nodes and edges in the selection graph. For example, as shown in FIGS. 18 and 19, at the largest test scale (10 trains, 24 sections, and 10 trips per train), existing methods took over 450 seconds to compute the embedding, while the graph neural network embedding management technique was able to achieve an embedding of the revised selection graph in less than 20 seconds. This reduction in computation time makes GNNs applicable to scheduling problems in dynamic environments with strict time constraints, such as train scheduling problems.

[0115] As described herein, aspects of the present disclosure relate to extending the use of graph representations in reinforcement learning to scheduling scenarios in dynamic environments, such as railway operation scheduling scenarios. In embodiments, a choice graph for a railway scheduling problem may be generated considering additional blocking constraints not present in standard JSSP. Unlike conventional approaches, the present choice graph representation extends to complex scenarios involving multiple parallel graph sections (e.g., scheduling multiple trains in parallel) and multiple event cycles (e.g., trains performing multiple trips). In such scenarios, the computational time for the GNN embedding process using conventional approaches can be prohibitive.

[0116] In light of this, the graph neural network embedding management technique of the present disclosure is directed to reducing the size of the selection graph while maintaining the maximum amount of information. Thus, by utilizing the recurring structure present in some scheduling scenarios to represent a group of nodes corresponding to inactive event sequences (e.g., inactive train operations) as a single composite node, a revised selection graph with a reduced number of nodes and edges can be generated. By performing the embedding procedure on this revised selection graph, the computation time of the embedding procedure can be reduced to facilitate the rapid generation of scheduling solutions, thereby enabling the application of GNNs to scheduling scenarios (e.g., train scheduling) in dynamic environments.

[0117] A further aspect of the present disclosure relates to the use of proximity policy optimization methods for reinforcement learning, which facilitates flexible modification of the revised choice graph and enables the generation of generalized choice graphs for train scheduling, such that complex rail network elements such as intersections, multiple lines with different stops, etc. can be included.

[0118] It should be noted that in the description of the embodiments, reference is made to generating and modifying a selection graph. Here, generating a graph may refer to the process of generating data in tabular or matrix form that represents the various nodes and edges of the graph, and does not require the generation of a visual graph representation itself. Similarly, modifying a selection graph may include adding, deleting, or altering tabular or matrix data that represents the graph, and does not require modifying the visual graph representation.

[0119] The present invention may be a system, a method, and / or a computer program product, which may include computer-readable storage medium(s) having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0120] A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanical encryption devices such as punch cards or grooved ridge structures with instructions recorded thereon, and any suitable combination of the above. As used herein, a computer-readable storage medium should not be construed as being, per se, a transitory signal such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted by an electrical wire.

[0121] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0122] The computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to manufacture a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in the flowchart and / or block diagram blocks. These computer-readable program instructions may further be stored on a computer-readable storage medium that causes the computer, programmable data processing apparatus, and / or other apparatus to function in a particular way, such that the computer-readable storage medium having the instructions stored thereon comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in the flowchart and / or block diagram blocks.

[0123] The computer-readable program instructions may further be loaded into a computer, other programmable data processing device, or other device to cause a series of operational steps to be executed on the computer or other programmable device to create a computer-implemented process, such that the instructions executing on the computer, other programmable device, or other device perform the functions / acts specified in the flowchart and / or block diagram blocks.

[0124] Embodiments according to the present disclosure may be provided to end users via a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing capability that provides abstraction between computing resources and their underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be provisioned and released rapidly with minimal management effort or service provider interaction. Cloud computing thereby enables users to access virtual computing resources in the "cloud" (e.g., storage, data, applications, and even comprehensive virtualized computing systems) without regard for the underlying physical systems used to provide the computing resources (or the location of those systems).

[0125] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specialized logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It will also be appreciated that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs a specialized function or operation or executes a combination of dedicated hardware and computer instructions.

[0126] While the foregoing relates to exemplary embodiments of the present invention, other and further embodiments may be devised without departing from the basic scope of the present invention, which scope is determined by the following claims. The description of various embodiments of the present disclosure has been provided for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to explain the principles of the embodiments, practical applications or technical improvements of existing technologies, or to enable others of ordinary skill in the art to which the present disclosure pertains to understand the embodiments disclosed herein.

[0127] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit various embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural as well, unless the context clearly indicates otherwise. "Set," "group," "bundle," etc. are intended to include one or more. Furthermore, it will be understood that the words "comprise" and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. In the above detailed description of exemplary embodiments of various embodiments, reference has been made to the accompanying drawings (in which like numerals indicate like elements) that form a part of this specification, showing, by way of example, specific exemplary embodiments and in which various embodiments may be practiced. The above embodiments have been described in sufficient detail to enable one skilled in the art to practice the embodiments, but other embodiments may be used, and logical, mechanical, electrical, and other changes may be made without departing from the scope of the various embodiments. In the above description, numerous specific details have been set forth to provide a thorough understanding of the various embodiments. However, various embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the embodiments. [Explanation of symbols]

[0128] 100 Computer Systems 102 processors 104 memory 106 memory bus 108 I / O buses 109 Bus IF 110 I / O bus interface 112 Terminal Interface 113 Storage Interface 114 I / O device interface 115 Network Interface 116 User I / O Device 117 Storage device 124 Display System 126 Display device 130 Network 150 Scheduling Management Application

Claims

1. a data input for receiving an input data set including a first cycle of events defining a first sequence of events and a second sequence of events; a graph generator for generating an initial selection graph to represent the first event cycle, the initial selection graph defining a first node set corresponding to a first event set of the first event sequence, a second node set corresponding to a second event set of the second event sequence, and a first set of neighborhood information defining a relationship between each node of the first node set and the second node set; a graph corrector for generating a revised selected graph in which the second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; a node embedding unit for generating an embedded graph based on the revised selected graph, in which the first set of neighborhood information is embedded in each node of the first node set and the second node set; A graph neural network embedding management device comprising:

2. 2. The graph neural network embedding manager of claim 1, wherein the graph corrector determines that the second node set satisfies the structural similarity criterion with respect to the first node set if the second node set is a repeat or an inverse repeat of the first node set.

3. the first set of nodes and the second set of nodes represent stages of a first element's progression through the first cycle of events; 2. The graph neural network embedded manager of claim 1, wherein the first sequence of events corresponds to a first time interval and the second sequence of events corresponds to a second time interval subsequent to the first time interval.

4. the input data set further includes a second cycle of events defining a third sequence of events and a fourth sequence of events; the graph generator generates, as the initial selection graph, a selection graph for representing the first event cycle and the second event cycle, which defines: the first node set corresponding to the first event set of the first event sequence; the second node set corresponding to the second event set of the second event sequence; a third node set corresponding to a third event set of the third event sequence; a fourth node set corresponding to a fourth event set of the fourth event sequence; and a second neighborhood information set defining a relationship between each node of the first node set, the second node set, the third node set, and the fourth node set; the graph modification unit generates, as the revised selection graph, a selection graph in which the second node set is replaced by the first composite node and the fourth node set is replaced by the second composite node if the fourth node set satisfies the structural similarity criterion with respect to the third node set; 4. The graph neural network embedding management device according to claim 3, wherein the node embedding unit generates, as the embedded graph, a selection graph in which the second neighborhood information set is embedded in each node of the first node set, the second node set, the third node set, and the fourth node set.

5. the third set of nodes and the fourth set of nodes represent stages of progression of a second element through the second cycle of events; 5. The graph neural network embedded manager of claim 4, wherein the third sequence of events corresponds to the first time interval and the fourth sequence of events corresponds to the second time interval.

6. 6. The graph neural network embedding manager of claim 5, wherein the second set of neighborhood information defines constraints on the progression of the first element through the first cycle of events and constraints on the progression of the second element through the second cycle of events.

7. a scheduling unit configured to generate, based on the embedded graph, a policy that predicts a scheduling action for a first transition of the embedded graph; a graph updater configured to generate an updated selection graph indicating a set of valid scheduling actions for a second transition of the embedded graph based on the scheduling actions; and The graph neural network embedding manager of claim 6, further comprising:

8. the first element is a first vehicle; the second element is a second vehicle; the first sequence of events is a first trip of the first vehicle; the second sequence of events is a second trip of the first vehicle; the third sequence of events is a first trip of the second vehicle; the fourth sequence of events is a second trip of the second vehicle; each node in the first node set, the second node set, the third node set, and the fourth node set corresponds to a waypoint of a vehicle route; 8. The graph neural network embedded management device of claim 7.

9. A graph neural network embedding management device comprising a data input unit, a graph generation unit, a graph modification unit, a node embedding unit, and a scheduling unit, wherein the data input unit receives an input data set including a first route defining a first trip of a first vehicle and a second trip of the first vehicle; the graph generator generating an initial selection graph representing the first route, the initial selection graph defining a first set of nodes corresponding to a first set of waypoints of the first journey, a second set of nodes corresponding to a second set of waypoints of the second journey, and a first set of neighborhood information defining relationships between nodes in the first set of nodes and the second set of nodes; the graph corrector generating a revised selected graph in which the second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; the node embedding unit generates an embedded graph based on the revised selection graph, in which the first neighborhood information set is embedded in each node of the first node set and the second node set in the revised selection graph; the scheduling unit determining a scheduling action for the first vehicle using a graph neural network that processes the embedded graph; A graph neural network embedding management method, including:

10. a graph neural network embedding manager; User terminal and 1. A graph neural network embedded management system comprising: The graph neural network embedding management device comprises: a data input for receiving an input data set from the user terminal, the input data set including a first cycle of events defining a first sequence of events and a second sequence of events; a graph generator for generating an initial selection graph to represent the first event cycle defining a first node set corresponding to a first set of events in the first event sequence, a second node set corresponding to a second set of events in the second event sequence, and a first set of neighborhood information defining a relationship between each node in the first node set and the second node set, wherein the first node set and the second node set represent stages of progression of a first element through the first event cycle, the first event sequence corresponding to a first time segment and the second event sequence corresponding to a second time segment subsequent to the first time segment; a graph corrector for generating a revised selected graph in which the second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; a node embedding unit for generating an embedded graph based on the revised selected graph, in which the first set of neighborhood information is embedded in each node of the first node set and the second node set in the revised selected graph; a scheduling unit configured to generate a schedule defining the progression of the first element through the first cycle of events based on the embedded graph and to output the schedule to the user terminal; 1. A graph neural network embedded management system comprising:

Images