Graph neural network bedding management device, graph neural network embedding management method, and graph neural network embedding management system

Abstract

To enable utilizing highly accurate scheduling of GNN in an operating environment with limited time.SOLUTION: A GNN-embedded management apparatus includes: a data input unit for receiving an input dataset for defining a first event sequence and a second event sequence; a graph generation unit which generates an initial selection graph for defining a first node set for the first event sequence, a second node set for the second event sequence, and a first neighborhood information set; a graph correction unit for generating a revised selection graph by replacing the second node set with a first composite node when the second node set satisfies a structural similarity criterion; and a node embedding unit for generating an embedded graph in which the first neighborhood information set is embedded in each of the nodes.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates generally to managing information embedding for graph neural networks (GNNs), and more particularly to using graph neural networks for scheduling problems. [Background technology]

[0002] In recent years, reinforcement learning (RL) has become an important tool for solving scheduling problems of any complexity, and has proven to be superior to approaches based on priority dispatching rules traditionally used for scheduling problems.

[0003] As an example, P. Junyoung et al. (Non-Patent Document 1) describe a technique in which RL is applied to job shop problems. More specifically, Non-Patent Document 1 states, "We propose a framework for learning to schedule job shop problems (JSSPs) using graph neural networks (GNNs) and reinforcement learning (RL). As a sequential decision-making problem, we formulate the scheduling process of JSSPs using a graph representation of states that considers the structure of JSSPs. In solving the formulated problem, the proposed framework uses GNNs to learn node features that embed the spatial structure of JSSPs represented as a graph (representation learning), and derives optimal scheduling policies that map the embedded node features to the best scheduling actions (policy learning). We train the above two modules end-to-end using a proximity policy optimization (PPO)-based RL strategy. We empirically demonstrate that the GNN scheduler outperforms execution-preferred dispatching rules and RL-based schedulers on various benchmark JSSPs due to its excellent generalization capability. We also confirm that the proposed framework learns transferable scheduling policies that can be used to schedule completely novel JSSPs (in terms of size and parameters) without further training." [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] P. Junyoung, C. Jaehyeong, K. Sang Hun, K. Youngkook, and P. Jinkyoo, "Learning to schedule job-shop problems: representation and policy," CoRR, vol. abs / 2106.01086, 2021. Summary of the Invention [Problem to be solved by the invention]

[0005] Typically, in RL-based approaches, a scheduler (often an RL agent) learns a dispatching policy that maps the current state of the system to scheduling actions while taking into account the sequential nature of the problem. To determine the optimal scheduling actions, it is important to provide the scheduler with a state representation that captures the maximum amount of information about the current state of the system.

[0006] Graph representation has proven to be an effective approach to capture the underlying structure of scheduling scenarios, as it preserves the state relationships of the scheduled system in the form of neighborhood information. Graph Neural Networks (GNNs) have been used to efficiently embed the above neighborhood information in a structured manner, so that the state embedded vectors can reflect the neighborhood information.

[0007] However, this embedding procedure is time-consuming, which is an obstacle for tasks that require a solution in a limited time frame, such as train rescheduling due to disruptions to the original schedule. Therefore, the time-consuming nature of the embedding procedure has limited the application of GNNs to dynamically changing environments such as train rescheduling.

[0008] In the technique disclosed in Non-Patent Document 1, for example, graph representation and RL are applied to the scheduling problem, but the entire selection graph is used for embedding by GNN, which leads to long computation time as mentioned above. Therefore, the technique disclosed in Non-Patent Document 1 is not suitable for application to scheduling problems in dynamically changing environments with strict time constraints, such as train rescheduling.

[0009] Therefore, in view of the above challenges, an approach is needed that can leverage the high-precision scheduling of GNNs in time-constrained and dynamic environments.

[0010] Therefore, an objective of the present disclosure is to provide a graph neural network embedding management technique that makes GNNs applicable in dynamic environments by reducing the computation time for embedding neighborhood information. [Means for solving the problem]

[0011] A representative example of the present disclosure relates to a graph neural network embedding management apparatus, comprising: a data input unit for receiving an input data set including a first event cycle defining a first event sequence and a second event sequence; a graph generation unit for generating an initial selection graph for representing the first event cycle, the initial selection graph defining a first node set corresponding to a first event set of the first event sequence, a second node set corresponding to a second event set of the second event sequence, and a first neighborhood information set defining a relationship between each node of the first node set and the second node set; a graph modification unit for generating a revised selection graph in which the second node set is replaced by a first composite node aggregating the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; and a node embedding unit for generating an embedded graph in which the first neighborhood information set is embedded in each node of the first node set and the second node set based on the revised selection graph. Effect of the Invention

[0012] According to the present disclosure, it is possible to provide a graph neural network embedding management technique that enables the application of GNNs in dynamic environments by reducing the computation time for embedding neighborhood information.

[0013] Problems, configurations, and advantages other than those mentioned above will become apparent from the following description of embodiments for carrying out the present invention. [Brief description of the drawings]

[0014] [Figure 1]FIG. 1 is a block diagram illustrating an exemplary computing architecture for implementing embodiments of the present disclosure. [Diagram 2] FIG. 2 is a diagram showing a selection graph for a job shop problem. [Diagram 3] FIG. 3 shows the embedding layer of a GNN. [Figure 4] FIG. 4 shows an example of a rail network including two trains and four rail sections. [Diagram 5] FIG. 5 illustrates a framework for training a GNN-RL scheduler. [Figure 6] FIG. 6 illustrates a choice graph for the train scheduling problem. [Figure 7] FIG. 7 illustrates the progression of a choice graph through multiple states in response to choices made by an agent. [Figure 8] Figure 8 shows the choice graph for two trains running in parallel and in opposite directions in a railway network. [Figure 9] FIG. 9 is a selection graph for two trains making multiple trips between two ends of a rail network. [Figure 10] FIG. 10 illustrates an example of the application of the graph modification process according to the present disclosure with respect to train scheduling. [Figure 11] FIG. 11 is a diagram illustrating an exemplary configuration of a graph neural network embedded management system according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a diagram showing an example set of neighborhood information embedded in the nodes of the revised selection graph. [Figure 13] FIG. 13 is a diagram illustrating an example flow of a graph neural network embedding management method according to an embodiment of the present disclosure. [Figure 14] FIG. 14 is a diagram illustrating an exemplary operation of a graph updater according to an embodiment of the present disclosure. [Figure 15]FIG. 15 illustrates the first part of an example of the progression of a choice graph for a train scheduling scenario through six stages. [Figure 16] FIG. 16 illustrates the second part of an example of the progression of a selection graph for a train scheduling scenario through six stages. [Figure 17] FIG. 17 is a first graph illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure (illustrated by the lower line) and existing techniques (illustrated by the upper line). [Figure 18] FIG. 18 is a second graph illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure (illustrated by the lower line) and existing techniques (illustrated by the upper line). [Figure 19] FIG. 19 is a third graph illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure (illustrated by the lower line) and existing techniques (illustrated by the upper line). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] In this specification, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described in this specification are not intended to limit the present invention according to the claims, and it should be understood that each of the elements and their combinations described with respect to the embodiments are not strictly necessary to implement aspects of the present invention.

[0016] Various aspects are disclosed in the following description and in the associated drawings. Alternative aspects may be devised without departing from the scope of the present disclosure. Additionally, well-known elements of the present disclosure will not be described in detail or will be omitted so as not to obscure the relevant details of the present disclosure.

[0017] The words "exemplary" and / or "example" are used herein to mean "serving as an example, instance, or illustrative example." Any aspect described herein as "exemplary" and / or "example" is not necessarily to be construed as preferred or advantageous over other aspects. Similarly, the phrase "aspects of the disclosure" does not require that all aspects of the disclosure include the discussed feature, advantage, or characteristic of operation.

[0018] Further, many aspects are described in terms of sequences of actions to be performed by, for example, elements of a computing device. It will be appreciated that various actions described herein can be performed by specific circuitry (e.g., an application specific integrated circuit (ASIC)), by program instructions executed by one or more processors, or a combination of both.

[0019] Additionally, the sequences of actions described herein can be considered to be generally embodied in any form of computer-readable storage medium storing a corresponding set of computer instructions that, when executed, can cause an associated processor to perform the functions described herein. As such, various aspects of the present disclosure may be embodied in many different forms, all of which are contemplated to be within the subject matter recited in the claims.

[0020] Referring now to the drawings, Figure 1 is a schematic block diagram of a computer system 100 according to an embodiment for implementing various embodiments of the present disclosure. The mechanisms and devices of the various embodiments disclosed herein are equally applicable to any suitable computing system. The main components of the computer system 100 include one or more processors 102, memory 104, terminal interface 112, storage interface 113, I / O (input / output) device interface 114, and network interface 115, all of which are communicatively coupled, directly or indirectly, for inter-component communication via memory bus 106, I / O bus 108, bus interface section 109, and I / O bus interface section 110.

[0021] Computer system 100 may include one or more general purpose programmable central processing units (CPUs) 102A and 102B, generally referred to herein as processors 102. In embodiments, computer system 100 may include multiple processors, although in particular embodiments computer system 100 may alternatively be a single CPU system. Each processor 102 executes instructions stored in memory 104 and may include one or more levels of on-board cache.

[0022] In an embodiment, memory 104 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing or encoding data and programs. In a particular embodiment, memory 104 represents the entire virtual memory of computer system 100 and may further include virtual memory of other computer systems coupled to computer system 100 or connected via a network. While memory 104 can be conceptually viewed as a single monolithic entity, in other embodiments memory 104 is a more complex device, such as a hierarchy of caches and other memory elements. For example, memory may exist in multiple levels of caches, which may be further divided by function, whereby one cache holds instructions and another cache holds non-instruction data used by the processor. Memory may also be distributed and associated with different CPUs or sets of CPUs, as known in any of a variety of so-called non-uniform memory access (NUMA) computer architectures.

[0023] Memory 104 may store all or a portion of various programs, modules, and data structures for handling data transfers as described herein. For example, memory 104 may store graph neural network embedded management application 150. In an embodiment, graph neural network embedded management application 150 may include instructions or statements that execute on processor 102 or that are interpreted by processor 102 to perform functions as described further below.

[0024] In certain embodiments, the graph neural network embedded management application 150 is implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices instead of or in addition to a processor-based system. In embodiments, the graph neural network embedded management application 150 may include data in addition to instructions or statements. In certain embodiments, a camera, sensor, or other data input device (not shown) may be provided in direct communication with the bus interface portion 109, the processor 102, or other hardware of the computer system 100. In such a configuration, the need for the processor 102 to access the memory 104 and the graph neural network embedded management application 150 may be reduced.

[0025] The computer system 100 may include a bus interface unit 109 that handles communication between the processor 102, the memory 104, the display system 124, and an I / O bus interface unit 110. The I / O bus interface unit 110 may be coupled to an I / O bus 108 to transfer data to and from various I / O units. The I / O bus interface unit 110 communicates via the I / O bus 108 with a number of I / O interface units 112, 113, 114, and 115, also known as I / O processors (IOPs) or I / O adapters (IOAs). The display system 124 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both types of data to a display device 126. Additionally, the computer system 100 may include one or more sensors or other devices configured to collect and provide data to the processor 102.

[0026] By way of example, computer system 100 may include biometric sensors (e.g., collecting heart rate data, stress level data), environmental sensors (e.g., collecting humidity data, temperature data, pressure data), motion sensors (e.g., collecting acceleration data, movement data), etc. Other types of sensors are possible. The display memory may be a dedicated memory for buffering video data. Display system 124 may be coupled with a display device 126, such as a standalone display screen, a computer monitor, a television, a tablet, or a display of a handheld device.

[0027] In one embodiment, the display device 126 may include one or more speakers for rendering audio. Alternatively, the one or more speakers for rendering audio may be coupled to the I / O interface unit. In an alternative embodiment, one or more of the functions provided by the display system 124 may be implemented in an integrated circuit that also includes the processor 102. Furthermore, one or more of the functions provided by the bus interface unit 109 may be implemented in an integrated circuit that also includes the processor 102.

[0028] The I / O interface unit supports communication with various storage devices and I / O devices. For example, the terminal interface unit 112 supports connection of one or more user I / O devices 116, which may include user output devices (such as a video display device, a speaker, and / or a television receiver) and user input devices (such as a keyboard, a mouse, a keypad, a touchpad, a trackball, a button, a light pen, or other pointing device). A user may use a user interface to operate the user input devices to provide input data and commands to the user I / O devices 116 and the computer system 100, and may also receive output data via the user output devices. For example, the user interface may be presented via the user I / O devices 116, such as displayed on a display device, played through a speaker, or printed by a printer.

[0029] The storage interface 113 supports the connection of one or more disk drives or direct access storage devices 117 (typically rotating magnetic disk drive storage devices, but alternatively other storage devices including arrays of disk drives or solid state drives such as flash memory that appear to a host computer as a single mass storage device). In some embodiments, the storage device 117 may be implemented by any type of secondary storage device. The contents of the memory 104, or any portion thereof, may be stored in the storage device 117 and retrieved from the storage device 117 as needed. The I / O device interface 114 provides an interface to any of a variety of other I / O devices, or other types of devices such as printers or fax machines. The network interface 115 provides one or more communication paths from the computer system 100 to other digital devices and computer systems, which may include, for example, one or more networks 130.

[0030] 1 illustrates a particular bus structure providing direct communication paths between processor 102, memory 104, bus interface 109, display system 124, and I / O bus interface section 110, in alternative embodiments, computer system 100 may include different buses or communication paths that may be arranged in any of a variety of forms, such as point-to-point links in a hierarchical, star, or web configuration, multiple hierarchical buses, parallel and redundant paths, or any other suitable type of configuration. Additionally, while I / O bus interface section 110 and I / O bus 108 are shown as single respective components, computer system 100 may actually include multiple I / O bus interface sections 110 and / or multiple I / O buses 108. Although multiple I / O interface sections are shown isolating I / O bus 108 from the various communication paths running to the various I / O devices, in other embodiments some or all of the I / O devices are directly connected to one or more system I / O buses.

[0031] In various embodiments, computer system 100 is a multi-user mainframe computer system, a single-user system or a server computer, or a similar device with little or no direct user interface, but which receives requests from other computer systems (clients). In other embodiments, computer system 100 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable type of electronic device.

[0032] The background and the problem solved by the present disclosure will now be described with reference to Figures 2 through 9.

[0033] Rail is an important and popular means of transporting people and goods. There is an increasing demand for rail services due to the growing population in urban areas. According to current trends, passenger and freight activity will more than double by 2050. Most conventional rail networks are currently located in North America, Europe, China, Russia, India, and Japan. These regions account for approximately 90% of global passenger movement on conventional rail, with India leading with 39%, followed by China with 27%, Japan with 11%, and the European Union with 9%.

[0034] In the period 2018-2020, the market size was estimated at an average of 114 billion euros. The market is expected to grow at a compound annual growth rate of 2.6 percent and reach an average annual valuation of 125 billion euros between 2021-2023. The development of railway infrastructure is increasing significantly to meet the above demand. This leads to the need for dense rail networks of stations, tracks and trains to operate at higher frequencies to accommodate the above demand. Thus, efficient scheduling of a large number of trains in those rail networks becomes an enormous challenge. This efficiency also affects passenger commute and goods transport times, waiting times at stations, fuel usage, etc. Techniques for this scheduling have been extensively studied in recent years in diverse fields such as job shops, workforces, warehouses, and transportation. The aim of this scheduling is to improve the efficiency of the process by optimizing KPIs such as makespan, waiting time, costs, etc.

[0035] The most common scheduling problem is the Job Shop Scheduling Problem (JSSP), which can be extended to include other areas of research in industry scheduling. The Train Scheduling Problem (TSP) is an extension of the JSSP, which has additional blocking and no-wait constraints. Therefore, the train scheduling problem is more complicated compared to the traditional JSSP. The TSP is an extensively studied subject, and there are exact, heuristic, and hybrid methods, as well as the proposed learning-based method. Exact methods cannot be used for large-scale scheduling problems due to their time complexity.

[0036] Heuristic and hybrid methods are based on specific rules defined based on domain knowledge and provide a faster approach to obtain results. These are called priority dispatching rules (PDR). However, PDR ignores the sequential nature of the scheduling problem. Learning-based methods have shown superior performance compared to PDR. In the reinforcement learning (RL) approach, a scheduler (often an RL agent) learns a dispatching policy that maps the current state of the system to scheduling actions while taking into account the sequential nature of the problem.

[0037] However, these RL methods typically require per-instance training to derive schedules, and it is difficult to apply the trained policies to novel scheduling problems. Graph representations have also been used to solve scheduling problems such as the Traveling Salesman Problem. Problem instances are represented as graphs, and embedding algorithms such as Graph Neural Networks (GNNs) are used to convert the graphs into vectors that outline the structural information of the problem. Thus, RL algorithms use node embeddings to generate scheduling actions. Details of graph representations and node embeddings are described in more detail later in this specification.

[0038] JSSP is one of the best-known problems in the field of combinatorial optimization. A traditional n / m JSSP can be described as the scheduling of n jobs on m machines, with the objective of minimizing the makespan of all jobs. Each of the n jobs consists of several operations on m dedicated machines and requires a certain sequential processing time. Also, the operations of a job can only be executed on one machine at a time, and each job needs to be processed exactly once on each machine. JSSP may be represented as a choice graph to facilitate the description of constraints and relationships between different operations and machines. Such a choice graph can be used in applying metaheuristics due to the ease with which complex relationships can be modeled.

[0039] An example of a job shop problem with three jobs and three machines is described with reference to Table 1 and Figure 2. Table 1 shows an example of a job shop problem with three jobs and three machines in tabular form. Figure 2 shows an example of a job shop problem with three jobs and three machines as a selection graph. [Table 1]

[0040] In FIG. 2, O ij represents the operation of job i on machine j, and p ij represents the processing time of job i on machine j. The solid edge between two nodes is a conjunction edge, indicating a given priority of the job's operations. The dashed edge between two nodes is a disjunction edge, representing a machine sharing requirement between the two operations. Note that the dashed edges are bidirectional, meaning that the order of scheduling of the nodes connected by the edges is not known in advance. These edges become unidirectional once the operations are scheduled. Thus, scheduling can be understood to determine the direction of the disjunction edges. The two nodes at the top and bottom are dummy nodes, representing the start and end of the entire process. As will be explained later, the selection graph can also be used to model train scheduling problems.

[0041] The information in the graph is represented as node and neighborhood information. The nodes indicate the status of the ongoing schedule, and the edges indicate the neighborhood relationships between the nodes. Graph embedding is the process of converting the graph data of a selection graph into a vector format that outlines the information in the graph. Graph Neural Networks (GNNs) are a tool for embedding graph representations of systems. In general, GNNs contain a series of K embedding layers that utilize the neighborhood information of the nodes to generate the corresponding embedded form.

[0042] FIG. 3 is a diagram showing an example configuration of an embedding layer of a GNN. In FIG. τ (0) represents the graph of the transition τ before passing through the first embedding layer. τ (K) represents the graph of transitions τ after passing through K embedding layers. The nodes of this embedded graph are embedded with neighborhood information that describes their relationship with other nodes in the graph. This process of embedding neighborhood information needs to be performed every time a new state is provided to the agent of the RL process. The time for embedding depends on the number of nodes and edges in the input graph. Therefore, when the number of nodes and edges is large, the computation time for performing this embedding process increases, which becomes a challenge in large-scale networks. Therefore, as described later in this specification, an aspect of the present disclosure relates to providing a graph neural network embedding management technique to reduce the computation time for embedding neighborhood information to enable the application of GNNs in large-scale dynamic environments.

[0043] We now consider an example in which the graph representation technique and the GNN framework are applied to the train scheduling problem, with reference to Figs. 4 to 6 .

[0044] FIG. 4 illustrates an example of a railroad network 400. In general, a railroad network includes trains traveling on tracks that connect intermediate points such as stations. These tracks may be divided into sections. When reinforcement learning is utilized to generate a schedule for a railroad network, a scheduler determines which trains to schedule in a particular section at a given time. It is desirable for the scheduler to provide an efficient schedule that optimizes KPIs such as passenger waiting time or total travel time. As shown in FIG. 4, the railroad network 400 includes two trains, train 1 and train 2, traveling in opposite directions across four sections a, b, c, d. Additionally, the processing time for each section (i.e., the time it takes for a train to travel a particular section) is also shown (e.g., 8 units for sections a, b, and c, and 10 units for section d).

[0045] Next, a training framework 500 for training a GNN-RL scheduler 530 for a train scheduling problem is described with reference to Fig. 5. As shown in Fig. 5, the GNN-RL scheduler 530 inputs a Markov decision process (MDP) equation 520 in which a train scheduling problem instance 510 is represented as a choice graph, and a reinforcement learning method 540, such as proximity policy optimization, is used to train the GNN-RL scheduler 530 to learn a scheduling policy for train scheduling.

[0046] As described herein, the train scheduling problem extends the JSSP by adding other constraints, such as blocking conditions, which mean that a section of the rail network is blocked from new trains being placed on it until one of the trains currently placed on it has been scheduled onto the new available section (e.g., two trains cannot occupy the same section of rail track at the same time). The choice graph for the JSSP can be modified to account for the additional blocking constraints mentioned above.

[0047] FIG. 6 is a choice graph 600 for the train scheduling problem. In FIG. 6, S0 represents the initial state of the choice graph. Here, node a1 indicates the possibility of train 1 entering section a. Solid edges represent potential connections, and dashed edges represent blocking constraints of the railway scheduling problem as disjunction edges. S1 represents the state of the choice graph when an agent selects node a1. Here, the agent's selection of node a1 means that the agent chooses to assign train 1 to section a at a given time. The selected nodes are highlighted in the figure.

[0048] 7 illustrates the progression of a choice graph through states in response to selections made by an agent. In an embodiment, an agent may alternate between node selections for different trains in a rail network. As shown in FIG. 7, when the agent selects node a1, the choice graph progresses from initial state S0 to S1. The choice graph then progresses through states S2 and S3 in response to selections of nodes d2, b1, and c2, respectively, to reach state S4.

[0049] 6 and 7, selection graphs are shown for a simplified rail network with four sections and two trains, but in reality, a rail network will include many more sections and trains, and these trains will make multiple trips on those sections on a single day, so at any given time there will be a mix of trains running in parallel and counter directions.

[0050] FIG. 8 illustrates a selection graph 825 for two trains running in opposite directions in a rail network having eight sections, and a selection graph 850 for two trains running in parallel. In this specification, nodes are identified using a notation having the format of "section, train number, service number" as shown in FIG. 8. As an example, in FIG. 8, "a11" is used for a node representing train 1 occupying section a in service 1. Similarly, "c21" represents train 2, for example, occupying section c in service 1. Furthermore, in this specification, a service is defined as the movement of a train from one end of the rail network to the other end (e.g., from section a to section h, or from section h to section a). It can be seen from FIG. 8 that the selection graphs 825 and 850 are much more complex than the selection graphs illustrated for four sections in FIG. 6 and FIG. 7.

[0051] FIG. 9 shows a selection graph for two trains making multiple trips between two ends of a rail network. In FIG. 9, the upper selection graph set 925 shows the trips made by train 1, and the lower selection graph set 950 shows the trips made by train 2. Train 1, in its first trip, starts at section a and travels to section h. Train 2, in its first trip, starts at section h and travels to section a. When either of the trains reaches the last section of the rail network, it starts its next trip along the reverse order of sections. Note that this new trip (i.e., the reverse trip) is also disjunctively connected to all of the trips of each other train. As the number of trains and the number of trips per train increase, the complexity of the selection graph increases, which in turn increases the computation time for embedding the neighborhood information. This is a challenge in train rescheduling scenarios, especially when new schedules need to be generated fast by a trained GNN-RL scheduler based on the selection graph that represents the state of the rail network.

[0052] Thus, as described herein, aspects of the present disclosure relate to graph neural network embedding management techniques that enable the application of GNNs in dynamic environments by reducing the computation time for embedding neighborhood information. Next, with reference to Figures 10 to 19, graph neural network embedding management techniques according to the present disclosure are described.

[0053] 9, successive trips of trains form a unit that repeats over the remainder of the graph. For example, trip 3 of train 1 is structurally similar (e.g., repetitive) to trip 1 of train 1 and trip 2 of train 2. Thus, aspects of the present disclosure relate to leveraging the repetitive structure present in some scheduling scenarios to reduce the number of nodes and the resulting computation time for the selection graph embedding.

[0054] More specifically, aspects of the disclosure relate to compressing nodes of a selection graph that have a nearly repetitive structure with respect to other segments of the selection graph into a single composite node, where compressing refers to grouping multiple nodes together. When applying this approach to train scheduling, only ongoing trips maintain the original number of nodes connected by the original conjunction and disjunction edges. Since all other trips (future and past) can be expressed as repeating units of those ongoing trips, those other trips can be compressed into a single node representing each trip and expanded when required for scheduling options. Where expanding refers to splitting or dividing a composite node into the individual nodes contained therein.

[0055] Figure 10 shows an example of the application of the graph modification process according to the present disclosure in the context of train scheduling. As is evident by comparing Figures 9 and 10, the 8 nodes of Train 2 Trip 2 in the lower selection graph set 950 shown in Figure 9 are replaced by a single composite node 1010 in Figure 10. Similarly, the remaining trips (Train 1 Trip 2, Train 1 Trip 3, Train 2 Trip 3) that are not currently in progress or can be scheduled are represented by a single composite node. In this way, a reduction in the number of nodes and the resulting computation time for the selection graph embedding is possible.

[0056] Next, a graph neural network embedded management system according to an embodiment of the present disclosure will be described with reference to FIG.

[0057] Fig. 11 is a diagram illustrating an exemplary configuration of a graph neural network embedding management system 1100 according to an embodiment of the present disclosure. As shown in Fig. 11, the graph neural network embedding management system 1100 according to an embodiment of the present disclosure basically includes a user terminal 1120 and a graph neural network embedding management device 1150. The graph neural network embedding management device 1150 and the user terminal 1120 can be connected via a communication network such as the Internet (not shown in Fig. 11).

[0058] The user terminal 1120 is a computing device usable by a user of the graph neural network embedded manager 1150. In an embodiment, the user terminal 1120 is usable to submit an input data set 1122 describing a current state of a system for which a schedule generation is desired, and to subsequently receive a schedule 1172 generated based on the input data set 1122.

[0059] The graph neural network embedding management device 1150 is a computing device for generating a schedule 1172 based on the input data set 1122. As shown in FIG. 11, the graph neural network embedding management device 1150 basically includes a data input unit 1151, a graph generating unit 1152, a graph updating unit 1154, a graph correcting unit 1156, a scheduling unit 1158, and a storage unit 1161. In an embodiment, the graph neural network embedding management device 1150 may be implemented using the computer system 100 shown in FIG. 1. In this case, the data input unit 1151, the graph generating unit 1152, the graph updating unit 1154, the graph correcting unit 1156, the scheduling unit 1158, and the storage unit 1161 of the graph neural network embedding management device 1150 may be implemented as software modules of the graph neural network embedding management application 150 of the computer system 100.

[0060] Although aspects of embodiments of the present disclosure are described herein with reference to the example of train scheduling, it should be noted that the graph neural network embedded management techniques described herein are not so limited and may be suitably applied to a variety of tasks, for example, the neural network embedded management techniques described herein may be applied to the production management of products in a factory environment in addition to train scheduling.

[0061] The data input unit 1151 is a functional unit for receiving an input of an input data set 1122. The input data set 1122 may include data in a tabular or matrix format. In an embodiment, the input data set 1122 may include a first event cycle that defines a first event sequence and a second event sequence, the first event sequence corresponding to a first time frame and the second event sequence corresponding to a second time frame subsequent to the first event sequence. Here, the first event cycle refers to a sequence of events. The first event sequence refers to a first subset of events of the first event cycle, and the second event sequence refers to a second subset of events of the first event cycle.

[0062] As an example, if the input dataset 1122 includes data defining a rail network along with data indicating the current location, configuration, route, and destination of each train in the rail network, the progression of the train through different waypoints (e.g. stations) in a first trip may correspond to a first set of events, and the progression of the train through the waypoints in a second trip may correspond to a second set of events.

[0063] The graph generator 1152 is a functional unit for generating an initial selection graph (G0) based on the input data set 1122. Here, the initial selection graph may be a data structure in which a set of nodes and a set of edges are used to represent the progression of an element through a first event sequence defined by the input data set 1122. More specifically, the initial selection graph may define a first node set corresponding to a first event set of the first event sequence, a second node set corresponding to a second event set of the second event sequence, and a first neighborhood information set between each node of the first node set and the second node set. Here, the first neighborhood information set may include a predecessor node, a successor node, a disjunction node, a parallel node, a parallel directional disjunction complex node, and a backward directional disjunction complex node for each node of the initial selection graph.

[0064] The graph update unit 1154 updates the scheduling action (a τ ) for a particular transition in the selection graph, we select an effective scheduling action (A τ ) is the set of updated selection graphs (G τ The detailed process of the graph update unit 1154 will be described later in this specification, and therefore the description thereof will be omitted here.

[0065] The graph corrector 1156 is a functional unit for correcting a selection graph (e.g., an initial selection graph generated by the graph generator 1152 or an updated selection graph generated by the graph updater 1154). In an embodiment, the graph corrector 1156 corrects a revised selection graph (g) in which a second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set. τ ), where the structural similarity criteria may include a rule or standard that specifies the structural similarity between a first set of nodes and a second set of nodes.

[0066] In an embodiment, the graph corrector 1156 may determine that the second node set satisfies the structural similarity criterion with respect to the first node set if the second node set is a repetition (e.g., the second node set repeats in the same order as the first node set) or a reverse repetition of the first node set (e.g., the second node set repeats in the reverse order as the first node set). Here, a "composite node" refers to a single node used to represent multiple nodes in the selection graph while maintaining neighborhood information between the aggregate node and other nodes in the selection graph.

[0067] The scheduling unit 1158 is a functional unit for learning a policy 1160 for predicting a scheduling action based on the embedded graph. More specifically, as shown in FIG. 11, the scheduling unit 1158 embeds neighborhood information for each node of the revised selection graph to generate a policy 1160 for predicting a scheduling action based on the embedded graph (g τ (K) Based on the embedded graph, the scheduling unit 1158 determines a scheduling action (a τ ) for predicting the transitions of the embedded graph. By aggregating the scheduling actions predicted by the policy 1160 for multiple transitions of the embedded graph, it is possible to generate a schedule 1172 indicating recommended scheduling actions for the system defined by the input dataset 1122. This schedule 1172 may be temporarily stored in the storage unit 1161 and transmitted to the user terminal 1120 via a communication network such as the Internet.

[0068] The graph neural network embedding management system 1100 described above can reduce the computation time for embedding inter-node information, thereby enabling the application of GNNs to dynamic environments.

[0069] Next, an example of neighborhood information embedded in the nodes of the revised selection graph will be described with reference to FIG.

[0070] FIG. 12 illustrates an example of a neighborhood information set 1200 embedded in a node of a revised selection graph. As described herein, the neighborhood information set 1200 is information that defines relationships between nodes in the selection graph, and is embedded in each node of the selection graph by the node embedder 1159 shown in FIG. 11 to facilitate the generation of a schedule. Those relationships may define constraints on the progression of an element (e.g., a train) through the nodes of the selection graph. For example, a disjunction node may be used to express a blocking condition that prevents multiple elements from being scheduled to the same node at the same time.

[0071] As shown in FIG. 12, a neighborhood information set 1200 may define a predecessor node 1205, a successor node 1210, a disjunction node 1215, a parallel operation 1220, a parallel directional disjunction complex node 1225, and a backward disjunction complex node 1230 for each node of the revised selection graph.

[0072] As described herein, when this neighborhood information set 1200 is embedded into each individual node of the revised selection graph, if the number of nodes and edges is large (e.g., a train network has many stations and trains), the computation time for performing this embedding process increases, which becomes a challenge in large-scale networks. Therefore, as described herein, aspects of the present disclosure relate to providing a graph neural network embedding management technique for replacing multiple nodes (e.g., nodes that have structural similarity with other nodes in the selection graph) with a single composite node, thereby reducing the number of nodes and thereby reducing the computation time for embedding neighborhood information. In this way, it is possible to enable the application of GNNs in large-scale dynamic environments.

[0073] Next, with reference to FIG. 13, an exemplary flow of a graph neural network embedding management method according to an embodiment of the present disclosure will be described.

[0074] 13 is a diagram illustrating an example flow of a graph neural network embedding management method 1300 according to an embodiment of the present disclosure. The graph neural network embedding management method 1300 is a method for generating a schedule for a given scenario, and may be executed by various functional units of the graph neural network embedding management device 1150 shown in FIG.

[0075] Initially, in step S1310, the data input unit 1151 receives an input of an input data set. As described herein, the input data set may include table or matrix data defining a current state of the system for which it is desired to generate a schedule. In an embodiment, the input data set may include a first event cycle defining a first event sequence and a second event sequence, the first event sequence corresponding to a first time frame and the second event sequence corresponding to a second time frame subsequent to the first event sequence. In an embodiment, the input data set may be received from a user terminal. As an example, the input data set may define a configuration of a rail network, along with the location, route, and destination of each train in the rail network.

[0076] Next, in step S1320, the graph generator 1152 generates an initial selection graph having a first node set and a second node set. More specifically, the graph generator 1152 may generate, as the initial selection graph, a selection graph that defines a first node set corresponding to a first event set of a first event sequence, a second node set corresponding to a second event set of a second event sequence, and a first neighborhood information set between each node of the first node set and the second node set. Here, the first neighborhood information set may include a predecessor node, a successor node, a disjunction node, a parallel node, a parallel directional disjunction complex node, and a backward directional disjunction complex node for each node of the initial selection graph.

[0077] Next, in step S1330, the graph updater 1154 generates an updated selection graph indicating a set of valid scheduling actions for particular transitions of the selection graph based on the scheduling actions proposed by the scheduler 1158. It should be noted that during the first iteration of the graph neural network embedding management method 1300, since no scheduling actions have yet been proposed by the scheduler 1158, the graph updater 1154 does not generate an updated graph but passes the initial selection graph to the graph modifier 1156.

[0078] Next, in step S1340, the graph modifier 1156 generates a revised selection graph in which a set of nodes that meets the structural similarity criteria with respect to other sets of nodes in the revised selection graph are replaced with composite nodes. In an embodiment, the graph modifier 1156 may generate the revised selection graph by classifying the nodes in the selection graph as belonging to active event sequences (e.g., a moving train) or inactive event sequences (e.g., a train that is not moving), obtaining neighborhood information for the nodes that belong to the active event sequences, analyzing structural similarity between the nodes that belong to the active event sequences and the nodes that belong to the inactive event sequences, and replacing the nodes that belong to the inactive event sequences and meet the structural similarity criteria with composite nodes.

[0079] As described herein, in an embodiment, the graph corrector 1156 may determine that the second node set satisfies the structural similarity criteria with respect to the first node set if the second node set is a repeat (e.g., the second node set repeats in the same order as the first node set) or if the first node set is a reverse repeat (e.g., the second node set repeats in the reverse order as the first node set). In an embodiment, the graph corrector 1156 may determine whether the second node set satisfies the structural similarity criteria with respect to the first node set by analyzing the graph using existing graph analysis tools.

[0080] Next, in step S1350, the graph corrector 1156 defines disjunction edges for the composite nodes in the revised selection graph based on the neighborhood information set. Here, defining disjunction edges refers to assigning disjunction relationships between the composite nodes and other nodes in the revised selection graph based on their directionality. In this way, even if a certain node set is replaced with a composite node, the disjunction relationships between elements of the graph can be maintained.

[0081] Next, in step S1360, the node embedding unit 1159 generates an embedded graph by embedding neighborhood information for each node of the revised selection graph generated in step S1340. Details of this node embedding process will be described later.

[0082] As described herein, this node embedding can be considered as a feature vector that contains sufficient information about the target node and its relationship information in the graph. Here, embedding techniques such as K-layer graph neural networks (GNNs) can be used. Such GNNs have K embedding layers through which the revised selection graph is passed sequentially. These embedding layers represent embedding functions that can be approximated using multi-layer perceptrons (MLPs). In an embodiment, the embedding function for each layer can be written as follows:

number

[0083] In the above equation, there are six differentiable functions that represent different aspects of the neighborhood of the node of interest v, where h v k Let denote the k-th embedded form of node v. For each position, ReLu(x) = max(0,x), and || is the vector concatenation operator.

[0084] f pr (.;θ1) is the preceding node information Σ i∈Np(v) h i k-1is a predecessor updater that takes input from p (v) is the list of predecessors of node v. f sc (.;θ2) is the successor node information Σ i∈Ns(v) h i k-1 A successor node updater that uses N s (v) is the list of successors of node v. f d (.;θ3) is the logical sum node information Σ i∈Nd(v) h i k-1 A disjunction node updater that uses N s (v) is the list of successors of node v. f Tr,pl (.;θ4) is a composite node updater that uses information of composite nodes that represent event sequences with the same directivity as that represented by the target node v, and Σ i∈NTr,pl(v) h i k-1 N in Tr,pl (v) is a list of complex nodes in the parallel direction. f Tr,op (.;θ5) is a complex node updater that uses information of a complex node that represents an event sequence with a directionality opposite to that represented by the target node v, and Σ i∈NTr,op(v) h i k-1 N in Tr,op (v) is a list of complex nodes in the reverse direction. f t (.;θ6) is the target node updater. It uses the concatenated vector and returns a vector of the size of the initial node information vector. h i k-1 represents the k-1-th node embedding of the target node v. h v 0 represents the initial node embedding, which is the node information of the target node v before embedding.

[0085] Next, in step S1370, the scheduling unit 1158 determines a scheduling action for a current transition of the embedded graph. In an embodiment, as described herein, the scheduling action may include selecting a node from the embedded graph. Here, the scheduling action for a particular transition τ of the embedded graph is τ The embedded graph at a particular transition τ can be expressed as g τ (K) The set of executable actions (e.g., selectable nodes) may be represented by the updated selection graph generated in step S1330. Here, the set of executable actions may be represented by A gτ It can be represented by A gτ :{at τ a τ v ;v∈τ is a processable node}. In addition, a τ v denotes the action of choosing node v at transition τ. Furthermore, an actor can be defined to generate a probability distribution over the available (feasible) actions at the τth transition.

number

[0086] where f l (h u (K) ;θ7) is a differentiable function that maps node embeddings to logit values ​​for each node. The actor generates a probability distribution over the available (feasible) actions to be taken by the target machine using the SoftMax function as shown in Equation 2 above. One of the feasible scheduling actions is then selected from the above probabilistic policy by the scheduling unit 1158. To train this scheduling policy, a policy gradient based reinforcement learning algorithm such as Proximity Policy Optimization (PPO) may be used. Since the PPO algorithm uses a critic that approximates the state values ​​of the MDP, the critic V(g τ(K) ) may be defined as follows:

number

[0087] In response to determining a scheduling action for the current transition of the selection graph, the method returns to step S1330, where the graph updater 1154 generates an updated selection graph indicating a set of valid scheduling actions for the next transition of the selection graph based on the scheduling actions proposed by the scheduling unit 1158 in step S1370. This process may be repeated until a scheduling action has been generated for each transition of the selection graph.

[0088] Next, in step S1380, once scheduling actions have been generated for each transition in the selection graph, the scheduling unit 1158 may aggregate the generated scheduling actions as a schedule for the scenario defined in the input data set and output the generated schedule to a user terminal.

[0089] It should be noted that although an example has been described in which the input data set includes a first event cycle having a first event sequence and a second event sequence, the invention is not limited thereto and a scenario in which the input data set includes multiple event cycles is also possible. In an embodiment, the event cycles may be temporally parallel to each other and may correspond to the progress of different elements (e.g. trains) than the first event sequence. By way of example, in addition to the first event cycle defining the first event sequence and the second event sequence, the input data set may further include a second event cycle defining a third event sequence and a fourth event sequence. The third event sequence may correspond to the same time frame as the first event sequence, and the fourth event sequence may correspond to the same time frame as the second event sequence. In this case, the processing steps proceed as described above, with the set of nodes that satisfy the structural similarity criterion being replaced with a composite node and the embedding procedure being performed with respect to the revised selection graph.

[0090] The graph neural network embedding management method 1300 described above can reduce the computation time for embedding neighborhood information, thereby enabling the application of GNNs to scheduling problems in dynamic environments.

[0091] Next, an exemplary operation of a graph updater according to an embodiment of the present disclosure will be described with reference to FIG.

[0092] 14 illustrates an example operation 1400 of the graph updater 1154 according to an embodiment of the present disclosure. As described herein, the graph updater 1154 may be configured to generate an updated selection graph that indicates a set of valid scheduling actions for a particular transition of the selection graph.

[0093] More specifically, the graph update unit 1154 updates the action a τ Receive the selected graph and set the state of the selected graph to G τ From G τ+1 and the effective scheduling action for the next transition Aτ+1 If there is no valid action, the time t is incremented until a valid scheduling action is available. If there is at least one valid action, the next transition G τ+1 Updated selection graph and valid scheduling actions for A τ+1 The set of , is output to the graph modifier 1156 and the scheduler 1158, respectively. An example operation 1400 of the graph updater 1154 is described below.

[0094] First, in step S1410, the graph update unit 1154 updates the selected graph (G τ ) to identify nodes that belong to an active event sequence. In an embodiment, each node in the selection graph may be associated with a status indicator of 0100, 0010, or 0001, where 0100 indicates that the node is not yet occupied by an element (e.g., a train) and belongs to an inactive event sequence, 0010 indicates that the node is currently occupied by an element and belongs to an active event sequence, and 0001 indicates that the node was occupied by an element in the past but is not currently occupied and belongs to an inactive event sequence. In this case, the graph updater 1154 may identify all nodes that have a status indicator of 0010. As an example, the graph updater 1154 may identify node "a1" as belonging to an active event sequence.

[0095] Next, in step S1420, the graph update unit 1154 checks the remaining processing time of the node identified in step S1410 based on the neighborhood information. For example, referring to node a1 as shown in Fig. 7, the processing time for section a is 8 units, so the graph update unit 1154 may determine that the remaining processing time is 8 units.

[0096] Next, in step S1430, if the remaining processing time of the node determined in step S1420 is 0 units, the graph update unit 1154 extracts a successor node of the selected node. If the remaining processing time of the node is not 0, the method proceeds to step S1460. For example, the processing time for node a1 is 8 units, so no successor node is extracted.

[0097] Next, in step S1440, the graph update unit 1154 checks whether the section of the graph corresponding to the successor node extracted in step S1430 is occupied by an element (for example, a train).

[0098] Next, in step S1450, the graph update unit 1154 adds the nodes in said interval of the graph that are not occupied by elements to a list of nodes that can be selected as valid scheduling actions for the next transition 1407. In this way, the nodes that can serve as valid scheduling actions for the currently active event sequence can be determined.

[0099] Next, in step S1460, the graph update unit 1154 identifies nodes in the graph that have no predecessor nodes based on the neighborhood information. For example, the graph update unit 1154 may identify node "d2" as a node that has no predecessor nodes.

[0100] Next, in step S1470, the graph update unit 1154 adds the node identified in step S1460 to the list of valid scheduling actions for the next transition 1407. In this way, nodes that are not part of an active event sequence but are in a state where they can be scheduled are identified and may be included as candidates for the next scheduling action.

[0101] Next, in step S1480, the graph update unit 1154 increments the time to t+1 and updates the status of the node having a status indicator of

[0010] to

[0001] . In addition, the graph update unit 1154 sends the list of valid scheduling actions 1407 to the policy 1160 to predict the scheduling action for the next transition.

[0102] In this manner, the graph updater 1154 can generate an updated choice graph that indicates a set of valid scheduling actions for a particular transition of the choice graph.

[0103] Next, an example in which the graph neural network embedding management method according to an embodiment of the present disclosure is applied to a train scheduling problem will be described with reference to FIG. 15 and FIG. 16 .

[0104] Figures 15 and 16 show an example of the progression of a choice graph of a train scheduling scenario through six stages. The example in Figures 15 and 16 is shown for two trains, Train 1 and Train 2, performing three trips (denoted as "Trip" in Figures 15 and 16) over a rail network that includes eight sections, a, b, c, d, e, f, g, and h.

[0105] As shown in Figure 15, in the first stage 1510, train 1 starts its first run (run 11) on section a of the rail network, and train 2 starts its first run (run 21) on section h of the rail network. Note that at the start of the first stage 1510, a set of nodes corresponding to the second run of the first train (run 12), the third run of the first train (run 13), the second run of the second train (run 22), and the third run of the second train (run 23) are represented using composite nodes. As described herein, this is because the runs are repeats or reverse repeats of the first run of the first train or the first run of the second train, thereby satisfying the structural similarity criteria described herein. Assume that in the first step 1510, the scheduler 1158 selected node a11 for placement of train 1.

[0106] Then, in a second stage 1520, node a11 is assigned for train 1, and the scheduler 1158 selects node h21 for placement of train 2. In a third stage 1530, node h21 is assigned for placement of train 2. Node selection and assignment continues in this manner, with the scheduler 1158 selecting nodes for placement of train 1 and train 2. For example, it may be assumed that the scheduler 1158 selects nodes b11, d11, and g11 for train 1, and nodes e21, d21, and c21 for train 2.

[0107] As shown in FIG. 16, in a fourth step 1540, the scheduling unit 1158 selects node h11 for the assignment of the first train. Since node h11 is a node included in the composite node for service 12, in a fifth step 1550, the scheduling unit 1158 expands the composite node for service 12, and at the same time, replaces the node set corresponding to service 11 with the composite node, since the node corresponding to service 11 is no longer an active service (e.g., an active event cycle). In addition, the scheduling unit 1158 selects node a21 for the assignment of the second train. Similarly, since node a21 is a node included in the composite node for service 22, in a sixth step 1560, the scheduling unit 1158 expands the composite node for service 22, and at the same time, replaces the node set corresponding to service 21 with the composite node, since the node corresponding to service 21 is no longer part of the active service.

[0108] In this way, by replacing nodes that meet the structural similarity criteria corresponding to inactive event cycles (e.g., inactive train operations) with composite nodes and expanding those composite nodes when necessary, it is possible to reduce the number of nodes and edges in the selection graph, thereby decreasing the computation time for the embedding procedure. As a result, it may be possible to apply GNNs to scheduling problems in dynamic environments.

[0109] As described herein, conventional embedding methods perform embedding for each node of the selection graph. In contrast, the graph neural network embedding management method of the embodiment of the present disclosure modifies the selection graph to replace a set of nodes (e.g., a set of nodes that belong to inactive event sequences and have a repetitive structure) with composite nodes to reduce the total number of nodes and edges. For the existing embedding methods and the graph neural network embedding management method of the embodiment of the present disclosure, the computation time for the embedding procedure over three embedding layers of the GNN was tested. These tests were performed using different parameters such as the number of trains, the number of trips, and the number of railway sections. The results of the above tests for a single embedding (over three embedding layers of the GNN) are shown in Figures 17 to 19.

[0110] Next, the computation time for embedding neighborhood information is compared between the graph neural network embedding management technique according to the embodiment of the present disclosure and the existing technique, with reference to Figures 17 to 19. Note that in Figures 17, 18, and 19, the upper lines of the graphs correspond to the existing technique, and the lower lines of the graphs correspond to the graph neural network embedding management technique according to the embodiment of the present disclosure.

[0111] FIG. 17 shows a graph 1700 illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure and existing techniques in a scenario where the number of trains varies from 2 to 10 in a trip of 4 trains per train across 8 rail sections.

[0112] FIG. 18 shows a graph 1800 illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure and existing techniques in a scenario where the number of trips changes from 8 to 24 with 10 trains and 10 trips per train.

[0113] FIG. 19 shows a graph 1900 illustrating a comparison of computation time for embedding between a graph neural network embedding management technique according to an embodiment of the present disclosure and existing techniques in a scenario where the number of rail sections changes from 8 to 24 with 10 trains and 10 trips per train.

[0114] As can be seen from graphs 1700, 1800, and 1900, the graph neural network embedding management technique according to the embodiment of the present disclosure can reduce the computation time for the embedding process across various scenarios. This reduction in computation time for embedding is achieved as a result of reducing the total number of nodes and edges in the selection graph. For example, as shown in FIG. 18 and FIG. 19, at the largest test scale (10 trains, 24 sections, and 10 trips per train), the existing method took more than 450 seconds to compute the embedding, while the graph neural network embedding management technique was able to achieve the embedding of the revised selection graph in less than 20 seconds. This reduction in computation time makes GNNs applicable to scheduling problems in dynamic environments with strict time constraints, such as train scheduling problems.

[0115] As described herein, aspects of the present disclosure relate to extending the use of graph representation in reinforcement learning to scheduling scenarios in dynamic environments, such as a railway operation scheduling scenario. In an embodiment, a selection graph for a railway scheduling problem may be generated considering additional blocking constraints not present in standard JSSP. Unlike conventional approaches, the present selection graph representation is extended to complex scenarios involving multiple parallel graph intervals (e.g., scheduling multiple trains in parallel) and multiple event cycles (e.g., a train performing multiple trips). The computation time for the GNN embedding process in such scenarios using conventional approaches may be prohibitively long.

[0116] In view of this, the graph neural network embedding management technique according to the present disclosure is directed to reducing the size of the selection graph while maintaining the maximum amount of information. Thus, by utilizing the recurring structure present in some scheduling scenarios to represent a set of nodes corresponding to inactive event sequences (e.g., inactive train operations) as a single composite node, a revised selection graph can be generated with a reduced number of nodes and edges. By performing the embedding procedure on this revised selection graph, the computation time of the embedding procedure can be reduced to facilitate the rapid generation of scheduling solutions, thereby enabling the application of GNNs to scheduling scenarios (e.g., train scheduling) in dynamic environments.

[0117] A further aspect of the present disclosure relates to the use of a proximity policy optimization method for reinforcement learning, which facilitates flexible modification of the revised choice graph and allows for the generation of a generalized choice graph for train scheduling, such that complex rail network elements such as intersections, multiple lines with different stops, etc. can be included.

[0118] It should be noted that in the description of the embodiments, reference is made to generating and modifying a selection graph, where generating a graph may refer to the process of generating data in tabular or matrix form that represents the various nodes and edges of the graph, and does not require the generation of a visual graph representation itself. Similarly, modifying a selection graph may include adding, deleting, or altering tabular or matrix data that represents the graph, and does not require modification of the visual graph representation.

[0119] The present invention may be a system, a method, and / or a computer program product. The computer program product may include computer readable storage medium(s) having computer readable program instructions for causing a processor to perform aspects of the present invention.

[0120] A computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, punch cards or mechanical encryption devices such as ridged structures with instructions recorded in the grooves, and any suitable combination of the above. As used herein, a computer-readable storage medium should not be construed as being, per se, a transitory signal, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted by an electrical wire.

[0121] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0122] The above computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to manufacture a machine, such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for performing the functions / operations specified in the flowchart and / or block diagram blocks. These computer readable program instructions may further be stored on a computer readable storage medium that can cause the computer, programmable data processing apparatus, and / or other apparatus to function in a particular manner, such that the computer readable storage medium having the instructions stored thereon comprises an article of manufacture containing instructions that implement aspects of the functions / operations specified in the flowchart and / or block diagram blocks.

[0123] The above computer readable program instructions may further be loaded into a computer, other programmable data processing device, or other device to cause a series of operational steps to be executed on the computer or other programmable device to create a computer-implemented process, such that the instructions executing on the computer, other programmable device, or other device perform the functions / operations specified in the flowchart and / or block diagram blocks.

[0124] Embodiments according to the present disclosure may be provided to end users via a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing facility that provides an abstraction between computing resources and their underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be provisioned and released at high speed with minimal management effort or service provider interaction. Cloud computing thereby allows users to access virtual computing resources in the "cloud" (e.g., storage, data, applications, and even comprehensive virtualized computing systems) without regard for the underlying physical systems (or the location of those systems) used to provide the computing resources.

[0125] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing a specialized logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than the order noted in the drawings. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may possibly be executed in reverse order, depending on the functionality involved. It will also be appreciated that each block of the block diagrams and / or flowchart illustrations, as well as combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that executes a specialized function or operation, or executes a combination of dedicated hardware and computer instructions.

[0126] Although the above relates to exemplary embodiments of the present invention, other and further embodiments may be devised without departing from the basic scope of the present invention, the scope of which is determined by the following claims. The description of various embodiments of the present disclosure has been provided for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used in this specification are selected to explain the principles of the embodiments, practical applications or technical improvements of existing technologies in the market, or to enable others with ordinary skill in the field to which the disclosure belongs to understand the embodiments disclosed herein.

[0127] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the various embodiments. As used herein, the singular forms "a," "an," and "the" are intended to include the plural as well, unless the context clearly indicates otherwise. "Set," "group," "bundle," and the like are intended to include one or more. Furthermore, it will be understood that the terms "comprise" and / or "comprising" as used herein specify the presence of stated features, wholes, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. In the above detailed description of exemplary embodiments of the various embodiments, reference has been made to the accompanying drawings, in which like numerals indicate like elements, which form a part of this specification, by way of example, certain exemplary embodiments are shown and in which the various embodiments may be practiced. The above embodiments have been described in sufficient detail to enable one skilled in the art to practice the embodiments, but other embodiments may be used and logical, mechanical, electrical, and other changes may be made without departing from the scope of the various embodiments. Numerous specific details have been set forth in the above description to provide a thorough understanding of the various embodiments. However, the various embodiments may be practiced without those specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail so as not to obscure the embodiments. [Explanation of symbols]

[0128] 100 Computer Systems 102 processors 104 Memory 106 Memory Bus 108 I / O bus 109 Bus IF 110 I / O bus interface 112 Terminal Interface 113 Memory Interface 114 I / O device interface 115 Network Interface 116 User I / O Device 117 Storage device 124 Display System 126 Display device 130 Network 150 Scheduling Management Application

Claims

1. a data input for receiving an input data set including a first event cycle defining a first event sequence and a second event sequence; a graph generator for generating an initial selection graph for representing the first event cycle, the initial selection graph defining a first set of nodes corresponding to a first set of events in the first event sequence, a second set of nodes corresponding to a second set of events in the second event sequence, and a first set of neighborhood information defining relationships between each node of the first set of nodes and the second set of nodes; a graph modifier for generating a revised selected graph in which the second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; a node embedding unit for generating an embedded graph based on the revised selection graph, in which the first set of neighborhood information is embedded into each node of the first node set and the second node set; A graph neural network embedding management device comprising:

2. 2. The graph neural network embedding management apparatus of claim 1, wherein the graph corrector determines that the second node set satisfies the structural similarity criterion with respect to the first node set if the second node set is a repeat or a reverse repeat of the first node set.

3. the first set of nodes and the second set of nodes represent stages of a first element's progression through the first cycle of events; 2. The graph neural network embedding manager of claim 1, wherein the first sequence of events corresponds to a first time interval and the second sequence of events corresponds to a second time interval subsequent to the first time interval.

4. the input data set further includes a second event cycle defining a third event sequence and a fourth event sequence; the graph generator generates, as the initial selection graph, a selection graph for representing the first event cycle and the second event cycle, which defines: the first node set corresponding to the first event set of the first event sequence, the second node set corresponding to the second event set of the second event sequence, a third node set corresponding to a third event set of the third event sequence, a fourth node set corresponding to a fourth event set of the fourth event sequence, and a second neighborhood information set defining relationships between each node of the first node set, the second node set, the third node set, and the fourth node set; the graph modification unit generates, as the revised selection graph, a selection graph in which the second node set is replaced by the first composite node and the fourth node set is replaced by a second composite node if the fourth node set satisfies the structural similarity criterion with respect to the third node set; 4. The graph neural network embedding management device according to claim 3, wherein the node embedding unit generates, as the embedded graph, a selection graph in which the second neighborhood information set is embedded in each node of the first node set, the second node set, the third node set, and the fourth node set.

5. the third set of nodes and the fourth set of nodes represent stages of progression of a second element through the second cycle of events; 5. The graph neural network embedding manager of claim 4, wherein the third sequence of events corresponds to the first time interval and the fourth sequence of events corresponds to the second time interval.

6. 6. The graph neural network embedding manager of claim 5, wherein the second set of neighborhood information defines constraints on the progression of the first element through the first event cycle and constraints on the progression of the second element through the second event cycle.

7. a scheduling unit configured to generate a policy based on the embedded graph, the policy predicting a scheduling action for a first transition of the embedded graph; a graph updater configured to generate an updated selection graph indicating a set of valid scheduling actions for a second transition of the embedded graph based on the scheduling actions; The graph neural network embedding manager of claim 6, further comprising:

8. the first element is a first vehicle; the second element is a second vehicle; the first sequence of events is a first trip of the first vehicle; the second sequence of events is a second trip of the first vehicle; the third sequence of events being a first trip of the second vehicle; the fourth sequence of events is a second trip of the second vehicle; each node in the first node set, the second node set, the third node set, and the fourth node set corresponds to a waypoint of a vehicle route; 8. The graph neural network embedding management apparatus of claim 7.

9. A graph neural network embedding management device comprising a data input unit, a graph generation unit, a graph modification unit, a node embedding unit, and a scheduling unit, comprising: the data input unit receiving an input data set including a first route defining a first trip of a first vehicle and a second trip of the first vehicle; generating an initial selection graph representing the first route, the graph generator defining a first set of nodes corresponding to a first set of waypoints of the first journey, a second set of nodes corresponding to a second set of waypoints of the second journey, and a first set of neighborhood information defining relationships between each node of the first set of nodes and the second set of nodes; generating a revised selected graph in which the second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; the node embedding unit generating an embedded graph based on the revised selection graph, in which the first set of neighborhood information is embedded in each node of the first node set and the second node set in the revised selection graph; the scheduling unit determining a scheduling action for the first vehicle using a graph neural network that processes the embedded graph; A graph neural network embedding management method comprising:

10. A graph neural network embedding management device; User terminal and A graph neural network embedded management system comprising: The graph neural network embedding management device comprises: a data input for receiving an input data set from said user terminal including a first event cycle defining a first event sequence and a second event sequence; a graph generator for generating an initial selection graph to represent the first event cycle defining a first node set corresponding to a first set of events in the first event sequence, a second node set corresponding to a second set of events in the second event sequence, and a first set of neighborhood information defining a relationship between each node of the first node set and the second node set, the first node set and the second node set representing stages of progression of a first element through the first event cycle, the first event sequence corresponding to a first time segment and the second event sequence corresponding to a second time segment subsequent to the first time segment; a graph modifier for generating a revised selected graph in which the second node set is replaced by a first composite node that aggregates the second node set if the second node set satisfies a structural similarity criterion with respect to the first node set; a node embedding unit for generating an embedded graph based on the revised selection graph, in which the first set of neighborhood information is embedded in each node of the first node set and the second node set in the revised selection graph; a scheduling unit configured to generate a schedule defining a progression of the first element through the first event cycle based on the embedded graph and to output the schedule to the user terminal; 1. A graph neural network embedded management system comprising:

Images