Generation apparatus, generation method, and generation program

Abstract

This facilitates the generation of training data necessary for performing reinforcement learning with high accuracy. [Solution] The generation device 100 is a device that generates training data for training a reinforcement learning model used in a production scheduler that plans a predetermined production schedule. The generation device 100 randomly selects target products from a group of products included in an order list, which is data on product orders, so as to satisfy conditions regarding the number of products to be ordered for production. The generation device 100 adds the minimum order quantity of the selected target products to the total order quantity, which is the sum of the order quantities for each product included in the order list, so as to satisfy predetermined constraint conditions regarding the order quantity of the products. The generation device 100 assigns delivery dates to the total order quantity to which the minimum order quantity has been added. The generation device 100 outputs an order list, which includes the total order quantity to which the minimum order quantity has been added and to which delivery dates have been assigned, as training data.

Description

[Technical Field]

[0001] The present invention relates to a generation apparatus, a generation method, and a generation program. [Background technology]

[0002] In production sites and other settings, a technique is known that uses a reinforcement learning-based allocation selection unit to plan and output a schedule for a list of product orders, in order to improve productivity while strictly adhering to delivery deadlines, taking into account system constraints such as equipment capacity and constraints on the workers operating the equipment (see, for example, Patent Document 1).

[0003] The reinforcement learning described above can learn a general and effective strategy for finding a schedule that is applicable to variable information when there is some fixed information, by learning various patterns of inventory and orders in the schedules it has experienced. However, scheduling techniques using reinforcement learning require a large amount of training data to train the reinforcement learning model. Therefore, a technique is known in which the distribution of the original data is analyzed along a certain feature axis, and new data is generated based on the distribution of the original data (see, for example, Non-Patent Document 1). [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-177565 [Non-patent literature]

[0005] [Non-Patent Document 1] Machine learning-based software sensors for machine state monitoring - The role of SMOTE-based data augmentation,Alex Kummer et al., <URL:https: / / www.sciencedirect.com / science / article / pii / S2590123022004480 / pdfft?md5=a00556af25e05e4c642b8d5b9bb7d5ed&pid=1-s2.0-S2590123022004480-main.pdf> ,<Search date: November 25, 2020> [Overview of the project] [Problems that the invention aims to solve]

[0006] However, the conventional techniques described above have challenges in generating training data necessary for accurate reinforcement learning. For example, conventional techniques may generate unrealistic data that does not exist in the original data, and training a model with data generated using conventional techniques may result in degraded inference performance. [Means for solving the problem]

[0007] Therefore, in order to solve the above-mentioned problems and achieve the objective, the present invention is a generation device that generates training data for training a reinforcement learning model used in a production scheduler that plans a predetermined production schedule, and is characterized by comprising: a product selection unit that randomly selects target products from a group of products included in an order list, which is data relating to product orders, so as to satisfy conditions relating to the number of products to be ordered for production; an order adjustment unit that adds the minimum order quantity of the target products selected by the product selection unit to the total order quantity, which is the sum of the order quantities for each product included in the order list, so as to satisfy predetermined constraint conditions relating to the order quantity of the products; a delivery date assignment unit that assigns delivery dates to the total order quantity to which the minimum order quantity has been added by the order adjustment unit; and an output unit that outputs an order list including the total order quantity to which the minimum order quantity has been added by the order adjustment unit and to which the delivery date has been assigned by the delivery date assignment unit, as the training data. [Effects of the Invention]

[0008] The present invention has the effect of making it easier to generate training data for performing reinforcement learning with high accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a diagram illustrating the overall process performed by the generating apparatus according to this embodiment. [Figure 2] Figure 2 illustrates the challenges of SMOTE. [Figure 3] Figure 3 shows an example of the configuration of a generating apparatus according to an embodiment. [Figure 4] Figure 4 is a table diagram showing an example of the original dataset according to the embodiment. [Figure 5] Figure 5 is a table diagram showing an example of training data according to the embodiment. [Figure 6] Figure 6 shows an example of the data generation process for training according to the embodiment. [Figure 7]FIG. 7 is a diagram showing an example of a learning data generation process according to an embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of a procedure for a learning data generation process according to an embodiment. [Figure 9] FIG. 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of a generation device according to an embodiment. MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments (hereinafter referred to as "embodiments") will be described with reference to the drawings. In the following description, common components are denoted by the same reference numerals, and repeated descriptions are omitted. Also, the description of the embodiments described below does not limit the generation device, generation method, and generation program according to the present invention.

[0011] <Preliminary Explanation> First, a preliminary explanation of the present embodiment will be given. FIG. 1 is a diagram for explaining an overall view of the processing by the generation device 100 according to the present embodiment. The generation device 100 shown in FIG. 1 is an example of a computer that provides a technology for realizing the generation of appropriate learning data that satisfies a predetermined constraint condition with respect to the total value of the order quantities for each product.

[0012] (Background) In a production site, scheduling technologies for improving productivity while strictly observing delivery dates are being studied while considering system constraints such as equipment capabilities and constraints of workers who operate the equipment. However, there are problems in scheduling in a production site.

[0013] The scheduling problem in a production site is a problem of obtaining the production order of each product in each machine as a solution (schedule) based on fixed information (master data) specific to the factory, variable information (inventory, orders, etc.) and constraints, and objectives. Here, the master data refers to the production process order for each product, the number of machines constituting each process and the connection state between machines, the processing amount per unit time of each machine, the number of workers, and the like.

[0014] The scheduling problems described above often result in a vast number of solutions, making it difficult to explore all possibilities within a realistic timeframe. Therefore, it is generally difficult to efficiently find the true optimal solution, and the basic strategy is to search for the best possible approximate solution.

[0015] While the probability of obtaining a good solution generally increases with the number of search iterations in approximate solution search techniques, a long search time is required to obtain a practically satisfactory solution. For example, metaheuristic methods such as Genetic Algorithms are known as approximate solution search techniques.

[0016] The metaheuristic method described above does not learn general characteristics of scheduling problems; therefore, each time inventory, orders, etc., are changed, it becomes a different scheduling problem, requiring a long search time each time. Consequently, when scheduling devices using metaheuristic methods are put into operation on-site, this can sometimes contribute to a decrease in scheduling efficiency.

[0017] Therefore, in recent years, reinforcement learning has sometimes been used in the field of scheduling. Reinforcement learning is a method that consists of an agent and an environment, and repeats an experience step in which "the agent selects an action in a given state, the environment transitions to the next state that reflects that action, and evaluates the quality of that action as a reward" until a predetermined termination condition is met.

[0018] In reinforcement learning, an agent learns a policy to select the action that maximizes the value, which is the expected value of the cumulative sum of rewards (revenue) from a given experience step to the end state of the episode (the period from the initial state to the final state). Through various patterns of schedule inventory and orders experienced, reinforcement learning can learn a policy to find a general and effective schedule for variable information when given a certain amount of fixed information.

[0019] For example, in a production scheduler using reinforcement learning, training data is treated as tabular, and training is performed using tabular input data such as order lists and initial inventory lists (see, for example, Reference 1 listed below).

[0020] (Reference 1) Japanese Patent Publication No. 2020-177565

[0021] However, training a reinforcement learning model requires a large amount of training data. For example, if there is insufficient training data in reinforcement learning, the trained reinforcement learning model may overfit to a specific pattern and fail to generalize to unknown data other than the training data, potentially leading to decreased inference performance. Therefore, as a means of solving the above-mentioned data shortage problem, data augmentation techniques are sometimes used, which are processes that increase the amount of data by generating new data from existing datasets.

[0022] For example, a data augmentation technique for tabular data is SMOTE (Synthetic Minority Oversampling Technique), which generates data by sampling neighbors. Conventional SMOTE is a technique that analyzes the distribution of the original data along a certain feature axis and generates new data based on the distribution of the original data. SMOTE can balance the dataset by generating new samples between a small class of samples and its neighbors (see, for example, reference 2 listed below).

[0023] (Reference 2) Machine learning-based software sensors for machine state monitoring - The role of SMOTE-based data augmentation, Alex Kummer et al., <URL:https: / / www.sciencedirect.com / science / article / pii / S2590123022004480 / pdfft?md5=a00556af25e05e4c642b8d5b9bb7d5ed&pid=1-s2.0-S2590123022004480-main.pdf> ,<Search date: November 25, 2020>

[0024] However, SMOTE can generate unrealistic data that does not exist in the original dataset, and training a model with this generated unrealistic data can degrade its inference performance. Here, we will use Figure 2 to illustrate the challenges of SMOTE. Figure 2 is a diagram illustrating the challenges of SMOTE.

[0025] Figure 2 shows a graph of the monthly production order volume of products at a factory (hereinafter sometimes referred to as "order volume"), with the vertical axis representing the number, weight, volume, etc. of orders, and the horizontal axis representing "time (month)". The graph in Figure 2 shows product A (Figure 2 (1-1) to (1-3)) and product B (Figure 2 (2-1) to (2-3)). It is assumed that the "factory" in Figure 2 is constrained by production efficiency or product storage capacity to produce a total of 50 units of product per month.

[0026] For example, the total order quantity for January is "Product A: 39 units (Figure 2 (1-1)) + Product B: 9 units (Figure 2 (2-1)) = 48 units". Similarly, the total order quantity for February is "Product A: 29 units (Figure 2 (1-2)) + Product B: 19 units (Figure 2 (2-2)) = 48 units". Similarly, the total order quantity for March is "Product A: 49 units (Figure 2 (1-3)) + Product B: 1 unit (Figure 2 (2-3)) = 50 units".

[0027] On the other hand, in one example shown in Figure 2, when order quantity and time are considered as feature axes, data is generated based on SMOTE, represented by open circles (Figures (1-4) and (2-4)).

[0028] When the data generated based on SMOTE is totaled, it becomes "Product A: 49 units (Figure 2 (1-4)) + Product B: 18 units (Figure 2 (2-4)) = Total 67 units," which can result in the total order quantity exceeding 50 units. As mentioned above, conventional SMOTE has the problem of not being able to consider the total constraint because it independently determines the order quantity for each product based on the order quantity distribution for each product. In other words, conventional technology has the problem of generating training data necessary for accurate reinforcement learning.

[0029] (Overall overview of processing by the generation device 100) Therefore, the generation device 100 according to this embodiment generates a list of order data, which is data related to product orders such as product type, order quantity, and delivery date, as training data by expanding the original dataset that satisfies predetermined constraints on the total order quantity for each product.

[0030] In this embodiment, "constraints" refers to information regarding predetermined constraints on product order quantities, including the quantity of products that can be produced within a predetermined period at a factory, etc., and the minimum lot size (lower limit) for placing orders (production) for each product. Hereafter, this may simply be referred to as "constraints." In addition, a list of order data, which is data related to product orders such as product type, order quantity, and delivery date, may hereafter be referred to as the "order list."

[0031] Now, let's return to Figure 1 and continue the explanation. First, the generation device 100 selects a target product (hereinafter sometimes simply referred to as "target product") from a group of products that includes one or more products in the original dataset (order list) to be used for adjusting the order quantity (Figure 1 (1)).

[0032] Next, the generation device 100 adjusts the total order quantity included in the training data (order list) to satisfy the constraints, based on the minimum order quantity of the selected target product and the total order quantity obtained by summing the order quantities for each product included in the original dataset (order list) (Figure 1 (2)).

[0033] Next, the generation device 100 assigns delivery dates based on the production schedule to the total order quantity adjusted to satisfy the constraints (Figure 1(3)). Then, the generation device 100 outputs training data (order list) in which the total order quantity has been adjusted and delivery dates have been assigned (Figure 1(4)).

[0034] As described above, while conventional methods expanded data without considering constraints to generate training data, the generation device 100 can adjust the order quantity of each order data so that it falls within the upper and lower limits of the total order quantity of all products in the order list. Therefore, for example, using an unbalanced original dataset with uneven order frequencies for each product, it is possible to increase the training data for products with low order frequencies or decrease the training data for products with high order frequencies. Thus, the generation device 100 has the effect of making it easier to generate training data for performing reinforcement learning with high accuracy.

[0035] <Description of Generator 100> From here, the detailed functions of the generation apparatus 100 according to this embodiment will be described. Figure 3 is a diagram showing an example of the configuration of the generation apparatus 100 according to this embodiment.

[0036] The generation device 100 is a device that generates training data for training a reinforcement learning model used in a production scheduler that plans a predetermined production schedule. As shown in Figure 3, the generation device 100 has a communication unit 110, a storage unit 120, and a control unit 130. The generation device 100 also has an input unit (not shown) such as a keyboard or touch panel for receiving input from a user, and a display unit (not shown) such as a display or printer for displaying the results of information processing by the generation device 100 to the user.

[0037] (Communications Department 110) The communication unit 110 performs communication related to the output of generated training data (order list) and input of information regarding the source dataset (order list) and constraints used to generate the training data. The communication unit 110 is implemented by a NIC (Network Interface Card) or the like. The communication unit 110 can be connected to the network by wired or wireless connection as needed, and can send and receive information bidirectionally.

[0038] (Storage unit 120) The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) or flash memory, or storage devices such as hard disks or optical discs. The storage unit 120 stores data and programs used for various processes by the control unit 130. As shown in Figure 3, the storage unit 120 also has a source dataset DB 121, a constraints DB 122, and a training data DB 123.

[0039] (Original dataset DB121) The source dataset DB121 is a database that stores source datasets such as order lists used to generate training data. Here, an example of a source dataset stored in the source dataset DB121 will be explained using Figure 4. Figure 4 is a table diagram showing an example of a source dataset according to the embodiment.

[0040] The original dataset DB121 stores the "delivery date" and the "order quantity for each product" in association. For example, as shown in Figure 4, the original dataset DB121 stores time "0", product A "5", and product B "0" in a table format. Furthermore, the original dataset DB121 can store multiple original datasets in a format where a single table contains multiple products.

[0041] The "delivery date" mentioned above refers to information indicating the product's production schedule, and includes information indicating time based on a predetermined level of detail, such as "X days," "X weeks," or "X months." The "order quantity per product" includes information regarding the order quantity for each product in the order list, such as the number of units, weight, or volume to be ordered.

[0042] (Restriction condition DB122) The constraints DB122 is a database that stores information (constraints) regarding the constraints that the order adjustment unit 133, described later, considers when adjusting order quantities. Specifically, the constraints DB122 stores the "range of total order quantities" and the "minimum order quantity" as constraints.

[0043] The "Total Order Quantity Range" mentioned above includes information such as the upper and lower limits of the total order quantity for all products in the order list, which are determined based on the production capacity of each piece of equipment and each worker during the scheduling period. The "Minimum Order Quantity" includes the minimum order quantity (e.g., minimum lot size) for each product.

[0044] (Training data DB123) The training data DB123 is a database that stores training data (order lists) generated based on the original dataset (order lists). Here, an example of training data stored in the training data DB123 will be explained using Figure 5. Figure 5 is a table diagram showing an example of training data according to the embodiment.

[0045] The training data DB123 stores items similar to those stored in the source dataset DB121, such as "delivery date" and "order quantity per product," in a table format. For example, as shown in Figure 5, the training data DB123 stores time "0", product A "0", and product B "0" in a table format.

[0046] The training data stored in training data DB123 contains the same items as the original dataset stored in original dataset DB121, but since it is data-enhanced data, information such as order quantity may differ from the original dataset. In addition, training data DB123 can store multiple training data sets (order lists) generated based on the original dataset (order list).

[0047] (Control unit 130) The control unit 130 is implemented by a processor, MPU (Micro Processing Unit), CPU (Central Processing Unit), etc., executing various programs stored in the memory unit 120 using RAM as a working area. The control unit 130 is also implemented by an IC (Integrated Circuit), such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). As shown in Figure 3, the control unit 130 includes an acquisition unit 131, a product selection unit 132, an order adjustment unit 133, a delivery date assignment unit 134, and an output unit 135.

[0048] (Acquisition part 131) The acquisition unit 131 acquires the original dataset (order list) to be used to generate training data (order list) based on the original dataset. The acquisition unit 131 then stores the acquired original dataset (order list) in the original dataset DB 121.

[0049] Furthermore, the acquisition unit 131 acquires information such as the "range of total order quantity" and "minimum order quantity," which represent the upper and lower limits of the subtotal order quantity for the product or product group, as constraint conditions. The acquisition unit 131 then stores the acquired constraint conditions in the constraint condition DB 122.

[0050] Note that the range of the total order quantity described above may be set by referring to the factory operation rules, the efficiency of production resources, the capacity of the product storage area, etc. Also, the range of the total order quantity may be set to a value predicted by the generation device 100 by referring to the distribution of the total order quantity in the original dataset.

[0051] (Product Selection Unit 132) The product selection unit 132 randomly selects target products from the product group included in the order list so as to satisfy the conditions regarding the number of products to be ordered for production. Specifically, the product selection unit 132 randomly selects target products from the product group based on the product selection probability distribution calculated using at least one of the order quantity and the order frequency for each product.

[0052] For example, the product selection unit 132 determines the number of product types to be included in the generated order list (learning data). Next, the product selection unit 132 samples from the target product group based on the product selection probability distribution until the determined number of product types is reached. Then, the product selection unit 132 selects target products based on the sampling results.

[0053] The above-described product selection probability distribution may be designed as a uniform distribution for all products. Also, when the order quantity and the order frequency are different for each product, the order quantity and the order frequency for each product may be obtained from the original dataset, and the product selection probability distribution may be generated based on the softmax function. Also, when the order of the order quantity and the order frequency is different for each product, the product selection probability distribution may be generated based on the softmax function after logarithmic transformation.

[0054] Also, a temperature parameter T (domain: 0 < T) may be set for the softmax function shown in the following mathematical formula (1), and with 1 < T, the product selection probability distribution may be designed. By designing based on the mathematical formula (1), the product selection probability distribution can be designed so that products with a small product selection probability are also selected.

[0055] [Number]

[0056] Note that the "Z" in formula (1) i " may be the order quantity or order frequency for product i, or the order quantity × order frequency, or the logarithmic transformation of either of the above. Also, "K" is the number of products. And "T" is the temperature parameter as described above.

[0057] Furthermore, noise may be added to the product selection probability distribution to increase the versatility of the selected products. The noise may be designed to be distributed equally across all products, or it may be designed to be distributed differently for each product, such as a random amount.

[0058] In this embodiment, the product selection unit 132 does not allow duplication of product types. The number of product types may be determined, for example, by sampling using a uniform distribution within the range of the upper and lower limits of the number of product types in the order list of the original dataset (range of total order quantity).

[0059] (Order Adjustment Department 133) The order adjustment unit 133 adds the minimum order quantity of the target product selected by the product selection unit 132 to the total order quantity, which is the sum of the order quantities for each product included in the order list, which is the original dataset, in order to satisfy the constraints.

[0060] For example, if the order adjustment unit 133 has a constraint that the total order quantity of all products must not exceed an upper limit, it will repeat the process of adding only the minimum order quantity of the target product selected by the product selection unit 132, as long as it does not exceed the upper limit.

[0061] Furthermore, if the order adjustment unit 133 has a constraint that the total order quantity of all products must be equal to or greater than a lower limit, it repeats the process of adding the minimum order quantity of each selected target product until it exceeds the lower limit.

[0062] Furthermore, if there are upper and lower limit constraints on the subtotal order quantity for a specified product group rather than the total order quantity for all products, the order adjustment unit 133 repeats the process of adding only the minimum order quantity for the relevant product, as described above, within a range that exceeds the lower limit or does not exceed the upper limit.

[0063] (Delivery date allocation section 134) The delivery date allocation unit 134 allocates delivery dates to the total order quantity, which has been adjusted by the order adjustment unit 133 to include the minimum order quantity. Specifically, the delivery date allocation unit 134 allocates delivery dates by dividing the total order quantity into predetermined granularity periods determined based on a pre-set product production schedule.

[0064] For example, in the case of a factory that manages production on a weekly basis, the delivery date assignment unit 134 selects the delivery date (production delivery date) on a weekly basis. Alternatively, the delivery date assignment unit 134 may obtain and use the delivery date distribution for each product included in the original dataset, or the delivery date distribution without distinction between products.

[0065] For example, if the scheduling period is four weeks, the delivery date assignment unit 134 can assign delivery dates to each of the four weeks from the first to the fourth week based on a delivery date distribution such as a uniform distribution.

[0066] (Output section 135) The output unit 135 outputs an order list as training data, which includes the total order quantity to which the minimum order quantity has been added by the order adjustment unit 133 and to which the delivery date has been assigned by the delivery date assignment unit 134.

[0067] Specifically, the output unit 135 can change the format of the training data (order list) to match the specifications of the production scheduler when inputting it to the production scheduler. For example, the output unit 135 can output based on a two-dimensional table, as shown in the table diagram in Figure 5, where the columns are "product type" and the value of each cell is "product order quantity".

[0068] (An example of processing) From here, an example of the training data generation process realized by the generation device 100 according to this embodiment will be described using Figures 6 and 7. Figures 6 and 7 are diagrams showing an example of the training data generation process according to this embodiment.

[0069] Figure 6 shows an example where only product constraints exist (the first example). Figure 7 shows an example where constraints exist on product groups (subtotals of multiple products) (the second example).

[0070] (Example 1) First, we will explain an example of a scenario where only product constraints exist, using Figure 6. Note that the first example generates a one-month order list in a single process; however, if you need to generate order lists for multiple months, you may perform this as a batch process.

[0071] The generation device 100 (acquisition unit) acquires the order list (original dataset) from the original dataset DB121 (Figure 6 (1-1)).

[0072] The generation device 100 (product selection unit) randomly generates the number of products K to be ordered. Subsequently, the generation device 100 (product selection unit) randomly generates a product list that includes the randomly generated number of products K (Figure 6 (1-2)). Then, based on the generated product list, the generation device 100 (product selection unit) generates an empty order list in which the order quantity for each product is all "0" (Figure 6 (1-3)).

[0073] The generation device 100 (product selection unit) obtains the "range of total order quantity" from the constraint conditions DB 122 (Figure 6 (2-1)). Next, the generation device 100 (product selection unit) randomly generates an upper limit M of the total order quantity, which is the sum of the order quantities for each product included in the empty order list, based on the obtained range of total order quantities (Figure 6 (2-2)). Next, the generation device 100 (product selection unit) obtains information on the minimum order quantity for each product included in the product list (minimum order quantity) from the constraint conditions DB 122 (Figure 6 (2-3)).

[0074] The generating device 100 (product selection unit) randomly selects "Target Product X" from the product list (Figure 6 (2-4)). Next, the generating device 100 (product selection unit) selects the minimum order quantity N for Target Product X from the minimum order quantities for each product included in the acquired product list. x Load it (Figure 6 (2-5)).

[0075] The generating device 100 (product selection unit) determines the minimum order quantity N for the target product X. x This is added to the total order quantity, which is the sum of the order quantities for each product included in the empty order list. The generating device 100 (product selection unit) then determines whether the total order quantity, to which the minimum order quantity of the target product X has been added, meets a predetermined condition (Figure 6 (2-6)).

[0076] For example, the generating device 100 (product selection unit) calculates the total order quantity and the minimum order quantity N for the first target product X, which is the target product. x If the sum of these values ​​is less than or equal to a predetermined threshold (for example, the upper limit of the total order quantity M) (Yes in (2-6) of Figure 6), then the minimum order quantity N for the target product X is determined. x Add it to the empty order list (Figure 6 (2-7)).

[0077] On the other hand, the generating device 100 (product selection unit) calculates the total order quantity and the minimum order quantity N for the first target product X, which is the target product. x If the sum of the above exceeds a predetermined threshold (for example, the upper limit M of the total order quantity) (No. in (2-6) of Figure 6), the selection of the first target product X is canceled, and the second target product Y is randomly selected as the target product (the process from (2-4) to (2-6) of Figure 6 is repeated). Then, the generating device 100 (order adjustment unit) sets the minimum order quantity N for the second target product Y selected by the product selection unit. y Add this to the total order quantity (Figure 6 (2-7)).

[0078] If the sum of the total order quantity in the empty order list and the minimum order quantity MIN for each product does not meet the condition (No. (2-8) in Figure 6), the generating device 100 (order adjustment unit) returns to the steps from (2-4) in Figure 6 and repeats the process.

[0079] On the other hand, the generation device 100 (order adjustment unit) generates training data based on the adjusted empty order list if the sum of the total order quantity of the empty order list and the minimum order quantity MIN for each product satisfies the condition (Yes in (2-8) of Figure 6) ((2-9) of Figure 6).

[0080] Next, the generation device 100 (delivery date assignment unit) performs the delivery date assignment process (Figure 6 (2-10)). Then, the generation device 100 (output unit) outputs training data (order list) in which the order quantity has been adjusted and the delivery date has been assigned (Figure 6 (2-11)).

[0081] (Second example) Next, we will explain an example of a case where there are constraints on the product group, using Figure 7. Note that the second example shows how to generate a list of orders for one month in a single process, but if you need to generate order lists for multiple months, you can do so as a batch process.

[0082] The generation device 100 (acquisition unit) retrieves the order list (original dataset) from the original dataset DB121 and generates a product list and an empty order list (Figure 7 (1-1) to (1-3)). Note that the processing in Figure 7 (1-1) to (1-3) is the same as the processing in Figure 6 (1-1) to (1-3), so the explanation is omitted in the description of the second example.

[0083] The generation device 100 (product selection unit) obtains the "range of total order quantity" from the constraint DB 122 (Figure 7 (2-1)). Next, based on the obtained range of total order quantity, the generation device 100 (product selection unit) randomly generates an upper limit M of total order quantity, which is the sum of the order quantities for each product included in the empty order list (Figure 7 (2-2)).

[0084] Next, the generation device 100 (product selection unit) generates an upper limit value mz of the total order quantity for each product group (Fig. 7 (2-3)). Next, the generation device 100 (product selection unit) acquires information (minimum order quantity) regarding the minimum order quantity for each product included in the product list from the constraint condition DB122 (Fig. 7 (2-4)).

[0085] The generation device 100 (product selection unit) randomly selects "target product X" from the product list (Fig. 7 (2-5)). Next, the generation device 100 (product selection unit) reads the minimum order quantity N x related to the target product X from the minimum order quantities for each product included in the acquired product list (Fig. 7 (2-6)).

[0086] The generation device 100 (product selection unit) adds the minimum order quantity N x related to the target product X to the total order quantity, which is the sum of the order quantities for each product included in the empty order list. Then, the generation device 100 (product selection unit) determines whether the total order quantity to which the minimum order quantity of the target product X is added satisfies a predetermined condition (Fig. 7 (2-7)).

[0087] For example, when the sum of the total order quantity and the minimum order quantity N x related to the first target product X, which is the target product, is less than or equal to a predetermined threshold (e.g., the upper limit value m x of the total order quantity for each product group G z to which the first target product N z belongs) (Yes in Fig. 7 (2-7)), the minimum order quantity N x of the target product X is added to the empty order list (Fig. 7 (2-8)).

[0088] On the other hand, when the sum of the total order quantity and the minimum order quantity N x related to the first target product X, which is the target product, is greater than a predetermined threshold (e.g., the upper limit value m x of the total order quantity for each product group Gz to which the first target product N zIf the number exceeds (No. in (2-7) of Figure 7), the selection of the first target product X is canceled, and the second target product Y is randomly selected as the target product (the process from (2-5) to (2-7) of Figure 7 is repeated). Then, the generating device 100 (order adjustment unit) sets the minimum order quantity N for the second target product Y selected by the product selection unit. y Add this to the total order quantity (Figure 7 (2-8)).

[0089] If the total order quantity of the empty order list and the minimum order quantity MIN for each product do not meet the condition (No. (2-9) in Figure 7), the generating device 100 (order adjustment unit) returns to the steps from (2-5) in Figure 7 and repeats the process.

[0090] On the other hand, the generation device 100 (order adjustment unit) generates training data based on the adjusted empty order list if the sum of the total order quantity of the empty order list and the minimum order quantity MIN for each product satisfies the condition (Yes in (2-9) of Figure 7) ((2-10) of Figure 7).

[0091] Next, the generation device 100 (delivery date assignment unit) executes the delivery date assignment process (Figure 7 (2-11)). Then, the generation device 100 (output unit) outputs training data (order list) in which the order quantity has been adjusted and the delivery date has been assigned (Figure 7 (2-12)).

[0092] (Processing procedure) From here, the processing procedure by the generation device 100 according to this embodiment will be explained with reference to Figure 8. Figure 8 is a flowchart showing an example of the procedure for generating training data according to this embodiment.

[0093] First, the acquisition unit 131 acquires the order list (S101). Next, the product selection unit 132 randomly selects target products from the product group included in the order list so as to satisfy the conditions regarding the number of products to be ordered for production (S102).

[0094] The order adjustment unit 133 adds the minimum order quantity of the target product to the total order quantity to satisfy the constraints (S103). Next, the delivery date assignment unit 134 assigns delivery dates to the total order quantity to which the minimum order quantity has been added (S104).

[0095] The output unit 135 outputs an order list as training data, which includes the total order quantity with the minimum order quantity added and the delivery date assigned (S105). Then, the generation device 100 completes the process.

[0096] (effect) Next, we will explain the effects of the generation device 100 according to this embodiment. Conventional data augmentation methods such as SMOTE have the problem that they cannot take into account the total constraint because they independently determine the order quantity for each product based on the order quantity distribution for each product.

[0097] Therefore, the product selection unit 132 of the generation device 100 according to this embodiment randomly selects target products from the product group included in the order list so as to satisfy the conditions regarding the number of products to be ordered for production. The order adjustment unit 133 of the generation device 100 adds the minimum order quantity of the target products selected by the product selection unit 132 to the total order quantity, which is the sum of the order quantities for each product included in the order list, so as to satisfy the constraints. The delivery date assignment unit 134 of the generation device 100 assigns delivery dates to the total order quantity to which the minimum order quantity has been added by the order adjustment unit 133. The output unit 135 of the generation device 100 outputs an order list including the total order quantity to which the minimum order quantity has been added by the order adjustment unit 133 and to which delivery dates have been assigned by the delivery date assignment unit 134 as training data.

[0098] As described above, the generation device 100 of this embodiment has the effect of making it easy to generate training data for training reinforcement learning with high accuracy. As a result, the generation device 100 of this embodiment generates appropriate training data in which the order quantities of the products constituting the order list are adjusted to satisfy the constraints, thereby improving the inference accuracy of the production scheduler compared to conventional technology that does not consider the constraints.

[0099] The product selection unit 132 randomly selects target products from the product group based on a product selection probability distribution calculated using at least one of the order quantity and order frequency for each product.

[0100] In this way, the generator 100 suppresses bias in the selection of products by selecting target products based on a product selection probability distribution that is based on the order quantity or order frequency. In other words, the generator 100 can fairly select target products from a list of potential target products. As a result, the generator 100 can also suppress bias in the data when generating training data, and can generate training data for reinforcement learning with high accuracy.

[0101] The product selection unit 132, if the sum of the total order quantity and the minimum order quantity for the first target product exceeds a predetermined threshold, deselects the first target product and randomly selects the second target product as the target product. The order adjustment unit 133 adds the minimum order quantity for the second target product selected by the product selection unit 132 to the total order quantity.

[0102] Through the process described above, the generating device 100 can appropriately adjust the order quantity so as not to exceed the upper limit or exceed the lower limit, even if there are constraints such as upper or lower limits on the total order quantity of all products, by gradually adding the minimum order quantity of each selected product to the order quantity of that selected product.

[0103] Furthermore, if the sum of the total order quantity and the minimum order quantity for the first target product exceeds a predetermined threshold for the product group to which the first target product belongs, the product selection unit 132 deselects the first target product and randomly selects the second target product as the target product. The order adjustment unit 133 adds the minimum order quantity for the second target product selected by the product selection unit 132 to the total order quantity.

[0104] Through the process described above, the generation device 100 can also be adjusted so as not to exceed the specified range, even when upper and lower limits are set for subtotal order quantities for each product group consisting of several product varieties, rather than the total order quantity for all products. Therefore, the generation device 100 has the effect of enabling the generation of appropriate training data that satisfies the constraints.

[0105] The delivery date assignment unit 134 assigns delivery dates by dividing the total order quantity and allocating it to predetermined granularity periods determined based on the pre-set product production schedule. Through the above process, the generation device 100 can generate appropriate training data that takes the production schedule into consideration.

[0106] <Variation> The following describes modifications that can be implemented by the generation apparatus 100 according to this embodiment.

[0107] (Data, etc.) The product, target product, order quantity, minimum order quantity, total order quantity, total order quantity order list (original dataset or training data), constraints, names of the functional parts of the generation device 100, steps, processes, names of steps or processes, etc., used in the description of the above embodiment are merely examples and can be changed at will.

[0108] For example, while it was explained that the source dataset DB121 stores "delivery date" and "product and individual product identification information" in association, the source dataset can be stored based on other items, content, and data formats, not just the example shown in Figure 4. Similarly, while it was explained that the constraints DB122 stores "total order quantity range" and "minimum order quantity" as constraints, constraints can be stored based on other items, content, and data formats. Furthermore, while it was explained that the training data DB123 stores "delivery date" and "product and individual product identification information" in association, the training data can be stored based on other items, content, and data formats, not just the example shown in Figure 5.

[0109] (Flowcharts, etc.) In flowcharts, each step may be rearranged as long as it does not create inconsistencies, and some steps may be omitted. Furthermore, conjunctions such as "next," "continue," "in addition," "at this time," and "on this occasion" in flowchart descriptions do not limit the order or timing of the processes in the flowchart.

[0110] (others) Of the processes described in the embodiments and modifications described above, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0111] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0112] The components described above include those that can be easily conceived by those skilled in the art, those that are substantially identical, and those that fall within the so-called equivalent range. Furthermore, the embodiments and modifications described above can be combined as appropriate, as long as the processing content is not contradictory.

[0113] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means," "circuit," etc. For example, a control unit can be replaced with a control means or a control circuit.

[0114] Although several embodiments have been described in detail above with reference to the drawings, these are merely examples, and it is possible to implement these embodiments in various modified and improved forms based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the invention.

[0115] <Hardware Configuration> The generation device 100 according to this embodiment is implemented by a computer 1000 having the configuration shown in Figure 9. Figure 9 is a hardware configuration diagram showing an example of a computer that implements the functions of the generation device 100 according to this embodiment. The computer 1000 has a configuration in which a CPU 1100, RAM 1200, ROM 1300, auxiliary storage device 1400, communication interface 1500, and input / output interface 1600 are connected by a bus 1800.

[0116] The CPU 1100 operates based on programs stored in the ROM 1300 or auxiliary storage device 1400, and controls various parts. The ROM 1300 stores boot programs executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0117] The auxiliary storage device 1400 stores programs executed by the CPU 1100, and data used by such programs. The communication interface 1500 receives data from other devices via a predetermined communication network NW (including wireless communication via a closed network in this embodiment) and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network NW. The CPU 1100 controls output devices such as displays and printers, and input / output devices 1700 such as keyboards and mice via the input / output interface 1600. The CPU 1100 acquires data from the input / output devices 1700 via the input / output interface 1600. The CPU 1100 also outputs the generated data to the input / output devices 1700 via the input / output interface 1600.

[0118] For example, when the computer 1000 functions as various devices according to this embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit by executing a program loaded on the RAM 1200. [Explanation of Symbols]

[0119] 100 generator 110 Communications Department 120 Storage section 121 Original Dataset DB 122 Constraint DB 123 Training Data Database 130 Control Unit 131 Acquisition Department 132 Product Selection Section 133 Order Adjustment Department 134 Delivery date allocation department 135 Output section

Claims

1. A generation device that generates training data for training a reinforcement learning model used in a production scheduler that plans a predetermined production schedule, A product selection unit randomly selects target products from a group of products included in an order list, which is data related to product orders, in order to satisfy the conditions regarding the number of products to be ordered for production. An order adjustment unit adds the minimum order quantity of the target product selected by the product selection unit to the total order quantity, which is the sum of the order quantities for each product included in the order list, in order to satisfy predetermined constraints regarding the order quantity of the product. A delivery date assignment unit assigns delivery dates to the total order quantity, which is calculated by adding the minimum order quantity using the order adjustment unit. An output unit outputs an order list as training data, which includes the total order quantity to which the minimum order quantity has been added by the order adjustment unit and to which the delivery date has been assigned by the delivery date assignment unit. A generating device characterized by having [a certain feature].

2. The aforementioned product selection unit is Based on a product selection probability distribution calculated using at least one of the order quantity and order frequency for each product, the target product is randomly selected from the product group. The generating apparatus according to feature 1.

3. The aforementioned product selection unit is If the sum of the total order quantity and the minimum order quantity for the first target product exceeds a predetermined threshold, The selection of the first target product is removed, and the second target product is randomly selected as the target product. The aforementioned order adjustment unit, The minimum order quantity for the second target product selected by the product selection unit is added to the total order quantity. The generating apparatus according to feature 1 or 2.

4. The aforementioned product selection unit is If the sum of the minimum order quantity for the first target product, which is the aforementioned target product, exceeds a predetermined threshold for the product group to which the first target product belongs, The selection of the first target product is removed, and the second target product is randomly selected as the target product. The aforementioned order adjustment unit, The minimum order quantity for the second target product selected by the product selection unit is added to the total order quantity. The generating apparatus according to feature 1 or 2.

5. The aforementioned delivery date allocation unit is: The delivery date assignment is performed by dividing the total order quantity and allocating it to a predetermined granularity period determined based on a pre-set product production schedule. The generating apparatus according to feature 1 or 2.

6. A generation method to be executed by a generation device that generates training data for training a reinforcement learning model used in a production scheduler that plans a predetermined production schedule, A product selection process involves randomly selecting target products from a group of products included in an order list, which is data related to product orders, in order to meet the conditions regarding the number of products to be ordered for production. An order adjustment step which adds the minimum order quantity of the target product selected in the product selection step to the total order quantity which is the sum of the order quantities for each product included in the order list, in order to satisfy predetermined constraints regarding the order quantity of the product; A delivery date allocation step is performed to allocate delivery dates to the total order quantity, which is obtained by adding the minimum order quantity through the order adjustment step. Output step, which outputs an order list as training data, including the total order quantity to which the minimum order quantity has been added by the order adjustment step and to which the delivery date has been assigned by the delivery date assignment step, A method for generating a product, characterized by including the following:

7. A generation program to be executed by a generation device that generates training data for training a reinforcement learning model used in a production scheduler that plans a predetermined production schedule, A product selection procedure involves randomly selecting target products from a group of products included in an order list, which is data related to product orders, in order to meet the conditions regarding the number of products to be ordered for production. An order adjustment procedure that adds the minimum order quantity of the target product selected by the product selection procedure to the total order quantity obtained by summing the order quantities of each product included in the order list, in order to satisfy predetermined constraints regarding the order quantity of the product; A delivery date assignment procedure for assigning delivery dates to the total order quantity, which is obtained by adding the minimum order quantity through the order adjustment procedure, Output procedure to output an order list as training data, which includes the total order quantity to which the minimum order quantity has been added by the order adjustment procedure and to which the delivery date has been assigned by the delivery date assignment procedure, A generation program characterized by including the following.

Images