Generation device, generation method, and computer-readable storage medium
By generating product selection, order adjustment, and delivery schedule allocation functions, the problem of generating inaccurate data in existing technologies has been solved, achieving high-precision learning data generation and improving the inference accuracy of the production scheduler.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AZBIL CORP
- Filing Date
- 2025-09-01
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies tend to generate inaccurate data when generating learning data for reinforcement learning, leading to degraded inference performance and failing to meet the accuracy requirements of production scheduling.
The generation device generates learning data, including product selection, order adjustment, and delivery date allocation, to meet the product order quantity and delivery date allocation under constraints, ensuring that the generated data meets the total order quantity and delivery date requirements of the production schedule.
It achieves high-precision learning data generation, improves the inference accuracy of the production scheduler, and ensures the effectiveness of the production scheduling scheme.
Smart Images
Figure CN122155589A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a generating apparatus, a generating method, and a computer-readable storage medium. Background Technology
[0002] As a method for improving productivity in production sites and other environments while strictly adhering to delivery deadlines, considering system constraints such as equipment capacity or constraints on operators, the following technique is known: For a list of product quantities, a scheduling scheme is formulated and output using an assignment selection unit that has been trained by a reinforcement learning unit (see, for example, Patent Document 1).
[0003] The reinforcement learning described herein can learn strategies for seeking general and effective scheduling schemes for variable information when given fixed information by observing various patterns of inventory and ordering of the scheduling schemes it has experienced. However, on the other hand, in scheduling techniques that utilize reinforcement learning, a large amount of training data is required to learn the reinforcement learning model. Therefore, techniques are known to analyze the distribution of the original data on a certain feature axis and generate new data based on the distribution of the original data (for example, see Non-Patent Literature 1).
[0004] [Existing Technical Documents]
[0005] [Patent Literature]
[0006] [Patent Document 1] Japanese Patent Application Publication No. 2020-177565
[0007] [Non-patent literature]
[0008] [Non-Patent Literature 1] Machine learning-based software sensors for machine state monitoring - The role of SMOTE-based data augmentation, Alex Kummer et al., <Uniform Resource Locator (URL): https: / / www.sciencedirect.com / science / article / pii / S2590123022004480 / pdfft?md5=a00556af25e05e4c642b8d5b9bb7d5ed&pid=1-s2.0-S2590123022004480-main.pdf>, <Retrieved November 25, 2024> Summary of the Invention
[0009] [The problem the invention aims to solve]
[0010] However, the prior art has challenges in generating learning data for reinforcement learning with good accuracy. For example, the prior art may generate inaccurate data that does not exist in the original data, and if the model is learned using data generated by the prior art, the inference performance may sometimes deteriorate.
[0011] [Technical means to solve the problem]
[0012] Therefore, in order to solve the aforementioned problem and achieve the objective, the generation apparatus of the present invention generates learning data for learning a reinforcement learning model used by a production scheduler that formulates a prescribed production scheduling plan. The generation apparatus is characterized by comprising: a product selection unit that randomly selects target products from a group of products included in an order list, which is data related to product ordering, in a manner that satisfies conditions related to the number of products to be ordered; an order adjustment unit that adds the minimum order quantity and the total order quantity of the target products selected by the product selection unit to satisfy prescribed constraints related to the order quantity of the products, the total order quantity being obtained by summing the order quantities of each product included in the order list; a delivery date allocation unit that allocates delivery dates for the total order quantity for which the minimum order quantity has been added by the order adjustment unit; and an output unit that outputs the learning data as an order list containing the total order quantity for which the minimum order quantity has been added by the order adjustment unit and the delivery date has been allocated by the delivery date allocation unit.
[0013] [The effects of the invention]
[0014] This invention enables the easy generation of learning data that can be used for reinforcement learning with good accuracy. Attached Figure Description
[0015] Figure 1 This is a diagram illustrating the overall image of the processing performed by the generation apparatus of the embodiment.
[0016] Figure 2 This is a diagram illustrating the topic of SMOTE.
[0017] Figure 3 This is a diagram illustrating an example of the structure of the generation apparatus of the embodiment.
[0018] Figure 4 This is a tabular diagram representing an example of the original dataset used in the implementation method.
[0019] Figure 5 This is a table diagram illustrating an example of learning data for implementing the method.
[0020] Figure 6 This is a diagram illustrating an example of the generation and processing of learning data in an implementation method.
[0021] Figure 7 This is a diagram illustrating an example of the generation and processing of learning data in an implementation method.
[0022] Figure 8 This is a flowchart illustrating an example of the process for generating and processing learning data in an implementation method.
[0023] Figure 9 This is a hardware structure diagram of an example of a computer that illustrates the functions of the generation device for implementing the embodiments.
[0024] Explanation of icon numbers
[0025] 100: Generating device
[0026] 110: Ministry of Communications
[0027] 120: Storage Department
[0028] 121: Original dataset DB
[0029] 122: Constraints DB
[0030] 123: Learning to use data databases
[0031] 130: Control Department
[0032] 131: Acquisition Department
[0033] 132: Product Selection Department
[0034] 133: Order Adjustment Department
[0035] 134: Delivery Schedule Allocation Department
[0036] 135: Output Department Detailed Implementation
[0037] Hereinafter, embodiments will be described with reference to the accompanying drawings (hereinafter referred to as "Embodiments"). Furthermore, in the following description, common structural components will be labeled with the same reference numerals, and repeated descriptions will be omitted. Additionally, the description of the embodiments described below does not limit the production apparatus, production method, or production program of the present invention.
[0038] <Preface>
[0039] First, a preface to this embodiment will be presented. Figure 1 This is a diagram illustrating the overall image of the processing performed by the generation apparatus 100 of this embodiment. Figure 1 The generation apparatus 100 shown is an example of a computer that provides a technology for generating appropriate learning data to achieve the specified constraints of the total value of the order quantity for each product.
[0040] (background)
[0041] Research is underway on scheduling techniques used in production sites that maximize productivity while strictly adhering to delivery deadlines, taking into account system constraints such as equipment capacity and operator limitations. However, challenges exist in production site scheduling.
[0042] Production scheduling is a problem that, based on the factory's inherent fixed information (master data), variable information (inventory, orders, etc.), constraints, and objectives, aims to determine the production sequence of each product within each machine as the solution (scheduling plan). Here, master data refers to the production sequence of each product, the number of machines constituting each process, the connection status between machines, the throughput of each machine per unit time, and the number of operators.
[0043] Regarding the scheduling problem, the total number of solutions often becomes enormous, making it difficult to search for all of them within realistic time. Therefore, it is generally difficult to efficiently find the truly optimal solution; the basic approach is to search for the best possible approximation.
[0044] Regarding approximate solution search techniques, generally, the more searches performed, the higher the probability of obtaining a good solution. However, to obtain a practically good solution, a long search time is required. For example, metaheuristic methods such as genetic algorithms are known as approximate solution search techniques.
[0045] Because the metaheuristic method does not learn general characteristics of scheduling problems, it creates new scheduling problems whenever there are changes in inventory or orders, each requiring a long search time. Therefore, in the stage of using scheduling devices that utilize metaheuristic methods in the field, it sometimes becomes a reason for reduced scheduling efficiency.
[0046] Therefore, reinforcement learning has been sometimes used in the field of scheduling in recent years. Reinforcement learning is a method that involves an agent and an environment, repeating the empirical steps of "in a certain state, the agent chooses an action to transition the environment state to the next state reflecting the action, and evaluates the quality of the action in the form of a reward" until a specified termination condition is met.
[0047] In reinforcement learning, an agent learns a strategy that selects actions that maximize value. This value is the expected sum of rewards (gains) accumulated from an empirical step up to the final state of an episode (the period from the initial state to the final state). Furthermore, reinforcement learning can learn strategies that are general and effective for variable information given a fixed set of scheduling schemes, by studying a stockpile of experienced scheduling schemes and various ordering patterns.
[0048] For example, in a reinforcement learning-based production scheduler, the learning data is treated as a table, and the learning process is performed using input data in the form of tables such as order lists or initial inventory lists (for example, see reference 1 described below).
[0049] (Reference 1) Japanese Patent Application Publication No. 2020-177565
[0050] However, training reinforcement learning models requires a large amount of training data. For example, in reinforcement learning, if there is insufficient training data, the learned reinforcement learning model may be overfitted to specific patterns and unable to generalize to unknown data outside the training data, resulting in reduced inference performance. Therefore, as a means to address the problem of insufficient data, data augmentation techniques are sometimes used, which are processes that increase the amount of data by generating new data from existing datasets.
[0051] For example, in data expansion techniques targeting tabular data, the Synthetic Minority Oversampling Technique (SMOTE) is known. Existing SMOTE techniques analyze the distribution of the original data along a certain feature axis and generate new data based on that distribution. SMOTE can achieve dataset balance by generating new samples between minority class samples and their neighboring samples (e.g., see reference 2 below).
[0052] (Reference 2) Machine learning-based software sensors for machine state monitoring - The role of SMOTE-based data augmentation, Alex Kummer et al., <URL: https: / / www.sciencedirect.com / science / article / pii / S2590123022004480 / pdfft?md5=a00556af25e05e4c642b8d5b9bb7d5ed&pid=1-s2.0-S2590123022004480-main.pdf>, <Accessed November 25, 2024>
[0053] However, SMOTE may generate inaccurate data that is not present in the original dataset. If the model is learned using this inaccurate data, inference performance may sometimes deteriorate. Here, we use... Figure 2 Explain the topic of SMOTE. Figure 2 This is a diagram illustrating the topic of SMOTE.
[0054] exist Figure 2In the text, a chart is presented for each month's production order quantity (hereinafter sometimes referred to as "order quantity") in the factory, with the vertical axis representing the "order quantity" (representing the quantity, weight, volume, etc. of the order) and the horizontal axis representing "time (month)". Furthermore, in... Figure 2 The chart shown illustrates product A ( Figure 2 (1-1) to (1-3) and product B ( Figure 2 (2-1) to (2-3)). Furthermore, in Figure 2 In the "factory" in the story, it is assumed that there is a constraint that a total of 50 products can be produced each month due to factors such as production efficiency or the capacity of the product storage warehouse.
[0055] For example, the total order volume in January was "Product A: 39 units ( Figure 2 (1-1))+Product B: 9 ( Figure 2 (2-1)) = 48. Similarly, the total order quantity for February was "Product A: 29 ( Figure 2 (1-2))+Product B: 19 ( Figure 2 (2-2)) = 48. Similarly, the total order quantity for March was "Product A: 49 ( Figure 2 (1-3)) + Product B: 1 ( Figure 2 (2-3)) = 50.
[0056] On the other hand, for example in Figure 2 In one example shown, when order quantity and time are considered as characteristic axes, data represented by hollow circles is generated based on SMOTE. Figure 2 (1-4) and (2-4)).
[0057] Furthermore, if the data generated based on SMOTE is aggregated, it becomes "Product A: 49 ( Figure 2 (1-4)) + Product B: 18 ( Figure 2 (2-4)) = 67 in total, and sometimes the total order quantity exceeds 50. As mentioned above, in the existing SMOTE, since the order quantity of each product is determined independently based on the order quantity distribution of each product, there is a problem that the total constraint cannot be taken into account. In other words, in the existing technology, there is a problem in generating learning data for reinforcement learning with good accuracy.
[0058] (Overall overview of the processing performed by the generating apparatus 100)
[0059] Therefore, the generation apparatus 100 of this embodiment generates a list of order data, such as product variety, order quantity, and delivery date, as learning data, by expanding the original dataset that satisfies the specified constraints on the total value of the order quantity for each product.
[0060] Furthermore, in this embodiment, "constraints" refers to information related to prescribed constraints concerning the quantity of products that can be produced in a factory or similar facility within a specified period, or the order quantity of products including the minimum batch size (lower limit) when ordering (producing) each product. This will sometimes be simply referred to as "constraints." Additionally, a list of order data related to product ordering, such as product variety, order quantity, and delivery date, will sometimes be referred to as an "order list."
[0061] Here, return to Figure 1 Continuing the explanation, firstly, the generation device 100 selects the target product (hereinafter sometimes simply referred to as the "target product") from the product group containing more than one product in the original dataset (order list) for the order quantity adjustment processing. Figure 1 (1)).
[0062] Next, the generation device 100 adjusts the total order quantity in the learning data (order list) in a manner that satisfies the constraints, based on the minimum order quantity of the selected object product and the total order quantity obtained by summing the order quantities of each product included in the original dataset (order list). Figure 1 (2)).
[0063] Next, the production unit 100 allocates delivery dates based on the production scheduling scheme to the total order quantity adjusted to meet the constraints. Figure 1 (3)). Then, the generating device 100 outputs learning data (order list) in which the total order quantity is adjusted and the delivery date is assigned ( Figure 1 (4)).
[0064] As described above, the generation device 100 expands the data to generate learning data without considering constraints in the prior art, but it can adjust the order quantity of each order data to converge to the upper and lower limits of the total order quantity of all products in the order list. Therefore, for example, it can use an imbalanced original dataset with biases in order frequency for each product, increasing the learning data for products with low order frequency, or decreasing the learning data for products with high order frequency. Thus, the generation device 100 effectively generates learning data for reinforcement learning with good accuracy.
[0065] <Description of Generating Device 100>
[0066] The detailed functions of the generation apparatus 100 in this embodiment will be described below. Figure 3 This is a diagram illustrating an example of the structure of the generation apparatus 100 according to an embodiment.
[0067] The generation device 100 is a device for generating learning data, which is used to learn the reinforcement learning model used by the production scheduler to formulate a prescribed production scheduling plan. Figure 3 As shown, the generation device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. Additionally, the generation device 100 includes an input unit (not shown) such as a keyboard or touchscreen for receiving input from users, or a display unit (not shown) such as a monitor or printer for displaying the results of information processing performed by the generation device 100 to users.
[0068] (Communication Department 110)
[0069] The communication unit 110 performs communications related to the output of the generated learning data (order list) and the input of information related to the original dataset (order list) or constraints used in the generation of the learning data. The communication unit 110 is implemented via a network interface card (NIC) or the like. Furthermore, the communication unit 110 can connect to a network via wired or wireless means as needed to transmit and receive information bidirectionally.
[0070] (Storage Department 120)
[0071] The storage unit 120 is implemented, for example, using semiconductor memory elements such as random access memory (RAM) and flash memory, or storage devices such as hard disks and optical disks. The storage unit 120 stores data and programs used by the control unit 130 for various processes. Additionally, such as Figure 3 As shown, the storage unit 120 has a raw dataset database (DB) 121, a constraint database (DB) 122, and a learning data database (DB) 123.
[0072] (Original dataset DB 121)
[0073] The original dataset DB 121 is a database that stores the original dataset, such as the order lists, used to generate the data for learning. Here, we use... Figure 4 This describes an example of the original dataset stored in the original dataset DB 121. Figure 4 This is a tabular diagram representing an example of the original dataset used in the implementation method.
[0074] The original dataset DB 121 stores a correspondence between "delivery date" and "order quantity per product". For example, as Figure 4As shown, the original dataset DB 121 stores time "0", product A "5", and product B "0" in a table format. Furthermore, the original dataset DB 121 can store multiple original datasets in a single table containing multiple products.
[0075] The "delivery date" refers to information about the product's production scheduling plan, including information on timeframes based on granularities such as "○ day," "○ week," and "○ month." Additionally, "order quantity per product" includes order quantity-related information such as the order quantity, order weight, and order volume for each product in the order list.
[0076] (Constraint DB 122)
[0077] The constraint database DB 122 is a database that stores information (constraints) related to the constraints considered by the order adjustment department 133 when adjusting the order quantity, as described later. Specifically, the constraint database DB 122 stores the "range of total order quantity" and the "minimum order quantity" as constraints.
[0078] The "total order quantity range" includes information such as the upper and lower limits of the total order quantity of all products in the order list, determined based on the production capacity of each piece of equipment and each operator during the scheduling period. Additionally, the "minimum order quantity" includes the lower limit of the order quantity (e.g., minimum batch size) for each product.
[0079] (Learning with Data DB 123)
[0080] DB 123, the learning data database, stores learning data (order lists) generated based on the original dataset (order list). Here, it uses... Figure 5 This illustrates an example of learning data stored in learning data DB 123. Figure 5 This is a table diagram illustrating an example of learning data for implementing the method.
[0081] The learning process uses data DB 123 to store the "delivery date" and "order quantity per product" of the same items as those in the original dataset stored by the original dataset DB 121. For example, as Figure 5 As shown, the learning data DB123 stores time "0", product A "0", and product B "0" in tabular form.
[0082] Furthermore, the learning data stored in learning data DB 123 contains the same items as the original dataset stored in original dataset DB 121, but since it is expanded data, information such as order quantity may differ from the original dataset. Additionally, learning data DB 123 can store multiple learning data sets (order lists) generated based on the original dataset (order list).
[0083] (Control Department 130)
[0084] The control unit 130 is implemented by using RAM as a working area to execute various programs stored in the storage unit 120 via a processor, microprocessor (MPU), or central processing unit (CPU). Alternatively, the control unit 130 may be implemented using an integrated circuit (IC) such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). Figure 3 As shown, the control unit 130 includes an acquisition unit 131, a product selection unit 132, an order adjustment unit 133, a delivery schedule allocation unit 134, and an output unit 135.
[0085] (Acquisition Department 131)
[0086] The acquisition unit 131 acquires the original dataset (order list) used to generate learning data (order list) based on the original dataset. Then, the acquisition unit 131 saves the acquired original dataset (order list) in the original dataset DB121.
[0087] Additionally, the acquisition unit 131 acquires information such as the "range of total order quantity" or "minimum order quantity" representing the upper and lower limits of the subtotal order quantity of the product or product group as a constraint condition. Then, the acquisition unit 131 stores the acquired constraint conditions in the constraint condition DB 122.
[0088] Furthermore, the range of the total order quantity can be set with reference to the factory's operating rules, the efficiency of production resources, the capacity of product placement areas, etc. Alternatively, the range of the total order quantity can also be set as a value predicted by the generation device 100 with reference to the distribution of the total order quantity in the original dataset.
[0089] (Product Selection Department 132)
[0090] The product selection unit 132 randomly selects target products from the product group included in the order list in a manner that satisfies conditions related to the number of products ordered. Specifically, the product selection unit 132 randomly selects target products from the product group based on a product selection probability distribution calculated using at least one of the order quantity and order frequency of each product.
[0091] For example, the product selection unit 132 determines the number of product varieties to be included in the order list (learning data) to be generated. Next, the product selection unit 132 samples from the target product group based on the product selection probability distribution until the determined number of product varieties is reached. Then, the product selection unit 132 selects the target products based on the sampling results.
[0092] Regarding the product selection probability distribution, a uniform distribution can be designed for all products. Alternatively, if the order quantity or frequency differs for each product, the order quantity or frequency for each product can be obtained from the original dataset, and the product selection probability distribution can be generated based on a softmax function. Furthermore, if the order quantity or frequency differs in magnitude for each product, a product selection probability distribution can be generated based on a softmax function after logarithmic transformation.
[0093] Alternatively, the temperature parameter T (domain: 0 < T) can be set to 1 < T for the soft maximization function shown in equation (1) below, to design the product selection probability distribution. By designing based on equation (1), the product selection probability distribution can be designed such that even products with low selection probabilities are selected.
[0094] [Number 1]
[0095]
[0096] Furthermore, the "Z" in equation (1) i "This can be the order quantity or order frequency related to product i, or order quantity × order frequency, or any of the above logarithmic transformation values. In addition, "K" is the number of products. In addition, as mentioned above, "T" is the temperature parameter.
[0097] Furthermore, to improve the versatility of the selected products, noise can be added to the product selection probability distribution. The noise can be designed to be a uniform amount across all products, or it can be designed to be a different amount for each product, such as a random quantity.
[0098] Furthermore, in this embodiment, the product selection unit 132 does not allow duplicate product varieties. Additionally, the number of product varieties can be determined, for example, by sampling within a uniform distribution across the upper and lower limits of the number of product varieties in the order list of the original dataset (the range of the total order quantity).
[0099] (Ordering Adjustment Department 133)
[0100] The order adjustment unit 133 adds the minimum order quantity of the target products selected by the product selection unit 132 to the total order quantity in a manner that satisfies the constraints. The total order quantity is obtained by summing the order quantities of each product included in the order list, which is the original dataset.
[0101] For example, if there is a constraint that the total order quantity of all products cannot exceed the upper limit, the order adjustment unit 133 repeatedly performs the process of adding the minimum order quantity of the target products within the range not exceeding the upper limit for the target products selected by the product selection unit 132.
[0102] In addition, when there is a constraint that the total order quantity of all products must be above the lower limit, the order adjustment unit 133 repeatedly performs the process of adding the order quantities of the selected target products in units of minimum order quantity until the lower limit is exceeded.
[0103] Furthermore, when there are upper and lower limits on the subtotal order quantity for a specific product group rather than the total order quantity for all products, the order adjustment unit 133 repeats the process of adding the minimum order quantity of the target products as described above within the range of exceeding the lower limit or not exceeding the upper limit.
[0104] (Delivery Schedule Allocation Department 134)
[0105] Delivery schedule allocation unit 134 allocates delivery schedules for the total order quantity, including the minimum order quantity, which has been adjusted by order adjustment unit 133. Specifically, delivery schedule allocation unit 134 performs delivery schedule allocation by dividing and allocating the total order quantity into periods with a defined granularity based on a pre-set product production scheduling plan.
[0106] For example, in a factory where production is managed weekly, the delivery schedule allocation unit 134 selects the delivery schedule (production delivery schedule) on a weekly basis. Alternatively, regarding the delivery schedule distribution, the delivery schedule allocation unit 134 can also obtain the distribution of delivery schedules for each product included in the original dataset, or a distribution of delivery schedules that is not differentiated for each product.
[0107] As an example, when the scheduling period is 4 weeks, the delivery period allocation unit 134 can allocate delivery periods for each of the first to fourth weeks based on a delivery period distribution such as a uniform distribution.
[0108] (Output Section 135)
[0109] The output unit 135 outputs an order list containing the total order quantity, which has been adjusted by the order adjustment unit 133 and the delivery date has been allocated by the delivery date allocation unit 134, as learning data.
[0110] Specifically, when inputting data to the production scheduler, the output unit 135 can change the format of the learning data (order list) according to the specifications of the production scheduler and output it. For example, the output unit 135 can be based on, for example... Figure 5 The output is a two-dimensional table with columns set to "Product Variety" and cell values set to "Product Order Quantity" as shown in the table diagram.
[0111] (An example of what was handled)
[0112] The following uses Figure 6 and Figure 7 This describes an example of the generation and processing of learning data implemented by the generation apparatus 100 of this embodiment. Figure 6 and Figure 7 This is a diagram illustrating an example of the generation and processing of learning data in an implementation method.
[0113] Figure 6 This is the only instance (the first case) where there are only constraints related to the product. Additionally, Figure 7 This is one example (the second example) of a situation where there are constraints regarding product groups (subtotals of multiple products).
[0114] (first example)
[0115] First, use Figure 6 The explanation states that this is "only one example of a situation involving product constraints." Furthermore, while the first example describes generating a one-month order list in a single process, generating order lists for multiple months can be done through batch processing.
[0116] The generation device 100 (acquisition unit) obtains the order list (original dataset) from the original dataset DB 121. Figure 6 (1-1)).
[0117] The generating device 100 (product selection unit) randomly generates the number of products K to be ordered. Then, the generating device 100 (product selection unit) randomly generates a product list containing the randomly generated number of products K. Figure 6 (1-2)). Then, the generating device 100 (product selection unit) generates an empty order list with all order quantities of each product set to "0" based on the generated product list. Figure 6 (1-3)).
[0118] The generating device 100 (product selection unit) obtains the "range of total order quantity" from the constraint DB 122. Figure 6 (2-1)). Next, the generating device 100 (product selection unit) randomly generates an upper limit value M for the total order quantity based on the range of the acquired total order quantity, wherein the total order quantity is the sum of the order quantities of each product included in an empty order list ( ). Figure 6 (2-2)). Next, the generating device 100 (product selection unit) obtains information (minimum order quantity) related to the minimum order quantity of each product included in the product list from the constraint condition DB 122. Figure 6 (2-3)).
[0119] The generating device 100 (product selection unit) randomly selects "object product X" from the product list. Figure 6 (2-4)). Next, the generation device 100 (product selection unit) reads the minimum order quantity N of the target product X from the minimum order quantity of each product included in the acquired product list. x ( Figure 6 (2-5)
[0120] The generating device 100 (product selection unit) determines the minimum order quantity N of the target product X. x The total order quantity is added to the total order quantity, which is the sum of the order quantities of each product included in an empty order list. Then, the generation device 100 (product selection unit) determines whether the total order quantity, including the minimum order quantity of the target product X, meets the prescribed conditions. Figure 6 (2-6)
[0121] For example, the total order quantity and the minimum order quantity N of the first target product X as the target product. x If the total value is below a specified threshold (e.g., the upper limit M of the total order quantity), then... Figure 6 (2-6) Yes), the generating device 100 (product selection unit) will determine the minimum order quantity N of the target product X. x Add to an empty order list ( Figure 6 (2-7)).
[0122] On the other hand, the total order quantity and the minimum order quantity N of the first target product X as the target product... x If the total value exceeds a specified threshold (e.g., the upper limit M for the total order quantity), Figure 6 (2-6) No), the generating device 100 (product selection unit) deselects the first target product X and randomly selects the second target product Y as the target product (repeated). Figure 6(Processing steps (2-4) to (2-6)). Then, the generating device 100 (order adjustment unit) will determine the minimum order quantity N of the second target product Y selected by the product selection unit. y Add to the total order quantity ( Figure 6 (2-7)).
[0123] If the total order quantity in an empty order list does not match the minimum order quantity (MIN) for each product, then the condition is not met. Figure 6 (2-8) No), the generating device 100 (ordering adjustment department) returns to Figure 6 The steps after (2-4) are repeated.
[0124] On the other hand, if the total order quantity of an empty order list and the minimum order quantity (MIN) of each product satisfy the condition ( Figure 6 (2-8) is), the generating device 100 (order adjustment unit) generates learning data based on the empty order list whose order quantity has been adjusted. Figure 6 (2-9)).
[0125] Next, the production unit 100 (delivery schedule allocation unit) performs the delivery schedule allocation process. Figure 6 (2-10)). Then, the generating device 100 (output unit) outputs learning data (order list) with the order quantity adjusted and the delivery date assigned (order list) (). Figure 6 (2-11)).
[0126] (Second example)
[0127] Next, use Figure 7 This illustrates "an example of a situation where there are constraints regarding product groups." Furthermore, in the second example, a one-month order list is generated in a single process; however, in cases where multiple months' order lists are generated, batch processing can be used.
[0128] The generation device 100 (acquisition unit) acquires the order list (original dataset) from the original dataset DB 121, and generates a product list and an empty order list. Figure 7 (1-1) to (1-3)). Furthermore... Figure 7 The processing of (1-1) to (1-3) and Figure 6 The treatment of (1-1) to (1-3) is the same, so the explanation is omitted in the description of the second example.
[0129] The generating device 100 (product selection unit) obtains the "range of total order quantity" from the constraint DB 122. Figure 7(2-1)). Next, the generating device 100 (product selection unit) randomly generates an upper limit value M for the total order quantity based on the range of the acquired total order quantity, wherein the total order quantity is the sum of the order quantities of each product included in an empty order list ( ). Figure 7 (2-2)).
[0130] Next, the generation device 100 (product selection unit) generates the upper limit value mz of the total order quantity for each product group. Figure 7 (2-3)). Next, the generating device 100 (product selection unit) obtains information (minimum order quantity) related to the minimum order quantity of each product included in the product list from the constraint condition DB 122. Figure 7 (2-4)
[0131] The generating device 100 (product selection unit) randomly selects "object product X" from the product list. Figure 7 (2-5)). Next, the generation device 100 (product selection unit) reads the minimum order quantity N of the target product X from the minimum order quantity of each product included in the acquired product list. x ( Figure 7 (2-6)
[0132] The generating device 100 (product selection unit) determines the minimum order quantity N of the target product X. x The total order quantity is added to the total order quantity, which is the sum of the order quantities of each product included in an empty order list. Then, the generation device 100 (product selection unit) determines whether the total order quantity, including the minimum order quantity of the target product X, meets the prescribed conditions. Figure 7 (2-7)).
[0133] For example, the total order quantity and the minimum order quantity N of the first target product X as the target product. x The total value is the product group G to which the first object product X belongs. z The specified threshold (e.g., the upper limit m of the total order quantity for each product group) z In the following situations ( Figure 7 (2-7) is that the generating device 100 (product selection unit) will determine the minimum order quantity N of the target product X. x Add to an empty order list ( Figure 7 (2-8) in the middle.
[0134] On the other hand, the total order quantity and the minimum order quantity N of the first target product X as the target product... x The total value exceeds the product group G to which the first object product X belongs. z The specified threshold (e.g., the upper limit m of the total order quantity for each product group)z In the case of () Figure 7 (2-7) No), the generating device 100 (product selection unit) deselects the first target product X and randomly selects the second target product Y as the target product (repeated). Figure 7 (Processing steps (2-5) to (2-7)). Then, the generating device 100 (order adjustment unit) will determine the minimum order quantity N of the second target product Y selected by the product selection unit. y Add to the total order quantity ( Figure 7 (2-8)
[0135] If the total order quantity in an empty order list does not match the minimum order quantity (MIN) for each product, then the condition is not met. Figure 7 (2-9) No), the generating device 100 (ordering adjustment department) returns to Figure 7 The steps after (2-5) are repeated.
[0136] On the other hand, if the total order quantity of an empty order list and the minimum order quantity (MIN) of each product satisfy the condition ( Figure 7 (2-9) is), the generating device 100 (order adjustment unit) generates learning data based on the empty order list whose order quantity has been adjusted. Figure 7 (2-10)).
[0137] Next, the production unit 100 (delivery schedule allocation unit) performs the delivery schedule allocation process. Figure 7 (2-11)). Then, the generating device 100 (output unit) outputs learning data (order list) with the order quantity adjusted and the delivery date assigned ()). Figure 7 (2-12)).
[0138] (Processing flow)
[0139] The following uses Figure 8 The process flow of the generation apparatus 100 in this embodiment is explained. Figure 8 This is a flowchart illustrating an example of the process for generating and processing learning data in an implementation method.
[0140] First, the acquisition unit 131 acquires the order list (S101). Next, the product selection unit 132 randomly selects target products from the product group included in the order list in a manner that satisfies conditions related to the number of products to be produced (S102).
[0141] The order adjustment unit 133 adds the minimum order quantity and the total order quantity of the target products in a manner that satisfies the constraints (S103). Next, the delivery schedule allocation unit 134 allocates the delivery schedule for the total order quantity, which includes the minimum order quantity (S104).
[0142] The output unit 135 outputs an order list containing the total order quantity including the minimum order quantity and the assigned delivery date as learning data (S105). Then, the generation device 100 ends the process.
[0143] (Effect)
[0144] The effects of the generation apparatus 100 in this embodiment will be explained below. In existing data expansion methods such as SMOTE, since the order quantity of each product is determined independently based on the order quantity distribution of each product, there is a problem that the aggregate constraint cannot be taken into account.
[0145] Therefore, in this embodiment, the product selection unit 132 of the generation apparatus 100 randomly selects target products from the product group included in the order list in a manner that satisfies conditions related to the number of products to be ordered. The order adjustment unit 133 of the generation apparatus 100 adds the minimum order quantity and the total order quantity of the target products selected by the product selection unit 132 in a manner that satisfies constraints, wherein the total order quantity is obtained by summing the order quantities of each product included in the order list. The delivery date allocation unit 134 of the generation apparatus 100 allocates delivery dates for the total order quantity for which the minimum order quantity has been added by the order adjustment unit 133. The output unit 135 of the generation apparatus 100 outputs an order list containing the total order quantity for which the minimum order quantity has been added by the order adjustment unit 133 and the delivery date allocation unit 134 has allocated delivery dates as learning data.
[0146] As described above, the generation apparatus 100 according to this embodiment has the effect of easily generating learning data for reinforcement learning with good accuracy. As a result, the generation apparatus 100 of this embodiment can improve the inference accuracy of the production scheduler compared with the prior art that does not consider the constraints by generating appropriate learning data that adjusts the order quantity of products constituting the order list in a manner that satisfies the constraints.
[0147] The product selection unit 132 randomly selects target products from the product group based on a product selection probability distribution calculated from at least one of the order quantity and order frequency of each product.
[0148] Thus, the generation device 100 suppresses bias in the selected product by choosing the target product based on a product selection probability distribution based on order quantity or order frequency. That is, the generation device 100 can select the target product from candidates that may become the target product in a balanced state. As a result, the generation device 100 can also suppress data bias when generating learning data, generating learning data for reinforcement learning with good accuracy.
[0149] If the sum of the total order quantity and the minimum order quantity of the first target product exceeds a predetermined threshold, the product selection unit 132 cancels the selection of the first target product and randomly selects a second target product as the target product. The order adjustment unit 133 adds the minimum order quantity of the second target product selected by the product selection unit 132 to the total order quantity.
[0150] Through the aforementioned processing, even when there are upper or lower limits on the total order quantity of all products, the generating device 100 can appropriately adjust the order quantity by gradually adding the order quantity of the selected products in units of the minimum order quantity of the selected goods, so as not to exceed the upper or lower limit.
[0151] Furthermore, if the sum of the total order quantity and the minimum order quantity of the first target product exceeds a predetermined threshold for the product group to which the first target product belongs, the product selection unit 132 deselects the first target product and randomly selects a second target product as the target product. The order adjustment unit 133 adds the minimum order quantity of the second target product selected by the product selection unit 132 to the total order quantity.
[0152] Through this processing, even when upper and lower limits are specified for the subtotal order quantity of each product group consisting of several product varieties, rather than the total order quantity of all products, the generation device 100 can be adjusted to not exceed the range. Therefore, the generation device 100 serves to appropriately generate learning data that satisfies the constraints.
[0153] The delivery schedule allocation unit 134 performs delivery schedule allocation by dividing and allocating the total order quantity into periods with a defined granularity determined based on a pre-set production scheduling plan for the products. Through this process, the generation device 100 can generate appropriate learning data that takes into account the production scheduling plan.
[0154] <Variation Example>
[0155] Hereinafter, a modified example implemented by the generation apparatus 100 of this embodiment will be described.
[0156] (Data, etc.)
[0157] The products, target products, order quantities, minimum order quantities, total order quantities, total order quantity order lists (original datasets or learning data), constraints, names of functional units of the generation apparatus 100, steps, processes, and names of steps or processes used in the description of the embodiments are merely examples and can be arbitrarily changed.
[0158] For example, it is explained that the original dataset DB 121 stores "delivery date" in a correspondence with "product and individual product identification information," but it is not limited to this. Figure 4 As shown in the example, the original dataset can be stored based on other items, content, and data formats. Furthermore, it is explained that constraint DB 122 stores the "range of total order quantity" and "minimum order quantity" as constraints, but constraints can be stored based on other items, content, and data formats. Additionally, it is explained that learning data DB 123 stores "delivery date" and "product and individual product identification information" in a corresponding manner, but is not limited to... Figure 5 As shown in the example, learning data can be stored based on other projects, content, and data formats.
[0159] (Flowcharts, etc.)
[0160] The steps in a flowchart can be interchanged and implemented within the bounds of non-contradiction, and there may also be steps that are not implemented. In addition, the connecting words such as "next," "following," "furthermore," "at this time," and "at this point" in the flowchart description do not limit the order or timing of the implementation of the processes in the flowchart.
[0161] (other)
[0162] All or part of the processes described in the embodiments and variations as automatically performed processes can also be performed manually, or all or part of the processes described as manually performed processes can be performed automatically by known methods. Furthermore, information regarding the processing flow, specific names, various data, or parameters shown in the documents or figures can be arbitrarily changed, unless specifically stated otherwise. For example, the various information shown in the figures is not limited to the information illustrated.
[0163] Furthermore, the structural components of each device shown in the illustrations are functional conceptual components and may not be physically constructed as depicted. That is, the specific forms of the dispersion and integration of each device are not limited to those shown in the illustrations. They can be constructed by dispersing and integrating all or part of them in any functional or physical manner, depending on various loads or usage conditions.
[0164] The structural components include structural components that can be easily conceived by those skilled in the art, substantially identical structural components, and structural components of equal scope. Furthermore, the embodiments and variations described above can be appropriately combined within the scope where the processing content is not contradictory.
[0165] Furthermore, the term "section" (or "module") or "unit" mentioned above can be replaced with "component" or "circuit," etc. For example, the control section can be replaced with a control component or control circuit.
[0166] The above description of several embodiments is based on the accompanying drawings. However, these embodiments are examples and can be represented by the forms described in the disclosure section of the invention. Various modifications and alterations can be made to these embodiments based on the knowledge of those skilled in the art.
[0167] <Hardware Structure>
[0168] The generating apparatus 100 of this embodiment, for example, is generated by, for example, as... Figure 9 The computer 1000 with the structure shown is implemented. Figure 9 This is a hardware structure diagram illustrating an example of a computer that implements the functions of the generation apparatus 100 of the embodiment. The computer 1000 has a configuration in which a CPU 1100, RAM 1200, Read Only Memory (ROM) 1300, auxiliary storage device 1400, communication I / F (interface) 1500, and input / output I / F (interface) 1600 are connected via a bus 1800.
[0169] The CPU 1100 operates based on programs stored in the ROM 1300 or auxiliary storage device 1400, and controls various components. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, or programs that depend on the hardware of the computer 1000.
[0170] Auxiliary storage device 1400 stores programs executed by CPU 1100 and data used by those programs. Communication I / F 1500 receives data from other machines and sends it to CPU 1100 via a defined communication network NW (in this embodiment, this includes wireless communication based on a closed area), and also sends data generated by CPU 1100 to other machines via the defined communication network NW. CPU 1100 controls output devices such as displays or printers, and input / output devices 1700 such as keyboards or mice via input / output I / F 1600. CPU 1100 acquires data from input / output devices 1700 via input / output I / F 1600. Furthermore, CPU 1100 outputs generated data to input / output devices 1700 via input / output I / F 1600.
[0171] For example, when the computer 1000 functions as a device in this embodiment, the CPU 1100 of the computer 1000 performs the functions of the control unit by executing a program loaded on the RAM 1200.
Claims
1. A generation apparatus for generating learning data, said learning data being used to learn a reinforcement learning model used by a production scheduler to formulate a prescribed production scheduling plan, the generation apparatus being characterized by having: The product selection department randomly selects target products from the product group included in the order list, which is data related to product ordering, in a manner that satisfies the conditions related to the number of products ordered for production. The order adjustment department adds the minimum order quantity of the target products selected by the product selection department to the total order quantity in a manner that satisfies the prescribed constraints related to the order quantity of the products. The total order quantity is obtained by summing the order quantities of each product included in the order list. The delivery schedule allocation department allocates delivery schedules for the total order quantity, which includes the minimum order quantity, through the order adjustment department. as well as The output unit outputs an order list containing the total order quantity, which has been adjusted by the order adjustment unit and the delivery date has been allocated by the delivery date allocation unit, as the learning data.
2. The generating apparatus according to claim 1, characterized in that, The product selection unit randomly selects the target product from the product group based on a product selection probability distribution calculated using at least one of the order quantity and order frequency of each product.
3. The generating apparatus according to claim 1 or 2, characterized in that, If the sum of the total order quantity and the minimum order quantity of the first target product exceeds a predetermined threshold, the product selection unit will deselect the first target product and randomly select a second target product as the target product. The order adjustment department adds the minimum order quantity of the second target product selected by the product selection department to the total order quantity.
4. The generating apparatus according to claim 1 or 2, characterized in that, If the sum of the total order quantity and the minimum order quantity of the first target product exceeds a predetermined threshold for the product group to which the first target product belongs, the product selection unit will deselect the first target product and randomly select a second target product as the target product. The order adjustment department adds the minimum order quantity of the second target product selected by the product selection department to the total order quantity.
5. The generating apparatus according to claim 1 or 2, characterized in that, The delivery schedule allocation department performs the allocation of the delivery schedule by dividing the total order quantity into periods of specified granularity and allocating the total order quantity. The specified granularity of the period is determined based on a pre-set production scheduling plan for the product.
6. A generation method, executed by a generation device, the generation device generating learning data, the learning data being used to learn a reinforcement learning model used by a production scheduler to formulate a prescribed production scheduling plan, the generation method being characterized by comprising: The product selection process randomly selects target products from the product group contained in the order list, which is data related to the order of products, in a manner that satisfies the conditions related to the number of products ordered for production. The order adjustment process adds the minimum order quantity of the target product selected through the product selection process to the total order quantity in a manner that satisfies the specified constraints related to the order quantity of the product. The total order quantity is obtained by summing the order quantities of each product included in the order list. The delivery schedule allocation process allocates delivery schedules for the total order quantity, which has been adjusted by the ordering process and includes the minimum order quantity. as well as The output process outputs an order list containing the total order quantity, which has been adjusted by the order adjustment process and the delivery date has been allocated by the delivery date allocation process, as the learning data.
7. A computer-readable storage medium storing a generation program executed by a generation device, the generation device generating learning data, the learning data generating a reinforcement learning model used by a production scheduler to learn and formulate a prescribed production scheduling scheme, characterized in that the generation program comprises: The product selection process randomly selects target products from the product group contained in the order list, which is data related to the order of products, in a manner that satisfies the conditions related to the number of products ordered for production. The order adjustment process adds the minimum order quantity and the total order quantity of the target products selected through the product selection process in a manner that meets the specified constraints related to the order quantity of the products. The total order quantity is obtained by summing the order quantities of each product included in the order list. The delivery schedule allocation process allocates delivery schedules for the total order quantity, which includes the minimum order quantity, through the order adjustment process. as well as The output process outputs an order list containing the total order quantity, which has been adjusted through the order adjustment process and the delivery date has been allocated through the delivery date allocation process, as the learning data output.