Industrial automatic production scheduling prediction method and system based on feedforward rule learning

By applying the feedforward rule learning method of user behavior in industrial automation production scheduling scenarios, using the Transformer model for data vector processing and pre-training, the problem of small data scale is solved, the efficiency and accuracy of production scheduling are improved, and the cost is reduced and technical threshold.

CN119067268AInactive Publication Date: 2024-12-03QINGDAO NOAH INT TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411562938.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-05
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the industrial automation production scheduling scenario, due to the small data scale, the existing technology fails to effectively utilize feedforward rules to learn, resulting in low automation level and high labor costs.

Method used

Using the feedforward rule learning method based on user behavior, by collecting production resources, production materials and production scheduling parameters information, selecting the Transformer model for data vector processing and pre-training, obtaining production scheduling data, and using the pre-training model to connect to the enterprise system to obtain Data required for production scheduling.

Benefits of technology

It significantly reduces the threshold and cost of production planning, improves the efficiency and accuracy of automated production scheduling, reduces labor costs, and adapts to the compatibility needs of fixed rules and flexible scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067268A_ABST
    Figure CN119067268A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of industrial automatic production scheduling data analysis, and discloses an industrial automatic production scheduling prediction method and system based on feedforward rule learning. The method comprises the following steps: collecting production resources, production materials and production scheduling parameter information in an industrial automatic production scheduling scene; selecting a machine learning model with feedforward rule learning according to the task type; carrying out vectorization processing on different data with an association relationship in the obtained user behavior data, obtaining influence factors of planned production scheduling, and outputting restored business data; performing pre-training for the ultra-small data scale to obtain production scheduling data; and using the pre-trained machine learning model with feedforward rule learning to connect with an existing system of an enterprise to obtain data required by production scheduling. According to the method, large-scale data for training is provided in multiple scenes in reality, the efficiency is improved by means of artificial intelligence, the automation level is improved, and the labor cost is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial automation production scheduling data analysis, and in particular, relates to an industrial automation production scheduling prediction method and system based on feedforward rule learning. Background Art

[0002] With the rapid development of artificial intelligence technology, machine learning algorithms have been widely used in all walks of life. However, traditional machine learning algorithms usually require a large amount of training data, which is not available in many real-world scenarios. Taking the industrial automation production scheduling scenario as an example, even if industrial enterprises are large in scale and have existed for a long time, the production scheduling-related data they generate is not worth mentioning compared to the data scale required for common deep learning model training at the TB level, and can only be regarded as an ultra-small data scale.

[0003] In order to solve these problems, feedforward rule learning methods based on ultra-small data scale have been widely studied in recent years. Feedforward rule learning is a rule-based machine learning method that learns rules from data generated by user historical behavior to make predictions or decisions. Compared with traditional machine learning methods, feedforward rule learning has the advantages of less data requirements, strong interpretability, and strong robustness.

[0004] Through the above analysis, the problems and defects of the existing technology are: in reality, the industrial automation scheduling scenario is not combined with feedforward rule learning, which makes the industrial automation scheduling lack large-scale data available for training, resulting in low efficiency and high labor costs in the industrial automation scheduling. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the disclosed embodiments of the present invention provide an industrial automation production scheduling prediction method and system based on feedforward rule learning, specifically involving a feedforward rule learning tool and training model research with ultra-small data scale based on user behavior.

[0006] The technical solution is as follows: an industrial automation production scheduling prediction method based on feedforward rule learning, the method comprising: S1, collects production resources, production materials, and scheduling parameter information in industrial automation scheduling scenarios; S2, based on the collected information, selects a machine learning model with feedforward rule learning according to the task type; S3, with the process as the center, integrates the user behavior data collected during the production scheduling process, obtains the correlation between different data, and uses a machine learning model with feedforward rule learning to vectorize the different data with correlation in the obtained user behavior data, obtains the influencing factors of planned production scheduling, and outputs the restored business data; wherein, vectorization processing is to process the business data related to production scheduling into vector data, and encode the data into vector values ​​by extracting the location, sequence and characteristics affecting production scheduling; in the process of encoding data into vectors, the vector values ​​corresponding to the data are recorded, and the vector output by the model is restored to business data; S4, for small data scale, pre-train the machine learning model with feed-forward rule learning for vectorization processing to obtain production scheduling data; S5 uses the pre-trained machine learning model with feedforward rule learning to connect to the company's existing system to obtain the data required for production scheduling.

[0007] In step S1, the production resources are hierarchical structures of factories, workshops, work centers, and stations; The production materials use the process bill of materials PBOM, which contains the material composition of the product and the corresponding production process of the materials; Scheduling parameters are used to configure scheduling behaviors, including production shifts, production calendar, production sequence, scheduling priority, production rhythm, bottleneck process, and station capacity. In the industrial automation production scheduling scenario, the production scheduling results are also collected. The production scheduling results include: the planned time and planned location of the process.

[0008] In step S2, a machine learning model with feedforward rule learning is selected according to the task type, including: S2.1, model technology selection, select the Transformer model as the basic model architecture selection; S2.2, data format processing, using the hierarchical relationship between production resources and production materials and the data sequence specified by the production scheduling parameters as the coding space to obtain structured data; in conventional production and manufacturing scenarios, the workshops, work centers, and stations in the production resources have a hierarchical relationship with the products, materials, and processes in the production materials. During the production scheduling process, the production priority is adjusted to achieve product priority production and optimize the process route; the hierarchical relationship is expressed as a hierarchical sequence number, the priority is expressed as a sequence number, the hierarchical relationship is expressed as a nested relationship in the structured data, and the sequence number is a specific value in the structured data; S2.3, training and application, input production resources, production materials, and production scheduling parameters, calculate the resource relationship between processes through the attention of the Transformer model, and calculate the relationship between process plans through the Transformer attention operation algorithm; output process scheduling plans through feedforward rules, compare them with the manual scheduling results, find the plan of the same process in the corresponding manual scheduling results for the process plan output by the model, compare the time difference and the position difference between the two, and obtain the mean square error as the comparison result; adjust the model parameters through back propagation to approximate the manual scheduling results; use the stochastic gradient descent method and randomly select samples to update the derivatives to each neural network parameter; S2.4, pre-training and fine-tuning, the pre-training of machine learning models with feedforward rule learning is used to learn the universal scheduling rules of industrial enterprises. By comparing the scheduling data output by the model with the manual scheduling data, the difference between the two is fed back to adjust the neural network parameters, and the model output is adjusted to meet the manual scheduling specifications; the relationship between different data resources is solidified into the model weights; the focus of scheduling for different enterprises is adapted through model fine-tuning.

[0009] In step S3, the user behavior data collected during the production scheduling process is integrated with the process as the center to obtain the correlation between different data, including: Production resources are linked in series with the process in the form of factories, workshops, work centers, and processes; The means of production are connected in series with the processes in the form of projects, products, BOMs, process routes, and processes; Scheduling parameters are added as external conditions; The above data relationship associates each data with the process as the center and makes it into a nested JSON format, which is used as the input data of the machine learning model with feedforward rule learning. Production resources and means of production have a hierarchical relationship, which is expressed as a nested format of JSON data.

[0010] In step S3, a machine learning model with feedforward rule learning is used to vectorize different data with correlation relationships in the obtained user behavior data, including: According to the relationship between data and process, each level occupies a dimension, and the same dimension is indexed in the order of data as the vector value of the dimension; each production resource and means of production has a sequence, and the sequence is based on various types of data, starting from 1 as the index value, and the index value is the vector value; For each process, define the dimensions occupied by different data, use the specific data indexes in different dimensions as dimension values, and maintain the relationship between the data index value and the data ID.

[0011] In step S4, a machine learning model with feedforward rule learning for vectorization processing is pre-trained to obtain production scheduling data, including: S4.1, use the data processor to process the scene data into model input vectors, and restore the model output to scene usable data; loop the products to be scheduled, recursively traverse the product BOM, and find all processes according to the BOM node, material, process route, and process; S4.2, build the model, reduce the original Transformer model, and set the vector space dimension to 20; set 4 attention heads and 4 hidden layers, each hidden layer uses 4 layers of encoder stacking, and the output of each layer is the input of the next layer; S4.3, calculating the loss function, using the vector values ​​of the 14th dimension, the 15th dimension, and the 16th dimension to compare the model, using the deep neural network model of the Transformer algorithm to generate a plan, comparing the planned start time, planned end time, and station position set by the model with the actual planned data, and calculating the difference as the loss value; S4.4, training model, using existing production scheduling data as training data, processing uninput vectors and then starting training, first selecting the first process vector of the first product as the model input, and after obtaining the production scheduling data output by the model, using the loss function and the actual production scheduling to calculate the loss value; updating the model parameters based on the gradient descent function; when the loss value is lower than 5%, selecting the next process to append to the model input, repeating the loss calculation and back propagation and taking the next process until all products and all processes are trained; and repeating multiple training rounds; after the training is completed, the trained model data is saved through the Transformer framework, and matched with the data processor.

[0012] In step S4.1, all processes are found according to BOM nodes, materials, process routes, and processes, including: Search for related data by process, material, BOM level, product, and project path. Except for the process, each part of the data occupies the first vector dimension, the second vector dimension, the third vector dimension, and the fourth vector dimension respectively. After sorting, each part of the data is assigned an index starting from 1. The index value is the value in the vector dimension. Search for related data by process, work center, workshop, and factory path. Except for the process, each part of the data occupies the 5th vector dimension, the 6th vector dimension, and the 7th vector dimension respectively. The index after sorting the data is used as the specific value in each vector dimension. If the production scheduling parameters have a production rhythm, the daily allocation quantity is concatenated into integers and placed in the eighth dimension as a vector value. Otherwise, the dimension value is set to 0. If the production scheduling parameters have production shift settings, all on-the-job hours are concatenated into integers as vector values ​​and put into the 9th dimension, otherwise the dimension value is set to 0; If the production scheduling parameters are set with a production calendar, set a 7-digit number based on whether to work from Monday to Sunday, and put this number into the 10th dimension as a vector value. The default value is set to two days off. Put the process operation hours into the 11th dimension, the process route level number of the sequence into the 12th dimension, and the process sequence into the 13th dimension; The 14th dimension is used as the planned start time, the 15th dimension is used as the planned end time, and the 16th dimension is used as the station number; the last three dimension values ​​are all 0 by default; A 4-dimensional vector with a default dimension of 0 is reserved for subsequent expansion and fine-tuning; the vector is processed using the L2 normalization method.

[0013] In step S4.2, the original Transformer model is reduced, including: the output weight matrix is ​​Wo, the outputs of multiple attention heads are linearly transformed and fused into an output matrix, the value of the output weight matrix is ​​obtained through back propagation in the pre-training stage, and the dimension is aligned when the multi-head attention calculation results are fused with the help of the output weight matrix.

[0014] In step S5, the pre-trained machine learning model with feedforward rule learning is used to connect to the enterprise's existing system to obtain the data required for production scheduling, including: S5.1, model fine-tuning, collect the company's historical planning and scheduling data, adjust it to the json input format required by the tool, generate fine-tuning training data through the data processor, fine-tune the model, and save the model after the training is completed; S5.2, docking system, with products as batches, each batch starts from the first process and calls the model to produce the process plan, and saves the process plan to the model context; takes the next process and calls the model again to generate a new process plan, and saves the process plan to the model context, and processes all processes in sequence in a loop; then schedules the next product until all products are scheduled; appends the newly generated process plan to the end of the input and adds a new unscheduled process for the next round of generation.

[0015] Another object of the present invention is to provide an industrial automation production scheduling prediction system based on feedforward rule learning, which is implemented by the industrial automation production scheduling prediction method based on feedforward rule learning, and the system includes: Information collection module, used to collect production resources, production materials, and production scheduling parameter information in industrial automation scheduling scenarios; A machine learning model selection module with feedforward rule learning selects a machine learning model with feedforward rule learning according to the task type based on the collected information; The data processing module is used to integrate the user behavior data collected during the production scheduling process with the process as the center, obtain the correlation between different data, and use the machine learning model with feedforward rule learning to vectorize the different data with correlation in the obtained user behavior data, obtain the influencing factors of the planned production scheduling, and output the restored business data; wherein, the vectorization processing is to process the business data related to the production scheduling into vector data, and encode the data into vector values ​​by extracting the location, sequence and characteristics affecting the production scheduling; in the process of encoding the data into vectors, the vector values ​​corresponding to the data are recorded, and the vector output by the model is restored to the business data; The production scheduling data acquisition module is used to pre-train the machine learning model with feed-forward rule learning for vectorization processing for small data scale to obtain production scheduling data; The application module is used to utilize the pre-trained machine learning model with feedforward rule learning to connect with the company's existing system to obtain the data required for production scheduling.

[0016] Combining all the above-mentioned technical solutions, the beneficial effects of the present invention are as follows: the present invention uses industrial automation scheduling as a typical scenario to study the ultra-small data scale feedforward rule learning tool and training model based on user behavior. First, in the industrial automation scheduling scenario, the scenario characteristics and requirements are described in detail, and its application challenges under ultra-small data scale are analyzed. Then, a ultra-small data scale feedforward rule learning tool based on user behavior is proposed, which can effectively use user behavior data for rule learning and improve the generalization ability of rules. Finally, the tool is applied to industrial automation scheduling and optimized and adjusted according to actual conditions. In reality, there is a large amount of data available for training in multiple scenarios, and artificial intelligence is used to improve efficiency, increase automation level, and reduce labor costs.

[0017] The present invention will significantly reduce the threshold and cost of arranging production plans for enterprises, and no longer need to rely on experienced planners who have worked in the enterprise for many years, so the relevant personnel costs can be reduced. For enterprises that have never used production planning and scheduling, they can use this production method more easily through this invention. Planning production activities can significantly reduce the production costs caused by inventory occupancy, material waste, manpower waste, etc. caused by production disharmony. In the past, domestic and foreign production scheduling systems such as APS often followed a specific method to cooperate with manual adjustment to achieve, involving a large number of parameters and rules, requiring users to be familiar with a lot of professional knowledge and carefully adjust the parameters. The present invention innovatively uses machine learning technology to significantly reduce the technical threshold. Compared with the current popular large language model LLM, when using relevant machine learning technology in the present invention, the characteristics of industrial data are innovatively used to define the vectorized data available to the machine learning model, thereby realizing the application of AI technology without relying on traditional language models.

[0018] Industrial production scheduling software such as APS hopes to achieve the ultimate in two seemingly conflicting aspects: fixed rules and flexible adaptation. However, it is often difficult to achieve ideal results due to insufficient prediction of unknown scenarios. The present invention can easily be compatible with both fixed rules and specific scenario changes through the fine-tuning of pre-trained AI models with specific scenarios. The present invention overcomes the technical bias of the Transformer model, which is usually used in the field of natural language processing, and broadens the application field of deep learning technology by exploring the influencing factors of industrial data in planning and scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure; Figure 1 It is a flow chart of an industrial automation production scheduling prediction method based on feedforward rule learning provided by an embodiment of the present invention; Figure 2 It is a product BOM structure diagram provided by an embodiment of the present invention; Figure 3 It is a diagram of association between various types of data and processes provided by an embodiment of the present invention; Figure 4 is a schematic diagram of process input vectorization provided by an embodiment of the present invention; Figure 5 is a basic structural diagram of a reduced Transformer model provided by an embodiment of the present invention; Figure 6 It is a flow chart of the tool connection to the enterprise system application provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

[0021] The innovation of this invention is that it innovatively extracts the influencing factors of industrial data in production planning and construction of it into a vector space that can be used for deep learning, thereby migrating the Transformer technology mainly used in the field of natural language processing to the industrial field. By cleverly designing the input and output and training process of the model in the production scheduling business, it makes it possible to implement the industrial automation production scheduling prediction method based on feedforward rule learning.

[0022] This invention takes industrial automation production scheduling as a typical scenario, and proposes an ultra-small data scale feedforward rule learning tool and training model based on user behavior. First, the content of the industrial automation production scheduling scenario includes its scenario data, data characteristics and industrial needs. On this basis, the design method is outlined, including the selection of machine learning technology and model framework based on scenario characteristics. Then, it proposes how to collect and process user behavior data to realize an ultra-small data scale feedforward rule learning tool based on user behavior, which can effectively use user behavior data for rule learning and improve the generalization ability of rules. Finally, a training and application method of the model is proposed to achieve the application effect in the corresponding scenario.

[0023] This paper proposes the application potential of feedforward rule learning based on ultra-small data scale in real-world scenarios, and provides new ideas and methods for solving artificial intelligence application problems under conditions of data scarcity.

[0024] Embodiment 1, as Figure 1 As shown, the industrial automation production scheduling prediction method based on feedforward rule learning provided by the embodiment of the present invention includes: S1, collects production resources, production materials, and scheduling parameter information in industrial automation scheduling scenarios; S2, based on the collected information, selects a machine learning model with feedforward rule learning according to the task type; S3, with the process as the center, integrates the user behavior data collected during the production scheduling process, obtains the correlation between different data, and uses a machine learning model with feedforward rule learning to vectorize the different data with correlation in the obtained user behavior data, obtains the influencing factors of planned production scheduling, and outputs the restored business data; wherein, vectorization processing is to process the business data related to production scheduling into vector data, and encode the data into vector values ​​by extracting the location, sequence and characteristics affecting production scheduling; in the process of encoding data into vectors, the vector values ​​corresponding to the data are recorded, and the vector output by the model is restored to business data; S4, for small data scale, pre-train the machine learning model with feed-forward rule learning for vectorization processing to obtain production scheduling data; S5 uses the pre-trained machine learning model with feedforward rule learning to connect to the company's existing system to obtain the data required for production scheduling.

[0025] Example 2, as a specific implementation scheme of the present invention, the industrial automation production scheduling prediction method based on feedforward rule learning provided by the embodiment of the present invention is further described in detail below.

[0026] Scenario analysis: In the simplest way, production scheduling is the business behavior of arranging the process at what time and where (station) to perform production. It is necessary to plan and arrange the processes in the product material structure based on the existing production capacity and resource structure of the enterprise and the set production scheduling constraints. The following describes it from two aspects: scenario data and production scheduling behavior.

[0027] Scene data; Production resources: The production resources of industrial enterprises are usually arranged in a hierarchical structure of factory-workshop-work center-station.

[0028] Means of production: Regardless of whether the enterprise is order-oriented or inventory-oriented production, it can often be organized into projects. The following invention also takes projects as the means of production for planning. The project contains the products to be produced, and the products have corresponding bills of materials BOM. Considering the properties of the plan, the process bill of materials PBOM (Process BOM) is used here. The BOM contains the material composition of the product, and the materials correspond to the production process, and the object of the production plan - the process is in the BOM. It can be said that the BOM is the focus of production scheduling. The product BOM structure is as follows: Figure 2 shown.

[0029] Scheduling parameters: Scheduling parameters are used to configure scheduling behaviors. Some provide data such as production shifts and production calendars; some specify scheduling behaviors such as production sequence and scheduling priority; and some play a constraining role such as production tact, bottleneck processes, and station capacity. Scheduling needs to take different responses to different configuration parameters, such as using configured data and complying with constraints.

[0030] Scheduling results: As mentioned above, the scheduling results are the planned time and planned location of the process. Although different companies may have richer requirements for the presentation of the plan, most of them are based on these two basic elements. The present invention also uses these two elements as scheduling results.

[0031] Overview of the design approach.

[0032] From the above scenario analysis, we can see that what users want to get through production scheduling is when and where each process will perform the production task. This puts forward a requirement for the tool described in the present invention, that is, to select a suitable time and place for a given process. So this is a task that looks like "generating a plan" but is actually arranging time and place, and the time series data features are obvious. The present invention proposes to select a machine learning model with feedforward rule learning according to the task type.

[0033] Model technology selection: Traditional deep learning (such as RNN, LSTM, etc.) has certain advantages in processing time series data. But it has disadvantages: Difficulty handling long-term dependencies; Difficult to scale to high dimensions and complex inputs; The Transformer model was originally used in natural language processing, but its self-attention mechanism is very suitable for processing sequence data and can efficiently handle long-term dependencies and high-dimensional inputs. Advantages include: Self-attention mechanism: can better handle long-term dependencies, especially when the scheduling of each process depends on the scheduled processes.

[0034] Parallel processing: Compared with RNN, Transformer can be trained and inferred more efficiently.

[0035] Flexibility: The Transformer structure can be extended and adjusted to suit different tasks.

[0036] After comprehensive comparison, the Transformer model was finally selected as the basic model architecture.

[0037] Data format: Traditional Transformer models are mainly used for NLP tasks and deal with data in the natural language category. In addition to encoding the word itself, the relationship between word units is expressed through the word table vector space.

[0038] The present invention deals with production scheduling data, and the production scheduling unit is the process plan, so the present invention needs to use a method to represent the process plan. Here we face the uncertainty of the coding space - compared with the fixed coding space of natural language (such as utf-8 encoding), the process data, station data, production scheduling time range, etc. of different companies are all uncertain. Moreover, it is not just the data range that is different, but also what data the present invention ultimately wants to process needs to be clarified. Data such as project name, factory address, workshop location, etc. usually do not affect the production scheduling process.

[0039] This requires a simplified abstraction of production scheduling, which is expressed as a simple description of "what to do first and what to do later". Therefore, the sequence and hierarchical relationship are the data that the present invention has to deal with.

[0040] From the perspective of scenario analysis, both production resources and means of production are structured data with hierarchical relationships, and the same levels often have sequential relationships (including some production scheduling parameters that also specify the data order). Therefore, the present invention can use this hierarchy and order as the encoding space, which has been described in detail in the data structure.

[0041] Training and Application: Once the data structure is clear, the training objectives and application effects of the machine learning model with feedforward rule learning are basically clear.

[0042] The training of machine learning models with feedforward rule learning is based on existing user scheduling behavior data (historical scheduling plans) to let the model learn manual scheduling methods. Specifically, it inputs production resources, production materials, and scheduling parameters, and lets the Transformer model pay attention to calculate the resource relationship between processes. It outputs the process scheduling plan through feedforward rules, compares it with the manual scheduling results, and adjusts the model parameters through back propagation, so as to approach the manual scheduling results.

[0043] When applied, the same input sequence is used to produce the process scheduling plan.

[0044] Pre-training and fine-tuning: To address the problem of too little available training data in real-world scenarios, we use a method that combines pre-training of machine learning models with feedforward rule learning with fine-tuning in actual scenarios to optimize tool performance.

[0045] The pre-training of machine learning models with feedforward rule learning is used to learn the universal production scheduling rules of industrial enterprises, familiarize themselves with the relationship between different data resources, and solidify them into model weights. The model is then fine-tuned to adapt to the focus of production scheduling of different enterprises. This achieves the goal of optimizing tool adaptability with a small amount of data.

[0046] Data format: The design of data format mainly revolves around the model, that is, the data can be formatted into a way that the model can process, and the model output can be restored to business data. From the scenario analysis, it can be seen that production planning and scheduling mainly arrange specific plans around the process, so the processing of data format is also mainly based on the process.

[0047] Data relationship: If the process is the center and the data used in the entire production scheduling process is integrated, it is necessary to analyze the relationship between different data. From the perspective of scenario analysis, the data is mainly divided into production resources, production materials and production scheduling parameters. Among them, production resources are usually connected to the process in the form of factory-workshop-work center-process; production materials are connected to the process in the form of project-product-BOM-process route-process. As for the production scheduling parameters, they are attached as external conditions.

[0048] Therefore, the present invention associates various data with the process as the center and makes them into a nested json format. This json is also the input data of this tool. Figure 3 shown.

[0049] Vectorization: The data processed by machine learning models with feedforward rule learning are often multi-dimensional tensor data. Taking the Transformer model of NLP tasks as an example, its input is the vector of the word element token in the word table Embedding space, and the correspondence between the word vector and the original text is maintained. Therefore, the present invention should also process the process and its associated data into vector input, and maintain the association between the vector and the original data in order to restore the model output.

[0050] As mentioned in the design method overview section, production scheduling data does not have clear definitions and boundaries like natural language. The production resources and specific scheduling parameters of different companies may be different, so the hierarchy and sequence are mainly used as the encoding space here. Even if the specific settings of data of different companies (such as the number of workshops, process numbers, etc.) are different, this hierarchical relationship and sequence requirements are usually available, and the hierarchical relationship and data sequence itself are important factors affecting planning and scheduling.

[0051] Therefore, based on the relationship between data and process, each level occupies a dimension, and the same dimension is indexed in data order as the vector value of the dimension. For a certain process, the dimensions occupied by different data are directly defined, and the specific data indexes in different dimensions are used as dimension values. And the relationship between data index value and data (ID) is maintained. This relationship is a simple one-to-one relationship between data type (what data) and data ID and index value, which is to facilitate the restoration of model output.

[0052] Pre-training: In order for the tool to perform well even with very small data, the machine learning model with feedforward rule learning needs to learn some prior knowledge, that is, basic scheduling rules. Find some historical scheduling data that already exists in the industry, or use the existing scheduling system to build some simulation data, and perform scheduling to generate scheduling data. These data are used to pre-train the model. The following uses a scheduling result as an example to explain how to pre-train the model.

[0053] Data processor: The data processor is similar to the tokenizer of the traditional NLP task Transformer model, which processes the scene data into model input vectors and restores the model output to scene-usable data. However, the data relationship in industrial manufacturing is usually clear, so the traditional encoding form is used here.

[0054] Loop through the products to be scheduled, recursively traverse the product BOM, find all processes according to the BOM node, material, process route, and process, and do the following for each process: Search for related data along the path of "process-material-BOM level-product-project". Except for the process, each part of the data occupies the 1st vector dimension, the 2nd vector dimension, the 3rd vector dimension, and the 4th vector dimension respectively. After sorting the data (by default, sorting is done by data creation time. If there are corresponding production scheduling parameters such as product production scheduling priority and project priority, sorting is done by production scheduling parameter settings), the index is assigned starting from 1. The index value is the value in the vector dimension.

[0055] Related data is searched according to the path of "process, work center, workshop, factory". Except for the process, each part of the data occupies the 5th vector dimension, the 6th vector dimension, and the 7th vector dimension respectively. Similarly, the index after sorting the data is used as the specific value in each vector dimension.

[0056] For example, the associated data of "process, work center, workshop, factory" is usually industrial basic data, which has a simple hierarchical relationship, such as which work center a process is completed in, multiple work centers constitute a workshop, and a factory is divided into multiple workshops, etc.; If the production scheduling parameters have a production rhythm, the daily allocation quantity is concatenated into integers and placed in the 8th dimension as a vector value; otherwise, the dimension value is set to 0.

[0057] If the production scheduling parameters have production shift settings, all on-the-job hours are concatenated into integers as vector values ​​and put into the 9th dimension; otherwise, the dimension value is set to 0.

[0058] If the production scheduling parameters are set with a production calendar, then set a 7-digit number based on whether the employees are working from Monday to Sunday. For example, if the employees have two days off on weekends, the number is 1111100. Put this number into the 10th dimension as a vector value, and set it to the value of two days off on weekends by default.

[0059] Put the process operation hours into the 11th dimension; put the process route level number (the main process is 0, and the branch process takes the level number) into the 12th dimension; put the process sequence into the 13th dimension; Finally, the 14th dimension is used as the planned start time (second-level timestamp), the 15th dimension is used as the planned end time, and the 16th dimension is used as the station number. The last three dimension values ​​are all set to 0 by default, which is similar to the masked language modeling MLM processing method used in NLP task training.

[0060] A 4-dimensional vector with a default dimension of 0 is reserved for subsequent expansion and fine-tuning. Finally, in order to avoid the deviation of the model focus caused by the difference in the magnitude of data in different dimensions, the vector is processed using the L2 normalization method so that all data are in the same magnitude.

[0061] like Figure 4 As shown in the vectorization of process input, the input sequence of a process is a 20-dimensional vector.

[0062] The products are grouped together, and each product is a series of process input vectors. The following is an example of the JSON data. The entire JSON data is an array, each of which contains product information and a process list. The process list is an array of vectorized values: "productId": 1, "operationList": [2, 6, 3, 2, 2, 9, 12, 222, 8910111314151617, 1111100, 4.5, 1, 3, 0,0, 0, 0, 0, 0, 0], [ 3, 5, 6, 2, 3, 7, 11, 212, 7802121311621618000, 1100111, 5, 1, 4,0, 0, 0, 0, 0, 0, 0] "productId": 2, "operationList": [... ] Model building: The original Transformer model is used here. However, the complexity of the production scheduling scenario is much smaller than that of the NLP task, so the model size is appropriately reduced.

[0063] The vector space dimension is set to 20; considering that the amount of production scheduling data may be large, 4 attention heads are set to improve computing performance. 4 hidden layers are set, and each hidden layer uses 4 layers of encoder stacking. The output of each layer is the input of the next layer, which is a typical feedforward neural network. The structure is as follows Figure 5 The basic structure of the reduced Transformer model is shown below.

[0064] Among them, the output weight matrix Wo, the function of this matrix is ​​to perform linear transformation on the outputs of multiple attention heads so as to fuse them into an output matrix. The value of the output weight matrix is ​​also obtained through back propagation in the pre-training stage. Because the production schedule time range and the number of available stations are not certain each time, the present invention uses 24 hours a day for a year (the first hour is counted as 1, and then accumulated in sequence) and a maximum of 100 stations as the boundary, and each hour and each station are mixed in pairs to generate an output weight matrix of size 100×8760. When the results of multi-head attention calculations are fused with the help of the output weight matrix, dimension alignment is performed. The output matrix of multi-head attention calculations is often only a part of the input matrix (parallel attention calculations are performed in slices), so it needs to be merged into an output matrix of the same size as the input matrix through the weight matrix, which is the so-called dimension alignment in the text.

[0065] Loss function: The vector values ​​of the 14th, 15th and 16th dimensions set in step S4.1 are used to compare the model generation plan, that is, the planned start time, planned end time and station position set by the model are compared with the actual planned data to calculate the difference between them as the loss value.

[0066] Assume that the present invention has a process input sequence , the corresponding actual production schedule (target output) is , at each time step , the model outputs a predicted value , which means that given the input In the case of The production schedule forecast for each process is as follows: ; In the formula, For the The loss value of tokens, is the number of categories, is the real label The indicator function of the class, if the sample belongs to If the class is 1, otherwise it is 0; The model predicts The token belongs to The predicted probability of the class.

[0067] Training model: After the above steps are prepared, the model is pre-trained, including: the first step is to use the existing production scheduling data as training data to process the uninput vectors according to the method described in step S3. The second step is to start training. First, select the first process vector of the first product as the model input. After obtaining the production scheduling data output by the model, use the loss function described in step S4.3 and the actual production scheduling to calculate the loss value. Update the model parameters according to the gradient descent function; when the loss value is lower than 5%, select the next process to append to the model input, repeat the loss calculation and back propagation, and take the next process until all products and all processes have been trained. Use appropriate training rounds to observe the convergence effect of the model, and use the best effect as the end standard.

[0068] After the training is completed, the trained model data is saved with the help of the Transformer framework, and the data processor is used as the main component of this tool.

[0069] Tool application: There are two main parts to using this tool: First, fine-tune the model to suit the specific situation of the enterprise. This step requires the enterprise to have its own historical production scheduling data. This step can be omitted in certain circumstances (such as new enterprises or enterprises that clearly indicate that they do not need it); second, use the tool to connect to the system for automated production scheduling; Model fine-tuning: Collect the company's historical production planning data, adjust it to the JSON input format required by the tool, generate fine-tuning training data through the data processor, and then fine-tune the model. Save the model after training.

[0070] The fine-tuning process is similar to the pre-training process. After preparing the training data, load the pre-trained model, set appropriate hyperparameters and start training. When satisfactory training results are achieved, save the model again.

[0071] Docking system: connect to the company's existing systems (MES, ERP, APS, etc.) through secondary development, obtain the data required for production scheduling, accept production scheduling requests, and call the data processor to process the data.

[0072] Taking products as batches, each batch starts with the first process and calls the model to produce the process plan, and saves the process plan to the model context. Then take the next process and call the model again to generate a new process plan, and save the process plan to the model context, and process all processes in a loop. Then, schedule the next product until all products are scheduled.

[0073] Assume that the present invention has an input process sequence ,in, It is The goal of the model is to generate the next process plan. The formula is: ; In the formula, For a given input sequence In the case of The probability of The hidden state calculated by the Transformer model can be obtained by encoding the input sequence at that time To obtain; is the weight matrix, is the bias term.

[0074] Append the newly generated process plan to the end of the input and add new unscheduled processes for the next round of generation.

[0075] Arrange the production scheduling data and transfer it back to the enterprise system according to the enterprise system requirements. The tool docking enterprise system application process is as follows: Figure 6 shown.

[0076] Embodiment 3, an industrial automation production scheduling prediction system based on feedforward rule learning, comprising: Information collection module, used to collect production resources, production materials, and production scheduling parameter information in industrial automation scheduling scenarios; A machine learning model selection module with feedforward rule learning, used to select a machine learning model with feedforward rule learning according to the task type based on the above-collected information; The data processing module is used to integrate the user behavior data collected during the entire production scheduling process with the process as the center, obtain the correlation between different data, use the machine learning model with feedforward rule learning to vectorize the different data with correlation in the obtained user behavior data, obtain the influencing factors of the production scheduling, and output the restored business data; A production scheduling data acquisition module is used to pre-train the machine learning model with feedforward rule learning that performs vectorization processing in step S3 for ultra-small data scale to obtain production scheduling data; The application module is used to use the pre-trained machine learning model with feedforward rule learning to connect to the company's existing system to obtain the data required for production scheduling.

[0077] This paper proposes a user behavior-based ultra-small data-scale feedforward rule learning tool and applies it to industrial automation production scheduling. Experimental results show that the tool can effectively improve production scheduling efficiency and accuracy.

[0078] This paper proposes a user behavior-based ultra-small data-scale feedforward rule learning tool and training model, and applies it to industrial automation production scheduling. Experimental results show that the tool can effectively improve production scheduling efficiency and accuracy.

[0079] This tool allows the model to acquire prior knowledge of industrial planning and scheduling through model pre-training, avoiding the problem of insufficient training data in actual scenarios causing the model to be unable to complete training.

[0080] This tool uses a feedforward rule learning method to allow the model to learn data features layer by layer, and cooperates with model fine-tuning to achieve the effect of fully training the model.

[0081] Although the Transformer model used in this model is generally good at natural language processing, the Transformer model can also process this type of non-natural language data based on sufficient analysis and structured processing of the production scheduling data.

[0082] The present invention proposes a user behavior-based ultra-small data-scale feedforward rule learning tool, which can effectively utilize user behavior data for rule learning and improve the generalization ability of rules.

[0083] The tool is applied to industrial automation production scheduling. Experimental results show that the tool can effectively improve the efficiency and accuracy of production scheduling.

[0084] The present invention enriches the theory of ultra-small data scale machine learning and provides new ideas and methods for solving machine learning problems under ultra-small data scale.

[0085] The present invention expands the application field of feedforward rule learning and provides a new solution for scenarios such as industrial automation production scheduling.

[0086] The present invention can effectively improve the efficiency and accuracy of industrial automated production scheduling and reduce production costs.

[0087] The present invention can promote the application of artificial intelligence technology in the industrial field and promote the development of industrial intelligence.

[0088] Furthermore, the present invention improves the robustness and scalability of the ultra-small data scale machine learning method based on user behavior.

[0089] Apply this tool to other scenarios such as recommendation systems, anomaly detection, etc.

[0090] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.

Claims

1. An industrial automation production scheduling prediction method based on feedforward rule learning, characterized in that: The method includes: S1, collects production resources, production materials, and scheduling parameter information in industrial automation scheduling scenarios; S2, based on the collected information, selects a machine learning model with feedforward rule learning according to the task type; S3, with the process as the center, integrates the user behavior data collected during the production scheduling process, obtains the correlation between different data, and uses a machine learning model with feedforward rule learning to vectorize the different data with correlation in the obtained user behavior data, obtains the influencing factors of planned production scheduling, and outputs the restored business data; wherein, vectorization processing is to process the business data related to production scheduling into vector data, and encode the data into vector values ​​by extracting the location, sequence and characteristics affecting production scheduling; in the process of encoding data into vectors, the vector values ​​corresponding to the data are recorded, and the vector output by the model is restored to business data; S4, for small data scale, pre-train the machine learning model with feed-forward rule learning for vectorization processing to obtain production scheduling data; S5 uses the pre-trained machine learning model with feedforward rule learning to connect to the company's existing system to obtain the data required for production scheduling.

2. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 1 is characterized in that: In step S1, the production resources are hierarchical structures of factories, workshops, work centers, and stations; The production materials use the process bill of materials PBOM, which contains the material composition of the product and the corresponding production process of the materials; Scheduling parameters are used to configure scheduling behaviors, including production shifts, production calendar, production sequence, scheduling priority, production rhythm, bottleneck process, and station capacity. In the industrial automation production scheduling scenario, the production scheduling results are also collected. The production scheduling results include: the planned time and planned location of the process.

3. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 1 is characterized in that: In step S2, a machine learning model with feedforward rule learning is selected according to the task type, including: S2.1, model technology selection, select the Transformer model as the basic model architecture selection; S2.2, data format processing, using the hierarchical relationship between production resources and production materials and the data sequence specified by the production scheduling parameters as the coding space to obtain structured data; in conventional production and manufacturing scenarios, the workshops, work centers, and stations in the production resources have a hierarchical relationship with the products, materials, and processes in the production materials. During the production scheduling process, the production priority is adjusted to achieve product priority production and optimize the process route; the hierarchical relationship is expressed as a hierarchical sequence number, the priority is expressed as a sequence number, the hierarchical relationship is expressed as a nested relationship in the structured data, and the sequence number is a specific value in the structured data; S2.3, training and application, input production resources, production materials, and production scheduling parameters, calculate the resource relationship between processes through the attention of the Transformer model, and calculate the relationship between process plans through the Transformer attention operation algorithm; output process scheduling plans through feedforward rules, compare them with the manual scheduling results, find the plan of the same process in the corresponding manual scheduling results for the process plan output by the model, compare the time difference and the position difference between the two, and obtain the mean square error as the comparison result; adjust the model parameters through back propagation to approximate the manual scheduling results; use the stochastic gradient descent method and randomly select samples to update the derivatives to each neural network parameter; S2.4, pre-training and fine-tuning, the pre-training of machine learning models with feedforward rule learning is used to learn the universal scheduling rules of industrial enterprises. By comparing the scheduling data output by the model with the manual scheduling data, the difference between the two is fed back to adjust the neural network parameters, and the model output is adjusted to meet the manual scheduling specifications; the relationship between different data resources is solidified into the model weights; the focus of scheduling for different enterprises is adapted through model fine-tuning.

4. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 1 is characterized in that: In step S3, the user behavior data collected during the production scheduling process is integrated with the process as the center to obtain the correlation between different data, including: Production resources are linked in series with the process in the form of factories, workshops, work centers, and processes; The means of production are connected in series with the processes in the form of projects, products, BOMs, process routes, and processes; Scheduling parameters are added as external conditions; The above data relationship associates each data with the process as the center and makes it into a nested JSON format, which is used as the input data of the machine learning model with feedforward rule learning. Production resources and means of production have a hierarchical relationship, which is expressed as a nested format of JSON data.

5. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 1 is characterized in that: In step S3, a machine learning model with feedforward rule learning is used to vectorize different data with correlation relationships in the obtained user behavior data, including: According to the relationship between data and process, each level occupies a dimension, and the same dimension is indexed in the order of data as the vector value of the dimension; each production resource and means of production has a sequence, and the sequence is based on various types of data, starting from 1 as the index value, and the index value is the vector value; For each process, define the dimensions occupied by different data, use the specific data indexes in different dimensions as dimension values, and maintain the relationship between the data index value and the data ID.

6. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 1 is characterized in that: In step S4, a machine learning model with feedforward rule learning for vectorization processing is pre-trained to obtain production scheduling data, including: S4.1, use the data processor to process the scene data into model input vectors, and restore the model output to scene usable data; loop the products to be scheduled, recursively traverse the product BOM, and find all processes according to the BOM node, material, process route, and process; S4.2, build the model, reduce the original Transformer model, and set the vector space dimension to 20; set 4 attention heads and 4 hidden layers, each hidden layer uses 4 layers of encoder stacking, and the output of each layer is the input of the next layer; S4.3, calculating the loss function, using the vector values ​​of the 14th dimension, the 15th dimension, and the 16th dimension to compare the model, using the deep neural network model of the Transformer algorithm to generate a plan, comparing the planned start time, planned end time, and station position set by the model with the actual planned data, and calculating the difference as the loss value; S4.4, training model, using existing production scheduling data as training data, processing uninput vectors and then starting training, first selecting the first process vector of the first product as the model input, and after obtaining the production scheduling data output by the model, using the loss function and the actual production scheduling to calculate the loss value; updating the model parameters based on the gradient descent function; when the loss value is lower than 5%, selecting the next process to append to the model input, repeating the loss calculation and back propagation and taking the next process until all products and all processes are trained; and repeating multiple training rounds; after the training is completed, the trained model data is saved through the Transformer framework, and matched with the data processor.

7. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 6 is characterized in that: In step S4.1, all processes are found according to BOM nodes, materials, process routes, and processes, including: Search for related data by process, material, BOM level, product, and project path. Except for the process, each part of the data occupies the first vector dimension, the second vector dimension, the third vector dimension, and the fourth vector dimension respectively. After sorting, each part of the data is assigned an index starting from 1. The index value is the value in the vector dimension. Search for related data by process, work center, workshop, and factory path. Except for the process, each part of the data occupies the 5th vector dimension, the 6th vector dimension, and the 7th vector dimension respectively. The index after sorting the data is used as the specific value in each vector dimension. If the production scheduling parameters have a production rhythm, the daily allocation quantity is concatenated into integers and placed in the eighth dimension as a vector value. Otherwise, the dimension value is set to 0. If the production scheduling parameters have production shift settings, all on-the-job hours are concatenated into integers as vector values ​​and put into the 9th dimension, otherwise the dimension value is set to 0; If the production scheduling parameters are set with a production calendar, set a 7-digit number based on whether to work from Monday to Sunday, and put this number into the 10th dimension as a vector value. The default value is set to two days off. Put the process operation hours into the 11th dimension, the process route level number of the sequence into the 12th dimension, and the process sequence into the 13th dimension; The 14th dimension is used as the planned start time, the 15th dimension is used as the planned end time, and the 16th dimension is used as the station number; the last three dimension values ​​are all 0 by default; A 4-dimensional vector with a default dimension of 0 is reserved for subsequent expansion and fine-tuning; the vector is processed using the L2 normalization method.

8. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 6 is characterized in that: In step S4.2, the original Transformer model is reduced, including: the output weight matrix is ​​Wo, the outputs of multiple attention heads are linearly transformed and fused into an output matrix, the value of the output weight matrix is ​​obtained through back propagation in the pre-training stage, and the dimension is aligned when the multi-head attention calculation results are fused with the help of the output weight matrix.

9. The industrial automation production scheduling prediction method based on feedforward rule learning according to claim 1 is characterized in that: In step S5, the pre-trained machine learning model with feedforward rule learning is used to connect to the enterprise's existing system to obtain the data required for production scheduling, including: S5.1, model fine-tuning, collect the company's historical planning and scheduling data, adjust it to the json input format required by the tool, generate fine-tuning training data through the data processor, fine-tune the model, and save the model after the training is completed; S5.2, docking system, with products as batches, each batch starts from the first process and calls the model to produce the process plan, and saves the process plan to the model context; takes the next process and calls the model again to generate a new process plan, and saves the process plan to the model context, and processes all processes in sequence in a loop; then schedules the next product until all products are scheduled; appends the newly generated process plan to the end of the input and adds a new unscheduled process for the next round of generation.

10. An industrial automation production scheduling prediction system based on feedforward rule learning, characterized in that: The system is implemented by the industrial automation production scheduling prediction method based on feedforward rule learning as described in any one of claims 1 to 9, and the system includes: Information collection module, used to collect production resources, production materials, and production scheduling parameter information in industrial automation scheduling scenarios; A machine learning model selection module with feedforward rule learning selects a machine learning model with feedforward rule learning according to the task type based on the collected information; The data processing module is used to integrate the user behavior data collected during the production scheduling process with the process as the center, obtain the correlation between different data, and use the machine learning model with feedforward rule learning to vectorize the different data with correlation in the obtained user behavior data, obtain the influencing factors of the planned production scheduling, and output the restored business data; wherein, the vectorization processing is to process the business data related to the production scheduling into vector data, and encode the data into vector values ​​by extracting the location, sequence and characteristics affecting the production scheduling; in the process of encoding the data into vectors, the vector values ​​corresponding to the data are recorded, and the vector output by the model is restored to the business data; The production scheduling data acquisition module is used to pre-train the machine learning model with feed-forward rule learning for vectorization processing for small data scale to obtain production scheduling data; The application module is used to utilize the pre-trained machine learning model with feedforward rule learning to connect with the company's existing system to obtain the data required for production scheduling.

Citation Information

Patent Citations

  • Industrial glass production scheduling method, system and device based on deep learning time sequence model

    CN117391369A

  • Chip high multilayer ceramic packaging substrate production line multi-constraint automatic scheduling method

    CN117391423A

  • Time sequence prediction method and device based on multi-scale patch parallel prediction network

    CN118735059A

  • Pre-training language model fine tuning method based on lightweight feedforward network adapter

    CN118885558A

  • Machine learning based imaging method of determining authenticity of a consumer good

    GB202018278D0