A machine learning function block execution mechanism and method in a programmable controller

Abstract

The present invention provides a machine learning function block execution mechanism in a programmable controller (PLC), comprising a machine learning function block group, a machine learning extension module, and a machine learning executable module. The machine learning function block group is configured to call interface functions exported by the machine learning extension module; the machine learning extension module is configured to load and execute the machine learning executable module via the exported interface functions; and the machine learning executable module is configured to execute a dataset loading function, a model building function, a model network layer function, a model loading function, a model saving function, a model training function, a model prediction function, and a model testing function. This mechanism extends the runtime of traditional PLCs, enabling them to build and run machine learning models, providing a solid foundation for the intelligent control of industrial equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of programmable logic controllers, and in particular to a machine learning function block execution mechanism and method in a programmable controller. Background Art

[0002] PLCs are widely used in industrial equipment control. With the advent of Industry 4.0 and the booming development of smart manufacturing, PLCs need to possess intelligent control capabilities to adapt to complex industrial scenarios. Machine learning technology has matured over the past few decades and is widely used in image recognition, speech recognition, and natural language processing. Lightweight machine learning frameworks have also emerged for embedded devices, providing a solid foundation for the intelligent development of PLCs. Traditional PLC programming primarily focuses on process control of industrial equipment and cannot support complex intelligent control algorithms, nor does it support the building, training, and prediction of machine learning models.

[0003] To this end, the present invention proposes a machine learning function block execution mechanism in a programmable controller, which expands the existing PLC runtime to enable the PLC to have intelligent control capabilities. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a machine learning function block execution mechanism and method in a programmable controller, which can expand the existing PLC runtime, thereby enabling the PLC to have intelligent control capabilities.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A machine learning function block execution mechanism in a programmable controller includes a machine learning function block group, a machine learning extension module, and a machine learning executable module. The machine learning function block group is used to call an interface function exported by the machine learning extension module; the machine learning extension module is used to load and execute the machine learning executable module through the exported interface function; and the machine learning executable module is used to execute a data set loading function, a model building function, a model network layer function, a model loading function, a model saving function, a model training function, a model prediction function, and a model testing function.

[0007] Preferably, the machine learning function block group includes a machine learning extension module initialization function block, a machine learning executable module loading function block, a data set loading function block, a model building function block, a model network layer function block, a model loading function block, a model saving function block, a model training function block, a model testing function block, and a model prediction function block.

[0008] Preferably, the definition and implementation of the machine learning function block group complies with the IEC61131-3 standard, and is instantiated in the five programming languages ​​defined in the IEC61131-3 standard, and the written programs can be scheduled and executed by the PLC during runtime.

[0009] Preferably, the machine learning extension module includes an extension module initialization module, a machine learning executable module loading module, a machine learning execution module and a machine learning basic library.

[0010] Preferably, the interface functions exported by the machine learning extension module include an extension module initialization interface function, an executable module loading interface function, a data set loading interface function, a model building interface function, a model network layer interface function, a model loading interface function, a model saving interface function, a model training interface function, a model prediction interface function, and a model testing interface function.

[0011] Preferably, the model network layer functional block includes an input layer functional block, a convolutional layer functional block, a fully connected layer functional block, a pooling layer functional block, and a Softmax layer functional block which are connected in sequence.

[0012] The present invention also provides a method for executing a machine learning function block in a programmable controller, comprising the following steps:

[0013] S200, scheduling and executing machine learning function block group instances during PLC runtime;

[0014] S201, the machine learning function block group instance calls the machine learning extension module interface function;

[0015] S202. The machine learning extension module loads and executes the machine learning executable module.

[0016] Preferably, in step S201, the method of calling the interface function is:

[0017] The machine learning extension module initialization function block calls the extension module initialization interface function exported by the machine learning extension module; the executable module loading function block will call the executable module loading interface function with the value of the ModulePath pin as a parameter; the dataset loading function block will call the dataset loading function with the value of the DataPath pin as a parameter; the model building function block will call the model building function; the model network layer function block will call the model network layer interface function to calculate the input tensor and return the output tensor; the model loading function block will call the model loading function according to ModelPath; the model saving function block will call the model saving function according to SavePath; the model training function block needs to input parameters such as the training set object, number of training cycles, and learning rate; the model testing function block needs to input the test set object; the model prediction function block needs to input the image data to be predicted and the length, width, and number of channels of the image, and the prediction result will be returned after the prediction is completed.

[0018] Preferably, in step S202, the method for loading and executing is:

[0019] The extension module initialization interface function calls the extension module initialization module to initialize the extension module. The initialization process includes loading the machine learning basic library into the memory and initializing the extension module global variables; the machine learning executable module loading function calls the executable module loading module to load the machine learning executable module into the memory, allocates space for the functions and variables defined by the executable module, and initializes the stack space for the executable module to run; the dataset loading function, model building function, model network layer function, model loading function, model saving function, model training function, model testing function and model prediction function call the corresponding interface function of the machine learning executable module through the machine learning execution module.

[0020] The present invention has the following characteristics and beneficial effects:

[0021] This technical solution enables existing PLCs to perform machine learning tasks without the need for additional machine learning hardware modules. By extending the existing PLC runtime, machine models can be built and run on existing PLCs, rapidly enabling intelligent control capabilities while saving additional hardware costs and providing a solid foundation for intelligent control of industrial equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 This is a flow chart of the method provided in this embodiment.

[0024] Figure 2 The method module provided in this embodiment is composed of.

[0025] Figure 3 This is an architecture diagram of the machine learning functional block group provided in this embodiment.

[0026] Figure 4 This is a diagram of the machine learning extension module architecture provided for this embodiment.

[0027] Figure 5 This is a diagram of the machine learning executable module architecture provided for this embodiment.

[0028] Figure 6 This is a diagram of the defect classification network model architecture provided in this embodiment.

[0029] Figure 7 This is the training and testing results of the defect classification network model provided in this embodiment. DETAILED DESCRIPTION

[0030] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0031] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0032] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0033] The present invention provides a machine learning function block execution mechanism in a programmable controller, such as Figure 1 As shown, it includes a machine learning function block group 102, a machine learning extension module 103, and a machine learning executable module 104. The machine learning function block group 102 is used to call the interface function exported by the machine learning extension module 103; the machine learning extension module 103 is used to load and execute the machine learning executable module 104 through the exported interface function; the machine learning executable module 104, such as Figure 5 As shown, it is used to execute and implement the dataset loading function, model building function, model network layer function, model loading function, model saving function, model training function, model prediction function, and model testing function.

[0034] Further, such as Figure 3 As shown, the machine learning function block group 102 includes a machine learning extension module initialization function block, a machine learning executable module loading function block, a data set loading function block, a model building function block, a model network layer function block, a model loading function block, a model saving function block, a model training function block, a model testing function block, and a model prediction function block.

[0035] The definition and implementation of the machine learning function block group 102 comply with the IEC61131-3 standard, and are instantiated in the five programming languages ​​defined in the IEC61131-3 standard. The written programs can be scheduled and executed by the PLC during runtime.

[0036] Specifically, the five programming languages ​​are ladder diagram (LD), instruction list (IL), function block diagram (FBD), sequential function flow chart (SFC), and structured text (ST).

[0037] Further, such as Figure 4 As shown, the machine learning extension module includes an extension module initialization module, a machine learning executable module loading module, a machine learning execution module and a machine learning basic library.

[0038] Among them, the interface functions exported by the machine learning extension module 103 include extension module initialization interface function, executable module loading interface function, data set loading interface function, model construction interface function, model network layer interface function, model loading interface function, model saving interface function, model training interface function, model prediction interface function, and model testing interface function.

[0039] Specifically, the model network layer functional block includes an input layer functional block, a convolutional layer functional block, a fully connected layer functional block, a pooling layer functional block, and a Softmax layer functional block, which are connected in sequence.

[0040] The present invention also provides a method for executing a machine learning function block in a programmable controller, comprising the following steps:

[0041] S200, PLC runtime scheduling execution machine learning function block group instance

[0042] Specifically, in actual applications, PLC programs written in the PLC programming language are uniformly scheduled and executed by the PLC runtime. In order to run the machine learning code on the PLC, it is necessary to first instantiate the machine learning function block group using any of the five programming languages ​​specified in IEC61131-3, and use this group of function block instances to build a machine learning model, and implement functions such as model loading, model saving, model training, and model testing. Then, this code is downloaded to the PLC so that it can be executed by the PLC runtime. The machine learning function block group defined in this embodiment includes the following function blocks: machine learning extension module initialization function block, machine learning executable module loading function block, data set loading function block, model building function block, model network layer function block, model loading function block, model saving function block, model training function block, model testing function block, and model prediction function block. Specifically, the machine learning extension module initialization function block is responsible for initializing the machine learning extension module, including loading the machine learning basic library into the memory and initializing the global variables of the machine learning extension module; the machine learning executable module loading function block loads the machine learning executable module into the memory, allocates space for the variables defined by the executable module, and initializes the stack space for the executable module to run; the dataset loading function block will load the dataset required for model training and prediction; the model network layer function block can be used to build a model structure; the model building function block will build a model object based on the built model structure; the model loading function block will open the model file from the specified location and load the model object; the model saving function block will save the model object to the specified location; the model training function block will train the model according to the training set and training configuration; the model testing function block will test the performance of the model on the test set according to the test set; the model prediction function block will predict the input data through the model object. Figure 3As shown, the model network layer function block includes the following function blocks: input layer function block, convolution layer function block, fully connected layer function block, pooling layer function block, and Softmax layer function block. Specifically, the input layer function block will output a tensor object of a specified shape, which will serve as the input tensor of the convolution layer function block, the fully connected layer function block, the pooling layer function block, and the Softmax layer function block. The convolution layer function block will perform a convolution operation on the input tensor object, the fully connected layer function block will perform a fully connected operation on the input tensor object, the pooling layer function block will perform a pooling operation on the input object, and the Softmax layer function block will perform a Softmax operation on the input tensor object. Users can also customize new model network layer function blocks. The specific implementation method is as follows: the user instantiates several model network layer function blocks provided by this method, builds a customized network layer according to their own needs, and finally uses the PLC platform software to encapsulate the programming page of the network layer to obtain a new model network layer function block.

[0043] It's important to note that the model building block is also known as the model instantiation block. As you can see, to build a model, you need to tell the model building block what the model's structure is. For example, you need to specify the layers and how they connect to each other, so that it can return a model object. Therefore, you first need to instantiate a series of model network layer blocks and then connect them in sequence to generate the model structure. Finally, executing the model building block returns a model with the aforementioned structure. Therefore, it's understandable that the model network layer block is essential for building a model.

[0044] This embodiment uses a ladder diagram to construct a neural network model, such as Figure 6 As shown in the figure, first, the input layer function block of the neural network is instantiated. The output of this function block will be used as the input of the subsequent network layer function block; then, 17 convolution layer function blocks, 2 pooling layer function blocks, and 1 Softmax layer function block are instantiated respectively, and the function blocks are Figure 6 The blocks are arranged in this order, with the tensor output pin variables of each block acting as the tensor input pin variables of the subsequent block. Finally, a model building block is instantiated, and the tensor output pin variables of the input layer block and the softmax layer block serve as the tensor input pin variables of this block. Executing these blocks in sequence yields the constructed neural network model.

[0045] S201, the machine learning function block group instance calls the machine learning extension module interface function

[0046] Specifically, when the Exe pin is True, the machine learning extension module initialization function block will call the extension module initialization interface function exported by the machine learning extension module to implement the initialization of the machine learning extension module. If the initialization is successful, the Done pin is True, otherwise the Error pin is True; the executable module loading function block will call the executable module loading interface function with the value of the ModulePath pin as a parameter, and if the loading is successful, it will return a pointer value pointing to the executable module object; the dataset loading function block will call the dataset loading function with the value of the DataPath pin as a parameter, and if the loading is successful, it will return a pointer value pointing to the dataset object; the model building function block will call the model building function, and if the construction is successful, it will return a pointer value pointing to the model object; the model network layer function block will call the model network layer interface function, The input tensor is calculated and the output tensor is returned; the model loading function block will call the model loading function according to ModelPath, and will return the pointer value of the model object if the loading is successful; the model saving function block will call the model saving function according to SavePath, and if the saving is successful, the Done pin will be set to True; the model training function block needs to input parameters such as the training set object, number of training cycles, and learning rate. This function block will call the model training function, and will return the Loss value and accuracy if the training is completed; the model testing function block needs to input the test set object, and this function block will call the model testing interface function. If the test is completed, the Loss value and accuracy of the test set will be returned; finally, the model prediction function block needs to input the image data to be predicted, as well as the length, width, and number of channels of the image, and call the model prediction interface function. If the prediction is completed, the prediction result will be returned.

[0047] S202. The machine learning extension module loads and executes the machine learning executable module.

[0048] The structure of the machine learning extension module is as follows Figure 4As shown. It includes a set of exported interface functions and an extension module initialization module, a machine learning executable module loading module, a machine learning execution module, and a machine learning basic library. The interface function consists of three types of functions, namely, the extension module initialization function for initializing the extension module; the executable module loading function for loading the machine learning executable module; and a set of execution functions for executing the executable module. Among them, the extension module initialization interface function calls the extension module initialization module to implement the initialization of the extension module. The initialization process includes loading the machine learning basic library into the memory and initializing the extension module global variables; the machine learning executable module loading function calls the executable module loading module to load the machine learning executable module into the memory, allocates space for the functions and variables defined by the executable module, and initializes the stack space for the executable module to run; the dataset loading function, model building function, model network layer function, model loading function, model saving function, model training function, model testing function, and model prediction function call the interface function corresponding to the machine learning executable module through the machine learning execution module.

[0049] The structure of the machine learning executable module is as follows Figure 5As shown. It at least contains the specific implementation of the dataset loading function, model building function, model loading function, model saving function, model network layer function, model training function, model testing function and model prediction function. Specifically, the input parameter of the dataset loading function is: dataset path. If there is a dataset file that meets the requirements under the path, the file will be read and the dataset object will be returned; the input parameter of the model building function is: tensor object. The function constructs a model object with the network structure through the network layer connection information recorded inside the tensor object and returns the object; the model network layer function includes the following functions: input layer function, convolution layer function, fully connected layer function, pooling layer function, Softmax layer function, among which: the input parameter of the input layer function is: tensor shape The function will return a tensor object with the input shape; the input parameters of the convolution layer function are: tensor object, number of convolution kernels, convolution kernel size, convolution step, activation function, the function will perform convolution operation on the input tensor object and return the processed tensor object; the input parameters of the fully connected layer function are: tensor object, number of filter kernels, activation function, the function will perform full connection operation on the input tensor object and return the processed tensor object; the input parameters of the pooling layer function are: tensor object, filter kernel size, step, the function will perform full connection operation on the input tensor object and return the processed tensor object. The input parameters of the Softmax layer function are: tensor object, activation function, the function will perform the Softmax operation on the input tensor and return the processed tensor object; the input of the model loading function is: model path, if there is a model file that meets the requirements under the path, the file will be loaded and the model object will be returned; the input parameters of the model saving function are: model object, save path, the function will save the model object as a model file under the specified path; the input of the model training function is: model object, dataset object, number of training cycles, learning rate, loss function, optimizer type, the function will train the model object according to the given parameters and return the training results, such as accuracy and loss value; the input parameters of the model testing function are model object, training set object, the function will test the model on the test set and return the accuracy and loss value on the test set; the input parameters of the model prediction function are: input data, such as the image to be predicted, model object, the function will calculate the input data through the model object and return the prediction result.

[0050] This example provides a specific implementation method to verify the ability of the above technical solution to build and run machine learning models:

[0051] The dataset used in this example is a steel surface defect dataset. This dataset includes 6 types of steel surface defects, each with 300 images. The image size is 224×224 grayscale images. The dataset is split into training and test sets according to a 7:3 ratio. The model construction function constructs a convolutional neural network with 20 network layers. Its structure is as follows Figure 6 As shown. The network inputs an image of size 224×224×1 and outputs a prediction result of 6×1. In the forward propagation process, the input image is first input into the convolution layer with a convolution kernel size of 7×7, 64 convolution kernels, a step size of 2, and an activation function of Relu. Then it passes through a 3×3 maximum pooling layer with a step size of 2 to obtain an output of 112×112×64; then it is input into four concatenated convolution blocks to obtain an output of 7×7×512; finally, it passes through a global average pooling layer and a Softmax layer to obtain the prediction result. The model training function sets the batch size to 15, the number of training cycles to 100, the initial value of the learning rate to 0.001, and follows The attenuation factor is reduced, and the minimum learning rate is set to 0.00001; the model test function uses a test set of 6 categories and 30 pictures in each category for testing, and all test results are averaged to obtain the accuracy and loss value of the test set; the prediction function first scales the image to be predicted to 224×224, and then predicts the classification result through the trained model. The model training and prediction results are shown in Figure 2. Figure 7 shown.

[0052] The present invention provides a machine learning function block execution mechanism in a programmable controller, which can run machine learning code through a machine learning extension module, expanding the capability boundaries of existing PLCs and enabling them to have the ability to build and run machine learning models.

[0053] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions, and variations of these embodiments, including components, without departing from the principles and spirit of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for executing a machine learning function block in a programmable controller, characterized in that: The invention comprises a machine learning function block group (102), a machine learning extension module (103), and a machine learning executable module (104), wherein the machine learning function block group (102) is used to call the interface function exported by the machine learning extension module (103); the machine learning extension module (103) is used to load and execute the machine learning executable module (104) through the exported interface function; the machine learning executable module (104) is used to execute a data set loading function, a model building function, a model network layer function, a model loading function, a model saving function, a model training function, a model prediction function, and a model testing function. The specific steps are as follows: S200, scheduling and executing machine learning function block group instances during PLC runtime; S201, the machine learning function block group instance calls the interface function of the machine learning extension module; The method to call the interface function is: The machine learning extension module initialization function block calls the extension module initialization interface function exported by the machine learning extension module; the executable module loading function block calls the executable module loading interface function with the value of the ModulePath pin as a parameter; the dataset loading function block calls the dataset loading function with the value of the DataPath pin as a parameter; The model building function block will call the model building function; the model network layer function block will call the model network layer interface function, calculate the input tensor, and return the output tensor; The model loading function block will call the model loading function according to ModelPath; The model saving function block will call the model saving function according to SavePath; the model training function block requires input of the training set object, number of training cycles, and learning rate; The model test function block needs to input the test set object; the model prediction function block needs to input the image data to be predicted as well as the length, width and number of channels of the image, and the prediction result will be returned after the prediction is completed. S202. The machine learning extension module loads and executes the machine learning executable module.

2. The method for executing a machine learning function block in a programmable controller according to claim 1, wherein: The machine learning function block group (102) includes a machine learning extension module initialization function block, a machine learning executable module loading function block, a data set loading function block, a model building function block, a model network layer function block, a model loading function block, a model saving function block, a model training function block, a model testing function block, and a model prediction function block.

3. The method for executing a machine learning function block in a programmable controller according to claim 1, wherein: The definition and implementation of the machine learning function block group (102) conform to the IEC61131-3 standard, and are respectively instantiated in the five programming languages ​​defined in the IEC61131-3 standard, and the written programs can be scheduled and executed by the PLC during operation.

4. The method for executing a machine learning function block in a programmable controller according to claim 1, wherein: The machine learning extension module includes an extension module initialization module, a machine learning executable module loading module, a machine learning execution module and a machine learning basic library.

5. The method for executing a machine learning function block in a programmable controller according to claim 4, wherein: The interface functions exported by the machine learning extension module (103) include an extension module initialization interface function, an executable module loading interface function, a data set loading interface function, a model construction interface function, a model network layer interface function, a model loading interface function, a model saving interface function, a model training interface function, a model prediction interface function, and a model testing interface function.

6. The method for executing a machine learning function block in a programmable controller according to claim 2, wherein: The model network layer functional block includes an input layer functional block, a convolutional layer functional block, a fully connected layer functional block, a pooling layer functional block, and a Softmax layer functional block which are connected in sequence.

7. The method for executing a machine learning function block in a programmable controller according to claim 1, wherein: In step S202, the method of loading and executing is: The extension module initialization interface function calls the extension module initialization module to initialize the extension module. The initialization process includes loading the machine learning basic library into the memory and initializing the extension module global variables. The machine learning executable module loading function calls the executable module loading module to load the machine learning executable module into the memory, allocates space for the functions and variables defined by the executable module, and initializes the stack space for the executable module to run; the dataset loading function, model building function, model network layer function, model loading function, model saving function, model training function, model testing function and model prediction function call the corresponding interface function of the machine learning executable module through the machine learning execution module.

Images