An AI-based method for controlling industrial production process data of a SCADA system

Abstract

The application discloses an AI-based SCADA system industrial production process data control method, solves the problems of poor user interaction, inaccurate instruction issuing, complicated platform operation, intelligent deficiency, interactive limitation and model expansibility of the current SCADA system, collects voice data of human-computer interaction and identifies the method, establishes a dialogue interaction model based on a GPT-3 model fine-tuned knowledge base and emotion analysis, and through the original electric signal data of various types of collected equipment, the original electric signal data is processed, the digital quantity of the control signal is sent to the equipment through a preset point and an S7 communication protocol to turn on or off the equipment, the human-computer interaction efficiency, accuracy and safety are improved, the labor cost is reduced, the data processing efficiency and operation production efficiency are improved, and the system integration and expansibility are enhanced. The application belongs to the field of industrial control.

Description

Technical Field

[0001] This invention relates to the field of industrial control, and in particular to a method for controlling industrial production process data using an AI-based SCADA system. Background Technology

[0002] SCADA (Supervisory Control and Data Acquisition) systems are one of the core technologies in modern industrial automation. They enable remote monitoring, real-time control, and data acquisition of the entire production process, and are widely used in industries and fields such as power, water environment management, petrochemicals, transportation, and environmental protection. However, traditional SCADA systems still rely heavily on manual monitoring and input, which limits production efficiency and increases the possibility of errors. Furthermore, traditional SCADA systems often lack flexibility and cannot adapt to constantly changing production needs. Currently, SCADA systems primarily interact through Human-Machine Interfaces (HMIs), requiring users to input commands and view data via touchscreens, mice, and keyboards. Operators often manually read meters or analyze system spreadsheets to write summary reports. This interaction method has limitations, such as the need for on-screen operation, operator errors, discrepancies between reports and actual conditions, and inconvenience for operators in complex environments. For operators new to SCADA platforms, this is relatively unfamiliar, and the traditional interaction method undoubtedly increases their cognitive burden.

[0003] With the rapid development of technologies such as computer networks, data communication, and automation, industrial automation is moving towards the integration and digitalization of comprehensive enterprise monitoring, control, and management. While traditional industrial control systems have applied a range of information technologies and automation integration at different levels, they have largely focused on closed-loop processes and internal management within the production process. Common problems include poor user interactivity, inaccurate command issuance, cumbersome platform operation, and a lack of system intelligence, which to some extent seriously hinder the further development, utilization, and research of industrial control systems.

[0004] In recent years, with the development of the internet and artificial intelligence, some AI-based SCADA systems have gradually emerged. These systems can automate some monitoring and control tasks, improving efficiency and reducing errors. However, existing AI SCADA systems mainly rely on fixed algorithms and predefined rules, which typically require manual hard coding and maintenance. Furthermore, these systems largely focus on the hardware level, rarely considering upgrades to user-computer interaction or higher-level applications and expansions such as intelligent system operation. This limits the intelligence level of these systems, increases operating costs, and restricts their flexibility and responsiveness. Therefore, the lack of intelligence, limitations in interaction, and the scalability of the model are the difficulties and challenges of this invention. Summary of the Invention

[0005] To address the shortcomings of current AI-based SCADA systems, such as poor user interactivity, inaccurate command issuance, cumbersome platform operation, insufficient intelligence, limited interaction, and poor model scalability, this invention aims to improve human-computer interaction efficiency, accuracy, and security, reduce labor costs, increase data processing and operational efficiency, and enhance system integration and scalability. This invention provides a method for data control of industrial production processes using an AI-based SCADA system, comprising the following steps:

[0006] Step 1: Voice Interaction

[0007] During the voice interaction process of the SCADA system, voice data of human-computer interaction is collected and recognized. The voice data recognition adopts a recurrent neural network and uses the Jieba word segmentation model to perform Chinese word segmentation and classification to obtain effective command vocabulary combinations. Commands are matched in the vocabulary library to form a command vocabulary library and continuously maintain it.

[0008] Step 2 Model Integration Processing

[0009] A conversational interaction model was established based on a knowledge base fine-tuned by the GPT-3 model and sentiment analysis. After fine-tuning based on the natural language model and the DALL·E model, the SCADA system converted text into images and generated reports by calling the model interface. The conversational interaction model was called and returned results through the machine learning model provided by the OpenAI API, which uses the unified Hypertext Transfer Protocol (HTTP) interface.

[0010] Step 3 Data Acquisition and Control

[0011] The SCADA system acquires raw electrical signal data from various types of equipment to form raw datasets, converts the raw electrical signal data into digital or analog quantities to form target data, and transmits the target data through Siemens' S7 communication protocol or Modbus protocol. The SCADA system receives the target data and sends commands to perform data control, sending the digital quantities of equipment control signals to the equipment through preset points and the S7 communication protocol to enable or disable power.

[0012] Preferably, the speech data recognition in step 1 refers to encapsulating the acquired audio data into different audio container formats during the audio acquisition process to generate speech packets; converting the input audio into feature representations through hidden layers, performing time-series modeling on the features through recursive layers, and finally converting the features into a transcribed output sequence through fully connected layers and the Softmax function; and using a loss function to enable the network to be trained without detailed alignment information.

[0013] The audio-to-feature representation process involves preprocessing the audio data after acquisition. This preprocessing divides the audio data into small time segments to form a frame, and the audio signal within each frame is converted into spectral information. Short-time Fourier transform is then used for spectral analysis, converting the waveform signal in the time domain into a frequency domain representation to obtain the spectrum of each frame. The data in the spectrum is then used to construct the feature representation.

[0014] One type of recurrent neural network (RNN) is used to process sequential data. In speech recognition, it is used to model audio signals using RNN sequences, establishing a sequence model that establishes the relationship between audio signals and time dependencies.

[0015] Feature-to-sequence transcription output refers to converting the results of feature extraction and RNN sequence modeling into text transcription output corresponding to the input audio through a fully connected layer and a Softmax normalization function; in speech recognition tasks, it maps audio data to corresponding text transcription.

[0016] The loss function is represented by the CTC loss function; the CTC loss function is used to train the model for sequence transcription without detailed alignment information; the goal of the CTC loss is to maximize the probability that the model generates the correct output sequence for a given input sequence.

[0017]

[0018] Where X represents the input sequence and Y represents the target sequence; L(X,Y) is the CTC loss corresponding to the input sequence X and the target sequence Y; B -1(Y) is the set of all possible paths corresponding to the output label sequence Y; π is a path that represents the correspondence from the input sequence to the output label sequence; P(π|X) is the conditional probability of path π given the input sequence X.

[0019] Preferably, the loss function is used to train the network without detailed alignment information, including the following steps:

[0020] Step 1-101 Preprocessing

[0021] The Mel-frequency cepstral coefficient (MFCC) method is used to convert the raw audio data into training data suitable for RNN model processing, taking the input speech packet as the audio signal. The processing includes noise reduction, segmentation, sampling rate conversion, and feature extraction. The MFCC method utilizes human auditory characteristics, first mapping the linear spectrum to the Mel-frequency nonlinear spectrum based on auditory perception, and then converting it to the cepstral spectrum. The formula for converting ordinary frequencies to Mel frequencies is as follows:

[0022] mel(f) = 2595 * log 10 (1+f / 700)

[0023] mel(f) represents the Mel frequency, which is the Mel scale value corresponding to the linear frequency f and is used to simulate the human ear's perception of frequency.

[0024] Step 1-102: Construct a Recurrent Neural Network (RNN) model

[0025] A Recurrent Neural Network (RNN) model for processing sequential data is constructed using an RNN. An RNN model typically includes an input layer, one or more hidden layers, and an output layer. Hidden layers usually consist of Long Short-Term Memory (LSTM) units or Gated Recurrent Units (GRUs). By passing information through the recurrent structure of the RNN, the temporal relationships in the sequential data are obtained. The mathematical expression of the RNN model is as follows:

[0026] In a basic RNN unit, assuming the input is x at time synchronization t. t The hidden state (output) is h. t The update process of RNN is as follows:

[0027] h t =σ(W hx ·x t +W hh ·h t-1 +b h )

[0028] Where: h t It is the output of the hidden state of time synchronization t; xt It is the input for time synchronization t;

[0029] W hx W is the weight matrix input to the hidden state; hh It is the weight matrix from the hidden state at the previous time step to the hidden state at the current time step; b h σ is the bias vector of the hidden state; σ is the activation function.

[0030] Step 1-103: Train the model

[0031] The model is trained using labeled audio files and corresponding transcribed training data; during training, the CTC loss function is used to allow the network to learn the mapping from audio sequences to text sequences under detailed misalignment of the input training data.

[0032] Decoding Steps 1-104

[0033] The raw output of the trained model is decoded to obtain the final text result; the output of the CTC loss function is a probability distribution, which gives the probability of all possible output characters for each time step; the most likely output sequence is selected from this probability distribution using BeamSearch;

[0034] Steps 1-105: Evaluation and Optimization

[0035] The model's performance is evaluated using pre-prepared training, development, and test sets. Model parameters or structure are adjusted as needed. The development and test sets are derived from the partitioning and processing of the original dataset. The dataset is divided proportionally: 80% training set, 10% development set, and 10% test set. During actual evaluation, the original development and test datasets are input into the RNN model, and the RNN model's output predictions are recorded and compared with preset metrics to measure device performance. These expected metrics are empirical values ​​of device performance in real-world scenarios. The comparison results determine whether further optimization is needed.

[0036] Preferably, the Chinese word segmentation and classification processing using the Jieba word segmentation model in step 1 of the voice interaction further includes the following steps:

[0037] Step 1-201: Construct a prefix dictionary

[0038] Based on the provided default dictionary library, Jieba segmentation will build a prefix dictionary, and the prefix of each word will be stored in the dictionary;

[0039] Step 1-202 Dynamic Programming Word Segmentation

[0040] Jieba segmentation uses a prefix dictionary-based dynamic programming algorithm to find the optimal word segmentation method. For a sentence to be segmented, Jieba segmentation attempts to find a segmentation method that maximizes the sum of the frequencies of the word sequences corresponding to this segmentation method.

[0041] Step 1-203: Processing Unregistered Keywords

[0042] Hidden Markov Model (HMM) is used to process out-of-vocabulary (OV) words. For words that are not in the prefix dictionary, Jieba segmentation uses an OV word recognition technique based on the Hidden Markov Model. This model infers OV words based on the transition probabilities between characters.

[0043] Hidden Markov Models (HMMs) are models that use four labels and are trained using existing manually labeled word data. The basic formula of an HMM mainly consists of three parts:

[0044] 1) State transition probability matrix A:

[0045] a ij =P(i t+1 =q j |i t =q i )

[0046] This indicates that the state is q at time t. i Under the condition that the state transitions to state q at time t+1 j The probability of;

[0047] 2) Observation probability matrix B:

[0048] b jk =P(o t =v k |i t =q i )

[0049] This indicates that the state is q at time t. j Generate observations v under the condition of k The probability of;

[0050] 3) Initial state probability vector π:

[0051] π i =P(i1=q i )

[0052] This indicates that the state is q at time t=1. i The probability of;

[0053] The four tags include B: word beginning, M: word middle, E: word ending, and S: single word;

[0054] Step 1-204: Add a custom dictionary

[0055] Adding a user-defined dictionary: Jieba segmentation allows users to add custom dictionaries, overriding the default dictionary and handling special segmentation needs to obtain valid instruction phrases; obtaining valid instruction phrases refers to one or more instruction words that can be matched in the existing vocabulary.

[0056] Preferably, the step of performing command matching in the lexicon to form and maintain a command vocabulary in the voice interaction process refers to the following steps after obtaining valid command phrases through speech recognition and Chinese word segmentation:

[0057] Step 1-301: Loading the dictionary

[0058] Add words to the word segmentation dictionary, separated by \t. The dictionary format is the same as dict.txt, with one word per line. Each line consists of three parts: word, word frequency, and part of speech, separated by spaces. The order cannot be reversed and the words are added to the word tree. Load the custom dictionary at the specified path. The dictionary can be dynamically modified in the program using add_word and del_word. The word frequency of a single word can be adjusted using suggest_freq. The custom dictionary is added to the model using jieba.load_userdict.

[0059] Step 1-302 Command Matching

[0060] For matching words after speech word segmentation in the instruction library, the SCADA system searches for target words layer by layer from the process word library, equipment word library, and function word library of the instruction vocabulary library. After matching the target word in the process word library, it further matches the equipment, and so on. Three modes are used respectively: full matching, pinyin matching, and similarity calculation matching.

[0061] Step 1-303 Instruction Correction

[0062] After the instruction matching is completed, if the pronunciation is inaccurate or the similarity calculation is off, or if the instruction is not what the user actually wants to execute, the user can further modify the recognized instruction. The user will then make corrections and the corrected instruction mapping will be further maintained in the SCADA system instruction vocabulary library.

[0063] Step 1-304 Instruction Dictionary Maintenance

[0064] After word segmentation and recognition, instruction matching, and instruction correction are completed, the following four situations may occur:

[0065] a) If all matches are successful, proceed directly;

[0066] b. If the pinyin matching or similarity calculation is successful and the user has not corrected the settings, proceed directly.

[0067] c. Execute after user correction following pinyin matching or similarity calculation matching;

[0068] d. If the deviation after matching is too large and no match is found, and the user does not correct it, the execution is abandoned;

[0069] For cases a and d, steps 1-304 are not processed. For cases b and c, a mapping relationship is established between the matched words and the corrected words, and this relationship is maintained in the instruction vocabulary.

[0070] Step 1-304 Action Execution

[0071] After matching the specific actions and devices, the system sends an HTTP request to the SCADA server, and the SCADA system executes the on-site actions upon receiving the request.

[0072] Preferably, in step 1-302, the pinyin matching and similarity calculation in the instruction matching further use pinyin matching to make up for the accuracy of speech recognition. The go-pinyin library is used to realize the pinyin conversion, and after dictionary mapping, it is processed by tone processing, polyphonic character processing and non-Chinese character processing.

[0073] Similarity is calculated using Levenshtein edit distance, defined as the Levenshtein distance between two strings A and B as:

[0074]

[0075] Where i and j represent the index positions of strings A and B respectively, and a and b are the lengths of strings A and B respectively;

[0076] Once a matching instruction is received and the user confirms its issuance, if the vocabulary for that instruction does not appear in the dictionary, it will be automatically added to the dictionary.

[0077] Preferably, the fine-tuning of the natural language model and the DALL·E model in step 2 of the model integration process includes the following steps:

[0078] Step 201 Dataset Preparation

[0079] The input text and the corresponding target dataset output are consistent with OpenAI's usage policies and relevant laws and regulations. The data is a JSONL document, where each line is a suggestion completion pair.

[0080] The prompt-complete pair refers to the format of the training data used to fine-tune OpenAI's models. When fine-tuning the model, a dataset containing prompts and corresponding target outputs is typically provided. The prompt-complete pair is used to prompt the model to complete or complete a task given the prompt. A prompt-complete pair typically includes the following two main parts:

[0081] Prompt: This represents an input text, usually a sentence, question, or task description, used to guide the model to generate relevant output; a prompt is a description of the task you want the model to answer or complete.

[0082] Target Output: This is the text or answer generated by the expected model in association with the prompt; it is the correct answer or text generated by the expected model given the prompt.

[0083] Step 202: Create a fine-tuning environment

[0084] Create a fine-tuning environment on the OpenAI console, select the base model, and then select the fine-tuning task option. The base model includes the fine-tuning functional model GPT-3.5-turbo, the chatbot program natural language model, the image generation system DALL·E, and the command-line interface CLI.

[0085] Step 203: Set fine-tuning parameters

[0086] The fine-tuning task selects appropriate parameters, including the number of fine-tuning steps, learning rate, and batch size.

[0087] Step 204: Set fine-tuning data

[0088] Upload the prepared dataset to the fine-tuning environment, format the dataset according to OpenAI's guidelines, and follow the data upload instructions;

[0089] Step 205 begins fine-tuning

[0090] To start the fine-tuning process, select "Start using API for fine-tuning" on the OpenAI console. This will trigger the fine-tuning process, and OpenAI will train the model using the provided dataset.

[0091] Step 206: Monitor the fine-tuning progress.

[0092] To track the progress and performance of fine-tuning, the OpenAI console will provide the status and metrics of the fine-tuning task, including training steps and training loss, which can be adjusted as needed.

[0093] Step 207 uses the new fine-tuning model

[0094] After fine-tuning, select the already created model from the playground model in the OpenAI console, and determine whether the output result matches the expected metric. If the result is yes, call the good model directly through the API.

[0095] Preferably, the data acquisition in step 3 further includes the following steps:

[0096] Step 3-101: The SCADA system constructs an I / O driver layer, using an IGS gateway to integrate the protocols corresponding to various programmable logic controllers (PLCs) and instruments, and directly interacts with the hardware. The I / O driver temporarily stores the data read back in the Driver Image Table (DIT).

[0097] Step 3-102 Configure the Scan, Alarm and Control (SAC) service in the ifix task. The SAC service will handle periodic scanning of data, alarm processing and data control.

[0098] Step 3-103 configures the I / O drive address in the process database PDB data block, stores the data read from the DIT, and records the real-time final acquired digital / analog values ​​of all points; the historical database periodically collects data from the process database PDB and records history, serving as the data source supporting historical curves and reports. Simultaneously, it synchronizes the device status data to the relational database via the historical database; the relational database synchronizes the historical database data to the SQL Server server through the AccessClient data synchronization service to obtain report data. At the same time, alarm information is also synchronized to the SQL Server server through the historical alarm service started by .ifix; Digital quantities are a data type representing discrete states, typically collecting switch states or binary information; analog quantities are a continuously changing data type, typically collecting values ​​within a continuous range, such as voltage, current, temperature, and pressure; raw data acquisition refers to the raw electrical signal data directly obtained from sensors or devices. Raw signal data can be continuously changing analog data or discrete-state digital data; the SCADA system converts the raw signal data into digital or analog quantities to form target data.

[0099] Step 3-104: The SCADA engineering station includes the display and control of the configuration interface. The display and control of the configuration interface mainly includes graphical objects, controls, schedulers, and chart objects. Each station in the SCADA engineering station has an independent workstation. The SCADA deployment adopts a distributed deployment architecture. A master station is defined on the upper layer of each substation, and the data of each substation can be directly monitored through the master station.

[0100] Preferably, step 3, where the SCADA system obtains the raw dataset and sends commands for data control, refers to communicating and exchanging data with the device through the IoT Gateway plugin to collect and transmit data, and to perform device authentication, data encryption, and access control; converting the format of data from different devices; collecting real-time data, monitoring device status, performing remote operations, and integrating and collaborating with other systems.

[0101] The specific working steps of the IoT Gateway plugin are as follows:

[0102] Step 3-201 Equipment Configuration

[0103] Configure device parameters in the IoT Gateway plugin. Device parameters include the device's IP address, port number, and communication protocol.

[0104] Step 3-202 Data Acquisition

[0105] After the device is configured, the IoT Gateway plugin begins to collect data from the device through the OPC Server. The collected data includes the device's operating status and sensor readings.

[0106] Step 3-203 Data Processing

[0107] The IoT Gateway plugin collects raw binary data, converts the raw binary data and message data encapsulated through various protocols into target data in analog and digital form, and finally reports the collected target data values ​​through the TCP protocol.

[0108] Step 3-204 Data Transmission

[0109] The processed target data is transmitted to the SCADA system, which performs data analysis, equipment monitoring, and fault diagnosis.

[0110] Step 3-205 Command Sending

[0111] In addition to acquiring target data, the IoT Gateway plugin also sends control commands to the device. It sends the start signal digital quantity to the PLC's CPU through preset points and the S7 communication protocol. After receiving the signal, the PLC powers on the device. Here, a point refers to a data acquisition point or a data input / output point, representing a specific monitoring or control data item, used to indicate various parameters, states, or device interfaces that need to be monitored or controlled. The S7 communication protocol is a communication protocol customized by Siemens for its PLC series, used to transmit data and control commands between the computer and the PLC. In the SCADA system, the operator uses controls on the user interface to send control commands to preset points.

[0112] Preferably, the IoT Gateway plugin also integrates and works collaboratively with other systems. Through the access of the IoT Gateway, the system realizes the interconnection of devices and the integration of data, and performs device management, remote monitoring, data analysis and optimization. It uses the unified HTTP REST and MQTT interface at the upper layer. The upper layer model integration and remote control all rely on the HTTP REST and MQTT standard interface.

[0113] When the IoT Gateway plugin provides an HTTP interface, and the upper-layer service integrates the IoT Gateway plugin, after selecting the network adapter, specifying the port and the routing URL, the IoT Gateway plugin will start a cluster search REST server according to the above-configured parameters. The upper-layer service can then directly call the API to obtain and send data.

[0114] Among them, data acquisition refers to the upper-layer service acting as the caller, obtaining the device's operating status, faults, and parameter data through the HTTP interface, and using it as the basic data input for the integration of the upper-layer model; data delivery refers to sending the command data for controlling the device to start and stop to the IoT Gateway service through the HTTP interface, thereby realizing the final remote control.

[0115] This invention provides a method for data control of industrial production processes using an AI-based SCADA system. It addresses the problems of poor user interactivity, inaccurate command issuance, cumbersome platform operation, insufficient intelligence, limited interaction, and poor model scalability in current AI-based SCADA systems. This invention collects and recognizes voice data from human-computer interaction, establishes a conversational interaction model based on a knowledge base fine-tuned using the GPT-3 model and sentiment analysis, and processes the raw electrical signal data from various types of equipment. The digital control signal is then sent to the equipment via preset points and the S7 communication protocol to enable or disable power. This improves the efficiency, accuracy, and safety of human-computer interaction, reduces labor costs, increases data processing efficiency and operational efficiency, and enhances system integration and scalability. Attached Figure Description

[0116] Figure 1 This is a schematic diagram of a method for controlling industrial production process data using an AI-based SCADA system.

[0117] Figure 2 This is a schematic diagram of the overall process of voice interaction for a method of data control in industrial production processes using an AI-based SCADA system.

[0118] Figure 3 This is a schematic diagram of the voice acquisition process for an AI-based SCADA system for industrial production process data control.

[0119] Figure 4 This is a schematic diagram of the speech recognition model processing flow of a method for data control in industrial production processes using an AI-based SCADA system.

[0120] Figure 5 This is a flowchart of Chinese word segmentation processing for a method of industrial production process data control in an AI-based SCADA system.

[0121] Figure 6 This is a flowchart illustrating the instruction matching and maintenance process of an AI-based SCADA system for industrial production process data control.

[0122] Figure 7 This is a method for controlling industrial production process data in an AI-based SCADA system, using pinyin matching instructions. (Key steps diagram)

[0123] Figure 8 This is an AI-based method for industrial production process data control in SCADA systems. (Open AI intelligent model fine-tuning integration flowchart)

[0124] Figure 9This is a data acquisition and data flow flowchart of an AI-based SCADA system for industrial production process data control.

[0125] Figure 10 This is a schematic diagram of an AI-based SCADA system for controlling industrial production process data, specifically an IoT Gateway. Detailed Implementation

[0126] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements.

[0127] A method for data control of industrial production processes based on AI-powered SCADA systems, such as... Figure 1 As shown, the method includes the following steps;

[0128] Step 1: Voice Interaction

[0129] During the voice interaction process of the SCADA system, voice data of human-computer interaction is collected and recognized. The voice data recognition adopts a recurrent neural network and uses the Jieba word segmentation model for Chinese word segmentation and classification to obtain effective command vocabulary combinations. Commands are matched in the vocabulary library to form a command vocabulary library, which is continuously maintained. Here, the command vocabulary library refers to a set of commands that combine the actual equipment number on site with some switching commands from the user. By issuing a specific command from the command vocabulary library, a specific device can be switched on or off.

[0130] Step 2 Model Integration Processing

[0131] A conversational interaction model was built based on a knowledge base fine-tuned using the GPT-3 model and sentiment analysis. After fine-tuning based on natural language models and the DALL·E model, the SCADA system converts text into images and assists in report generation by calling the model interface. All models are called and return results through the machine learning models provided by the OpenAI API, which uses the unified Hypertext Transfer Protocol (HTTP) interface. The OpenAI API provides a series of machine learning models, each with its own characteristics, which can handle virtual tasks involving natural language or code.

[0132] Step 3 Data Acquisition and Control

[0133] The SCADA system acquires raw electrical signal data from various types of equipment to form a raw dataset. This raw electrical signal data is then converted into digital or analog quantities to form target data (e.g., for instrument data, the raw electrical signal is converted into floating-point analog data). The target data is transmitted via Siemens' S7 communication protocol or Modbus protocol. Upon receiving the target data, the SCADA system sends commands for data control. The digital control signal is sent to the equipment via a pre-defined point (i.e., the output of a PLC control module, with a fixed and unique address; a value of 1 indicates the point is powered on, and 0 indicates it is powered off, thus controlling the opening and closing of the equipment) and the S7 communication protocol to enable or disable the equipment.

[0134] In one embodiment, the speech data recognition in step 1 refers to encapsulating the acquired audio data into different audio container formats to generate speech packets during the audio acquisition process; converting the input audio into feature representations through hidden layers, performing time-series modeling on the features through recursive layers, and finally converting the features into a transcribed output sequence through fully connected layers and the Softmax function; and using a loss function to enable the network to be trained without detailed alignment information.

[0135] Audio conversion into feature representation refers to the process after audio data acquisition, through preprocessing such as noise reduction and audio normalization, the audio data is divided into small time segments called frames. Preprocessing divides the audio data into small time segments to form a frame, and the audio signal within each frame is converted into spectral information. Short-Time Fourier Transform (STFT) is used for spectral analysis, converting the time-domain waveform signal into a frequency-domain representation, resulting in the spectrogram of each frame. The data in the spectrogram is used to construct the feature representation; a commonly used feature representation is Mel-Frequency Cepstral Coefficients (MFCCs). MFCCs are coefficients extracted from the spectrogram that capture sound characteristics. Through a series of mathematical transformations, the spectrogram data is mapped to a low-dimensional feature space. These MFCCs represent the frequency and temporal characteristics of the audio (i.e., a set of coefficients extracted from the audio spectrogram that capture important numerical or temporal properties of the audio).

[0136] Sequence modeling refers to the Recurrent Neural Network (RNN) itself. An RNN is a type of neural network designed to process sequential data, aiming to capture temporal relationships and contextual information within the data. In speech recognition, RNNs are used to model audio signal sequences and establish the temporal dependencies between the audio signal and the data.

[0137] Feature-to-sequence transcription output refers to converting the results of feature extraction and RNN sequence modeling into text transcription output corresponding to the input audio through a fully connected layer and a Softmax normalization function; in speech recognition tasks, it maps audio data to corresponding text transcription.

[0138] The loss function is represented as the Connectionist Temporal Classification (CTC) loss function. The CTC loss function is used to train the model for sequence transcription without detailed alignment information. The goal of the CTC loss is to maximize the probability that the model generates the correct output sequence for a given input sequence.

[0139]

[0140] Where X represents the input sequence and Y represents the target sequence; L(X,Y) is the CTC loss corresponding to the input sequence X and the target sequence Y; B -1 (Y) is the set of all possible paths corresponding to the output label sequence Y; π is a path that represents the correspondence from the input sequence to the output label sequence; P(π|X) is the conditional probability of path π given the input sequence X.

[0141] In one embodiment, a loss function is used to train the network without detailed alignment information, including the following steps:

[0142] Step 1-101 Preprocessing

[0143] The Mel Frequency Cepstrum Coefficient (MFCC) method is used to convert the raw audio data into training data suitable for RNN model processing, taking the input speech packet as sound signal. The processing includes noise reduction, segmentation, sampling rate conversion, and feature extraction. The MFCC method utilizes human auditory characteristics, first mapping the linear spectrum to the Mel nonlinear spectrum based on auditory perception, and then converting it to the cepstrum. The key formula for converting ordinary frequencies to Mel frequencies is as follows:

[0144] mel(f) = 2595 * log10 (1+f / 700)

[0145] mel(f) represents the Mel frequency, which is the Mel scale value corresponding to the linear frequency f and is used to simulate the human ear's perception of frequency.

[0146] Step 1-102: Construct a Recurrent Neural Network (RNN) model

[0147] This paper describes how to build a neural network model for processing sequential data using a Recurrent Neural Network (RNN). RNNs are a widely used type of neural network model for processing sequential data. The model typically includes an input layer, one or more hidden layers, and an output layer. Hidden layers usually consist of Long Short-Term Memory (LSTM) units or Gated Recurrent Units (GRUs). By passing information through the recurrent structure of the RNN, the temporal relationships in the sequential data are obtained. The mathematical expression of the RNN model is as follows:

[0148] In a basic RNN unit, assuming the input is x at time synchronization t. t The hidden state (output) is h. t The update process of RNN is as follows:

[0149] h t =σ(W hx ·x t +W hh ·h t-1 +b h )

[0150] Where: h t This is the hidden state (output) of time synchronization t; x t It is the input for time synchronization t;

[0151] W hx W is the weight matrix input to the hidden state; hh It is the weight matrix from the hidden state at the previous time step to the hidden state at the current time step; b h σ is the bias vector of the hidden state; σ is the activation function.

[0152] Step 1-103: Train the model

[0153] The model is trained using labeled training data (i.e., audio files and corresponding transcriptions); during training, the CTC loss function is used to allow the network to learn the mapping from audio sequences to text sequences under detailed misalignment of the input training data.

[0154] Decoding Steps 1-104

[0155] The raw output of the trained model is decoded to obtain the final text result. The output of the CTC loss function is a probability distribution, which gives the probability of all possible output characters for each time step; beam search is used to select the most likely output sequence from this probability distribution.

[0156] Steps 1-105: Evaluation and Optimization

[0157] The model's performance is evaluated using pre-prepared training, development, and test sets. Model parameters or structure are adjusted as needed. The training, development, and test sets are based on the original dataset's partitioning and processing. The original dataset is divided proportionally (e.g., 80% training, 10% development, 10% test) to achieve a more precise match. These datasets are collected from commonly used instructions in actual production processes. During actual evaluation, the original development and test datasets are input into the RNN model, and the RNN model's output predictions are recorded and compared with preset metrics to measure device performance. These metrics are empirical values ​​of device performance in real-world scenarios, and the comparison results determine whether further optimization is needed.

[0158] In one embodiment, the Chinese word segmentation and classification processing using the Jieba word segmentation model in step 1 of the voice interaction further includes the following steps:

[0159] Step 1-201: Construct a prefix dictionary

[0160] Based on the provided default dictionary library, Jieba segmentation will build a prefix dictionary, and the prefix of each word will be stored in the dictionary;

[0161] Step 1-202 Dynamic Programming Word Segmentation

[0162] Jieba segmentation uses a prefix dictionary-based dynamic programming algorithm to find the optimal word segmentation method. For a sentence to be segmented, Jieba segmentation attempts to find a segmentation method that maximizes the sum of the frequencies of the word sequences corresponding to this segmentation method.

[0163] Step 1-203: Processing Unregistered Keywords

[0164] Hidden Markov Model (HMM) is used to process out-of-vocabulary (OV) words. For words that are not in the prefix dictionary, Jieba segmentation uses an OV word recognition technique based on the Hidden Markov Model. This model infers OV words based on the transition probabilities between characters.

[0165] Hidden Markov Models (HMMs) are models that use four labels and are trained on existing manually labeled corpora (a collection of text resources of a certain quantity and size). The basic formula of an HMM mainly consists of three parts:

[0166] 1) State transition probability matrix A:

[0167] a ij =P(i t+1 =q j |i t =q i )

[0168] This indicates that the state is q at time t. i Under the condition that the state transitions to state q at time t+1 j The probability of;

[0169] 2) Observation probability matrix B:

[0170] b jk =P(o t =v k |i t =q i )

[0171] This indicates that the state is q at time t. j Generate observations v under the condition of k The probability of;

[0172] 3) Initial state probability vector π:

[0173] π i =P(i1=q i )

[0174] This indicates that the state is q at time t=1. i The probability of;

[0175] The four tags include B: word beginning, M: word middle, E: word ending, and S: single word;

[0176] Step 1-204: Add a custom dictionary

[0177] Adding a user-defined dictionary: Jieba segmentation allows users to add custom dictionaries, overriding the default dictionary, and can handle special segmentation requirements to obtain valid instruction phrases; obtaining valid instruction phrases refers to one or more instruction words that can be matched in the existing vocabulary.

[0178] In one embodiment, step 1, voice interaction, involves matching commands in the dictionary to form and maintain a command vocabulary. This means that after obtaining valid command phrases through speech recognition and Chinese word segmentation, the command matching process in the dictionary further includes the following steps:

[0179] Step 1-301: Loading the dictionary

[0180] Add words to the word segmentation dictionary, separated by \t. The dictionary format is the same as dict.txt, with one word per line. Each line consists of three parts: word, word frequency, and part of speech, separated by spaces. The order cannot be reversed and the words are added to the word tree. Load the custom dictionary at the specified path. The dictionary can be dynamically modified in the program using add_word and del_word. The word frequency of a single word can be adjusted using suggest_freq. The custom dictionary is added to the model using jieba.load_userdict.

[0181] Step 1-302 Command Matching

[0182] For matching words after speech word segmentation in the instruction library, the SCADA system searches for target words layer by layer from the process word library, equipment word library, and function word library of the instruction vocabulary library. After matching the target word in the process word library, it further matches the equipment, and so on. Three modes are used respectively: full matching, pinyin matching, and similarity calculation matching.

[0183] Step 1-303 Instruction Correction

[0184] After the instruction matching is completed, if the pronunciation is inaccurate or the similarity calculation is off, the instruction may not be what the user actually wants to execute. Therefore, the user is provided with the option to further modify the recognized instructions. The user can make modifications and corrections, and the corrected instruction mapping will be further maintained in the SCADA system instruction vocabulary library.

[0185] Step 1-304 Instruction Dictionary Maintenance

[0186] After word segmentation and recognition, instruction matching, and instruction correction are completed, the following four situations may occur:

[0187] a) If all matches are successful, proceed directly;

[0188] b. If the pinyin matching or similarity calculation is successful and the user has not corrected the settings, proceed directly.

[0189] c. Execute after user correction following pinyin matching or similarity calculation matching;

[0190] d. If the deviation after matching is too large and no match is found, and the user does not correct it, the execution is abandoned;

[0191] For cases a and d, steps 1-304 are not processed. For cases b and c, a mapping relationship is established between the matched words and the corrected words, and this relationship is maintained in the instruction vocabulary.

[0192] Step 1-304 Action Execution

[0193] After matching the specific actions and devices, the system sends an HTTP request to the SCADA server, and the SCADA system executes the on-site actions upon receiving the request.

[0194] In one embodiment, the pinyin matching and similarity calculation in step 1-302 of instruction matching also use pinyin matching to further compensate for the accuracy of speech recognition. The go-pinyin library is used to realize the pinyin conversion, which is then processed through dictionary mapping, tone processing, polyphonic character processing and non-Chinese character processing.

[0195] Similarity is calculated using Levenshtein edit distance, defined as the Levenshtein distance between two strings A and B as:

[0196]

[0197] Where i and j represent the index positions of strings A and B respectively, and a and b are the lengths of strings A and B respectively;

[0198] Once a matching instruction is received and the user confirms its issuance, if the vocabulary for that instruction does not appear in the dictionary, it will be automatically added to the dictionary.

[0199] In one embodiment, a conversational interaction model is established based on a knowledge base fine-tuned by the GPT-3 model and sentiment analysis, and text is converted into images based on a natural language model and the DALL·E model fine-tuned to facilitate report generation.

[0200] The fine-tuning based on the GPT-3 model includes the following steps:

[0201] Step 201 Dataset Preparation

[0202] The input text and the corresponding target dataset output are consistent with OpenAI's usage policies and relevant laws and regulations. The data is a JSONL document, where each line is a suggestion completion pair.

[0203] The prompt-complete pairs refer to the format of training data used for fine-tuning OpenAI models. When fine-tuning a model, a dataset containing prompts and corresponding target outputs is typically required. Prompt-Pairs are called complete pairs because they are commonly used to prompt the model to complete or "complete" a task given a prompt, such as generating text, answering questions, or translating languages. This concept originates from the fields of natural language processing and machine learning, where one text sequence (prompt) is paired with another text sequence (target output) to guide the model in performing a task. The format of this prompt-complete pair dataset is typically JSONL (JSON Lines), where each line is a JSON document describing a prompt-complete pair.

[0204] A suggestion completion pair typically includes the following two main parts:

[0205] Prompt: This represents an input text, usually a sentence, question, or task description, used to guide the model to generate relevant output; a prompt is a description of the task you want the model to answer or complete.

[0206] Target Output: This is the text or answer the model is expected to generate in relation to the prompt. It is the correct answer or text you expect the model to generate given the prompt.

[0207] By providing a large amount of such prompt-complete data, the model is taught how to perform a specific task, generate correct text, or produce content that matches the target output based on the prompts. This process is also known as supervised learning; through fine-tuning, the model can gradually learn and improve its performance on a specific task.

[0208] The data is a JSONL document, where each line is a prompt completion pair, corresponding to a training example, in the following format:

[0209] {"prompt":"<prompt text> ","completion":"<ideal generated text> "}

[0210] {"prompt":"<prompt text> ","completion":"<ideal generated text> "}

[0211] {"prompt":"<prompt text> ","completion":"<ideal generated text> "}

[0212] Step 202: Create a fine-tuning environment

[0213] Create a fine-tuning environment on the OpenAI console, select the base model, and then select the fine-tuning task option. The base models include GPT-3.5-turbo, a natural language model, and DALL·E. After that, install the command-line interface (CLI) using the following commands:

[0214] pip install --upgrade openai.

[0215] Step 202: Create a fine-tuning environment

[0216] Create a fine-tuning environment on the OpenAI console, select the base model, and then select the fine-tuning task option. The base model includes the fine-tuning functional model GPT-3.5-turbo, the chatbot program natural language model, the image generation system DALL·E, and the command-line interface CLI.

[0217] Step 203: Set fine-tuning parameters

[0218] The fine-tuning task selects appropriate parameters, including the number of fine-tuning steps, learning rate, and batch size.

[0219] Step 204: Set fine-tuning data

[0220] Upload the prepared dataset to the fine-tuning environment, format the dataset according to OpenAI's guidelines, and follow the data upload instructions; OpenAI provides a formatting tool with the following commands:

[0221] openai tools fine_tunes.prepare_data-f<LOCAL_FILE>

[0222] in,<LOCAL_FILE> The parameter is the dataset path.

[0223] Step 205 begins fine-tuning

[0224] To initiate the fine-tuning process, select "Start Fine-tuning with API" on the OpenAI console. This will trigger the fine-tuning process, and OpenAI will train the model using the provided dataset. The command to start fine-tuning is as follows:

[0225] openai api fine_tunes.create-t<TRAIN_FILE_ID_OR_PATH> -m<BASE_MODEL>

[0226] The BASE_MODEL parameter is the name of the base model.<TRAIN_FILE_ID_OR_PATH> : Specifies the identifier or path of the training file (training dataset).

[0227] Step 206: Monitor the fine-tuning progress.

[0228] To track the progress and performance of fine-tuning, the OpenAI console will provide the status and metrics of the fine-tuning task, including training steps and training loss, which can be adjusted as needed.

[0229] Step 207 uses the new fine-tuning model

[0230] After fine-tuning, select the already created model as input from the playground model in the OpenAI console, and check if the output matches the expected metrics. If the result is yes, directly call the prepared model via API.

[0231] In one embodiment, step 3, data acquisition, further includes the following steps:

[0232] Step 3-101: The SCADA system constructs an I / O driver layer, using an IGS gateway to integrate various protocols corresponding to PLCs and instruments, and directly interacts with the hardware. The I / O driver temporarily stores the data read back in the Driver Image Table (DIT).

[0233] Step 3-102 Configure and run the SAC service in the ifix task. The SAC scanning, alarm, and control services will handle the periodic scanning of data, alarm processing, and data control.

[0234] Step 3-103 configures the I / O drive address in the process database PDB data block, stores the data read from the DIT, and records the real-time final acquired digital / analog values ​​of all points; the historical database periodically collects data from the process database PDB and records history, serving as the data source supporting historical curves and historical reports. At the same time, the status data of the equipment (such as the status data of pump and valve equipment to be synchronized to the WEB service if necessary) is synchronized to the relational database through the historical database; the relational database synchronizes the historical database data to the SQL Server server through the data synchronization service AccessClient to obtain report data. Simultaneously, alarm information is also synchronized to the SQL Server server through the historical alarm service started by ifix.

[0235] Digital quantities are a data type representing discrete states, typically acquiring switch states or binary information. They have only two possible values, usually represented as 0 and 1 (or false and true, off and on, etc.). Digital quantities are commonly used to monitor the state of switches, buttons, and sensors; for example, detecting whether a device is on or off, or detecting whether an emergency stop button is pressed. The electrical signal data in the raw data acquisition is usually converted into digital quantities, which are used to represent the on / off state of the device.

[0236] Analog signals are a type of continuously changing data, typically acquiring values ​​within a continuous range. They can be continuously varying physical quantities such as voltage, current, temperature, and pressure. Analog signals are commonly used to monitor and control parameters that require continuous adjustment, such as monitoring temperature, flow rate, and pressure. The electrical signal data from the raw data acquisition is usually converted into analog signals, which are then used to represent continuously changing physical quantities.

[0237] Raw data acquisition refers to the raw electrical signal data directly obtained from sensors or devices. This raw signal data can be analog (continuously changing) or digital (discrete). SCADA systems typically need to convert this raw signal data into digital or analog quantities for easier processing and analysis, and for use in monitoring and controlling processes within the system.

[0238] Step 3-104: The SCADA engineering station includes the display and control of the configuration interface. The display and control of the configuration interface mainly includes graphical objects, controls, schedulers, and chart objects. Each station in the SCADA engineering station has an independent workstation. SCADA deployment adopts a distributed deployment architecture. A master station is defined on the upper layer of each substation, and the data of each substation can be directly monitored through the master station.

[0239] In one embodiment, step 3, where the SCADA system obtains the raw dataset and sends commands for data control, refers to controlling the device. The device control communicates and exchanges data with the device through an IoT Gateway to collect and transmit data, and to authenticate, encrypt, and control access to the device. The SCADA system converts the format of data from different devices, collects real-time data, monitors device status, performs remote operations, and integrates and collaborates with other systems.

[0240] The specific steps for the IoT Gateway plugin (IoT Internet of Things gateway) to work are as follows:

[0241] Step 3-201 Equipment Configuration

[0242] Configure device parameters in the IoT Gateway plugin, including the device's IP address, port number, and communication protocol;

[0243] Step 3-202 Data Acquisition

[0244] After the device is configured, the IoT Gateway plugin begins to collect data from the device through the OPC Server. The data includes the device's operating status and sensor readings.

[0245] Step 3-203 Data Processing

[0246] The IoT Gateway plugin collects raw binary data, converts the raw binary data and message data encapsulated through various protocols into target data in analog and digital form, and finally reports the collected target data values ​​through the TCP protocol.

[0247] Step 3-204 Data Transmission

[0248] The processed target data is transmitted to the SCADA system, which performs data analysis, equipment monitoring, and fault diagnosis.

[0249] Step 3-205 Command Sending

[0250] In addition to data acquisition, the IoT Gateway plugin also sends control commands to devices. It transmits digital start signals to the PLC's CPU via preset points and the S7 communication protocol. Upon receiving the signal, the PLC powers on the device. A point refers to a data acquisition point or a data input / output point, representing a specific monitoring or control data item used to indicate various parameters, states, or device interfaces that need to be monitored or controlled. The S7 communication protocol is a communication protocol customized by Siemens for its PLC series, used to transmit data and control commands between the computer and the PLC. In SCADA systems, operators use controls on the user interface to send control commands to preset points.

[0251] In SCADA (Supervisory Control and Data Acquisition) systems, a "point" refers to a data acquisition point or a data input / output point, representing a specific monitoring or control data item. The point is a core concept in SCADA systems, used to represent various parameters, states, or device interfaces that need to be monitored or controlled. In device control, a point is the output of a PLC control module, with a fixed and unique address. A value of 1 on this address bit indicates that the point is powered on, and 0 indicates that it is powered off, thus controlling the opening and closing of the device.

[0252] S7 communication protocol: This is a communication protocol customized by Siemens for its PLC series. It is used to transmit data and control commands between a computer and the PLC. In a SCADA system, the operator uses controls on the user interface to send control commands to preset points (i.e., which input / output terminal). These commands are sent to the PLC via the S7 communication protocol, and then the PLC executes the corresponding control logic to control the operation of the equipment, such as powering it on or off.

[0253] In one embodiment, the IoT Gateway plugin also integrates and works collaboratively with other systems. Through the access of IoT Gateway, the system realizes the interconnection of devices and the integration of data, and performs device management, remote monitoring, data analysis and optimization. It uses the unified HTTP REST and MQTT interface at the upper layer. The upper layer model integration and remote control all rely on the HTTP REST and MQTT standard interface.

[0254] When the IoT Gateway plugin provides an HTTP interface, and the upper-layer service integrates the IoT Gateway plugin, after selecting the network adapter, specifying the port and the routing URL, the IoT Gateway plugin will start a cluster search REST server according to the above-configured parameters. The upper-layer service can then directly call the API to obtain and send data.

[0255] Among them, data acquisition refers to the upper-layer service (i.e., the caller) obtaining the device's operating status, faults, and parameter data through the HTTP interface, and using it as the basic data input for the integration of the upper-layer model (such as judging whether the device is normal based on the collected device parameters and status); data delivery refers to sending the command data for controlling the device to start and stop to the IoTGateway service through the HTTP interface, thereby realizing the final remote control.

[0256] This invention provides a method for data control of industrial production processes using an AI-based SCADA system. It addresses the problems of poor user interactivity, inaccurate command issuance, cumbersome platform operation, insufficient intelligence, limited interaction, and poor model scalability in current AI-based SCADA systems. This invention collects and recognizes voice data from human-computer interaction, establishes a conversational interaction model based on a knowledge base fine-tuned using the GPT-3 model and sentiment analysis, and processes the raw electrical signal data from various types of equipment. The digital control signal is then sent to the equipment via preset points and the S7 communication protocol to enable or disable power. This improves the efficiency, accuracy, and safety of human-computer interaction, reduces labor costs, increases data processing efficiency and operational efficiency, and enhances system integration and scalability.

[0257] Example 1

[0258] The core idea of ​​an AI SCADA system for industrial process control is to introduce intelligent models such as speech recognition, voice interaction, Chinese word segmentation, sentiment analysis, natural language processing, and natural language models into traditional SCADA systems. Simultaneously, it integrates the system with the upper layer using an OPC+IoT Gateway Rest Server architecture, fully leveraging the application of artificial intelligence in industrial control to improve production process and operational efficiency. The technical roadmap is as follows:

[0259] 1. Voice interaction

[0260] This invention simplifies and enhances human-machine interaction in the entire control process by introducing voice interaction into SCADA. The system's voice interaction process involves three key technologies: voice acquisition and recognition, Chinese word segmentation, and command matching and command library maintenance. The main process is as follows: Figure 2 The overall flowchart of voice interaction is shown below:

[0261] Voice acquisition utilizes FFmpeg, an open-source computer program that can record, convert, and stream digital audio and video. Another consideration for adopting this technology in this invention is to facilitate future video integration and expansion. The system acquires voice data using FFmpeg and then uses the WAV format to call a speech recognition AI model for speech recognition. After recognizing text, it combines Hidden Markov Models (HMM), prefix dictionaries, dynamic programming, and other algorithms to perform Chinese word segmentation. After obtaining valid vocabulary, it uses full matching and the Levenshtein model to calculate similarity in a pre-built cardinality tree to match corresponding or similar instructions. Once the instruction is confirmed by the user, it is issued, and confirmed instructions are dynamically maintained in the instruction library. The entire process and results are interacted with by providing voice feedback to the user through the Speech SAPI. Detailed key technologies include voice acquisition and recognition, Chinese word segmentation, instruction matching, and instruction library maintenance, such as... Figure 2 The key technical steps mentioned are as follows:

[0262] 1) Voice Acquisition and Recognition

[0263] (1) Voice acquisition

[0264] Voice acquisition and recognition are primarily used to convert user-expressed commands into text content. The quality of this module's implementation directly impacts the success of the entire subsequent process. Figure 3 The voice acquisition flowchart shows the voice acquisition process as follows:

[0265] • Install ffmpeg: Taking Windows as an example, download the binary file from the official website, extract it, and add it to the system variable PATH. After installation, use fmpeg-version to verify whether the installation was successful.

[0266] • View the audio device list: Use the command ffmpeg-list_devices true-f dshow-i dummy to view the local microphone devices and select the capture device to use.

[0267] • Select audio device: Select the microphone device found in the second step and check whether its parameters meet the acquisition requirements, mainly including resolution, frame rate, pixel format and other parameters. It is necessary to ensure that it is consistent with the parameters supported by the final speech recognition. This invention uses an audio sampling rate of 16000, 16 channels and mono.

[0268] Start collecting

[0269] This invention uses ffmpeg in conjunction with a computer microphone for voice acquisition. After pre-setting key parameters such as sampling rate, number of channels, and sample size, the voice acquisition command is as follows:

[0270] ffmpeg -f dshow -i audio=@device_cm_{microphone device} -ar 16000 -ac 1 -sample_size 16 -channels 1 -t 00:00:05filename -y

[0271] The libavdevice library handles various devices and input / output formats. It provides access and control over audio devices for audio capture and recording. libavdevice supports multiple platform and device interfaces, such as the multimedia programming framework DirectShow (Windows platform) and the Advanced Linux Audio Architecture ALSA (Linux platform), to interact with different audio devices. libavcodec handles encoding and decoding. Once audio data is captured from the device, the libavcodec library can be used to encode (e.g., compress) and decode (e.g., decompress) the audio data. It supports various audio encoders and decoders, such as AAC, MP3, and lossless audio encoders like PCM (Pulse Code Modulation). The libavformat library handles the encapsulation and decapsulation of audio data.

[0272] • Voice Packet Generation: During the audio acquisition process, libavformat is used to encapsulate the acquired audio data into different audio container formats in real time, such as waveform audio files WAV and MP3, for convenient storage and processing. This invention adopts the WAV format, which is the closest to lossless music format. The standard formatted WAV file is the same as the CD format for optical disc data storage, with a sampling frequency of 44.1K and 16-bit quantization. Therefore, the sound file quality is almost the same as that of a CD, improving the accuracy of subsequent speech recognition.

[0273] (2) Speech recognition

[0274] The speech recognition employed the open-source automatic speech recognition system Deep Speech, using Recurrent Neural Networks (RNNs). This model first transforms the input audio into feature representations through several hidden layers, then models these features as a time series using recurrent layers, and finally transforms the features into a series of possible transcribed outputs through fully connected layers and a softmax function. Furthermore, the Connectionist Temporal Classification (CTC) loss function is utilized, allowing the network to be trained without detailed alignment information. The detailed process is as follows... Figure 4 The speech recognition model processing flowchart is shown below;

[0275] Preprocessing: The input audio signal typically requires some preprocessing steps to transform the raw audio data into a form more suitable for model processing. These steps include noise reduction, segmentation, sample rate conversion, and feature extraction. Feature extraction includes calculating Mel-frequency cepstral coefficients (MFCCs) or using methods such as filter banks.

[0276] Building the model: Construct an RNN model, which typically includes an input layer, one or more hidden layers, and an output layer. Hidden layers usually consist of Long Short-Term Memory (LSTM) units or Gated Recurrent Units (GRUs), which are RNN variants specifically designed for processing sequential data.

[0277] Training the model: The model is trained using labeled training data (i.e., audio files and their corresponding transcriptions). During training, the CTC loss function is used, which allows the network to learn the mapping from audio sequences to text sequences without requiring detailed alignment of the input data.

[0278] Decoding: During the model inference phase, the model's raw output is decoded to obtain the final text result. CTC's output is a probability distribution, providing the probability of all possible output characters for each time step. Typically, techniques such as greedy decoding or beam search are used to select the most probable output sequence from this probability distribution.

[0279] Evaluation and optimization: Finally, the model's performance is evaluated using the development and test sets, and the model's parameters or structure are adjusted as needed to improve its accuracy.

[0280] Here's an example of the whole process: Suppose we have a simple audio clip that only contains one person saying "Hello, world!".

[0281] • Preprocessing: We first preprocess the audio clip using a process called Mel-spectrum analysis. Mel-spectrum analysis extracts features from the audio signal and transforms them into data that can be used for deep learning. This produces a matrix of values, where each value represents the energy of the sound at different frequencies and times. Now, our audio clip has been transformed into a form that the model can understand.

[0282] Model Building: Next, we input this Mel spectrum into a neural network consisting of several hidden layers. These layers transform the Mel spectrum into a deeper representation. We typically use one or more recurrent neural network layers to process these sequential data to capture the temporal properties of the sound. Finally, we use a fully connected layer and a softmax function to transform the deep representation into a series of possible outputs. In this example, the possible outputs are all the characters of the English alphabet, including spaces.

[0283] • Model Training: We use a loss function called Connectionist Temporal Classification (CTC) to train our model. The CTC loss function allows the model to be trained without knowing the exact alignment information between the audio and the text. We can provide an audio clip and its transcription, for example, the audio of "Hello, world!" and the text "HELLO WORLD", and the model will try to adjust its internal parameters to make its output for a given audio clip as close as possible to the given transcription.

[0284] CTC stands for Connectionist temporal classification, which can be understood as temporal classification based on neural networks. Training the acoustic model in speech recognition is supervised learning, requiring knowledge of the label for each frame for effective training. Forced alignment of the speech must be performed during the data preparation phase. While it's difficult to assign a label to a single frame of speech data, it becomes easier to determine the corresponding pronunciation label from dozens of frames.

[0285] • Decoding: After the model is trained and can generate output, we need to transform this output into meaningful text. For this, we need a decoder. In this example, the model might generate a series of character probabilities for the audio "Hello, world!". The decoder's task is to find the most probable character sequence based on these probabilities. For example, the decoder might receive the model's probabilities for "H", "E", "L", "L", "O", "" (space), "W", "O", "R", "L", "D" and transform them into the final output "HELLO WORLD".

[0286] Evaluation and Optimization: Next, we compare the model's output with the actual transcription. A commonly used evaluation metric is the Word Error Rate (WER). In this example, if the model's output is "HELLOWORLD", then the WER is 0 because the model's output is completely consistent with the actual transcription. If the model's output is "HELLOWRLD", then the WER is 0.5 because the model missed a word. Based on this evaluation metric, we can optimize the model's performance, such as adjusting the model's parameters, adding more training data, or changing the training strategy, etc., to obtain the final recognition result.

[0287] (3) Chinese word segmentation

[0288] The Chinese word segmentation uses the open-source word segmentation model "Jieba" to categorize user input commands, further improving the accuracy of the final remote control device. This model combines various technologies and algorithms to achieve the word segmentation function, such as... Figure 5 The Chinese word segmentation process is shown in the diagram. The main steps include the following:

[0289] • Constructing a prefix dictionary: Based on the provided corpus, Jieba segmentation constructs a prefix dictionary, storing the prefixes of each word in the dictionary. Taking the field process text as an example, if "detention pool" is a word, then both "detention" and "detention pool" will be stored as prefixes in the dictionary.

[0290] ·Word segmentation using dynamic programming: Jieba uses a dynamic programming algorithm based on a prefix dictionary to find the optimal word segmentation method. Specifically, for a sentence to be segmented, Jieba tries to find a segmentation method that maximizes the sum of the frequencies of the corresponding word sequence.

[0291] ·Handling out-of-vocabulary words using the Hidden Markov Model: For words not in the prefix dictionary, Jieba uses an out-of-vocabulary word recognition technique based on the Hidden Markov Model. This model infers out-of-vocabulary words based on the transition probabilities between characters.

[0292] ·Adding a user-defined dictionary: Jieba also allows users to add a custom dictionary, which can override the default dictionary and handle some special word segmentation requirements.

[0293] The following is a specific example:

[0294] Suppose we have a sentence "Open the regulating pond fan", and we want to segment this sentence.

[0295] ① Jieba first uses the prefix dictionary to find all possible segmentation methods. For example, "Open|Regulating pond|Fan", "Open|Regulating|Pool|Fan", etc.

[0296] ② Then, Jieba uses a dynamic programming algorithm based on the prefix dictionary to find the optimal segmentation method. Assuming that in the corpus, the sum of the frequencies of "Open", "Regulating pond" and "Fan" is the largest, then the optimal segmentation method is "Open|Regulating pond|Fan".

[0297] ③ For out-of-vocabulary words, such as "Regulating pond" if it is not in the dictionary, then Jieba will use the Hidden Markov Model to handle it. Based on the transition probabilities between characters, Jieba may infer that "Regulating pond" is a word.

[0298] ④ Finally, if the user adds a custom dictionary and defines "Regulating pond" as a word in the dictionary, then Jieba will segment the words according to the user's dictionary, and the final segmentation result may be "Open|Regulating pond|Fan".

[0299] The following key technologies are mainly used:

[0300] Implementing efficient word graph scanning based on a prefix dictionary: Using a prefix dictionary, that is, as long as the prefix of a word is in the dictionary, then this word can be matched. This can efficiently generate a directed acyclic graph (DAG) of the sentence to be segmented.

[0301] Dynamic programming is used to find the maximum probability path: Dynamic programming is performed on the word graph to find the maximum probability path for segmenting sentences. This ensures that the output word segmentation result is the optimal segmentation based on word frequency.

[0302] Out-of-vocabulary word recognition based on HMM model: For words not found in the dictionary, an HMM (Hidden Markov Model) based on the word-forming ability of Chinese characters is used for recognition. Specifically, it uses four labels (B: word beginning, M: word middle, E: word ending, S: single character word) and is trained using existing manually annotated corpora. The basic formula of HMM mainly includes three parts:

[0303] State transition probability matrix A:

[0304] a ij =P(i t+1 =q j |i t =q i )

[0305] This indicates that the state is q at time t. i Under the condition that the state transitions to state q at time t+1 j The probability of;

[0306] II. Observation probability matrix B:

[0307] b jk =P(o t =v k |i t =q i )

[0308] This indicates that the state is q at time t. j Generate observations v under the condition of k The probability of;

[0309] III. Initial state probability vector π:

[0310] π i =P(i1=q i )

[0311] This indicates that the state is q at time t=1. i The probability of;

[0312] (4) Instruction matching and instruction library maintenance

[0313] After obtaining valid instruction phrases through speech recognition and Chinese word segmentation, the system begins matching instructions within the dictionary. During application, the system automatically and continuously optimizes and improves the instruction library and refines the process model library, such as... Figure 6 As shown in the flowchart for instruction matching and maintenance, the main process is as follows:

[0314] · Lexical database loading: Although Jieba has the ability to recognize new words, adding new words manually can ensure higher accuracy. For specific on-site process entities and common vocabulary, add them to the word segmentation dictionary, separated by \t. The dictionary format is the same as dict.txt, with one word per line; each line is divided into three parts: word, word frequency (optional), and word nature (optional), separated by spaces, and the order cannot be reversed. After filling in the word tree, specify the custom dictionary path in the code for loading. At the same time, use add_word(word, freq=None, tag=None) and del_word(word) to dynamically modify the dictionary in the program. This is a supplement to the "add custom dictionary" function. Use suggest_freq(segment, tune=True) to adjust the word frequency of a single word so that it can (or cannot) be distinguished.

[0315] · Lexical database loading: Use jieba.load_userdict to add the custom lexicon to the model.

[0316] · Instruction matching: This step mainly targets the vocabulary after speech word segmentation and matches it in the pre-maintained instruction library. For example, if the word segmentation contains "regulation pond", the system will search layer by layer in the process entity lexicon, equipment lexicon, and function lexicon. After matching "regulation pond" in the process entity lexicon, it will further match the equipment, and so on; in this step, in order to improve the accuracy of matching, three modes of exact matching, pinyin matching, and similarity calculation matching are adopted respectively.

[0317] · Instruction correction: After the instruction matching is completed, due to inaccurate pronunciation or deviation in similarity calculation, the instruction may not be what the user actually wants to execute. Therefore, this step provides the user with the ability to further modify the recognized instruction. For example, due to accent or dialect problems, "regulation pond" is recognized as "bridge sequence pond", and the user can modify and correct it. The corrected instruction mapping will be further maintained in the system lexicon to improve the subsequent recognition accuracy.

[0318] · Instruction library maintenance: After word segmentation recognition, instruction matching, and instruction correction, the following 4 situations will occur:

[0319] ① Exact matching is successful, execute directly;

[0320] ② Pinyin matching or similarity calculation matching is successful and the user has not corrected, execute directly;

[0321] ③ Execute after the user corrects after pinyin matching or similarity calculation matching;

[0322] ④ The deviation after matching is too large and no match is found, and the user has not corrected, give up execution;

[0323] For cases 1 and 4, this step does not perform any processing. For cases 2 and 3, a mapping relationship is established between the matched words and the corrected words, and this relationship is maintained in the instruction vocabulary database.

[0324] • Action execution: After matching the specific action and device, the system sends an HTTP request to the SCADA server. Upon receiving the request, the SCADA system executes the on-site action of the device.

[0325] The key technical points mentioned in the above steps are as follows:

[0326] For systems in different professional fields, a basic instruction dictionary is pre-maintained based on process equipment and operational habits. At the same time, common error-prone instructions are handled separately. When the system starts up, a radix tree is built to improve search efficiency. During the matching phase, full matching, pinyin matching, and similarity calculation are used to improve fault tolerance and matching accuracy.

[0327] ①Similarity calculation uses Levenshtein edit distance:

[0328]

[0329] Once a precise command is matched and the user confirms its issuance, if the command term does not appear in the dictionary, it will be automatically added to the dictionary through dynamic learning to improve the accuracy of subsequent command matching.

[0330] ② To further improve the accuracy of command matching, this invention additionally employs pinyin matching to further compensate for the accuracy of speech recognition. The system mainly uses the "go-pinyin" library to implement pinyin conversion, such as... Figure 7 The key steps of the Pinyin matching instruction are shown in the diagram; the main effects are as follows: (1) Improve the efficiency and accuracy of human-computer interaction: By introducing voice interaction technology, manual operation is reduced. The complexity of operation is reduced, and a more natural and intuitive interaction method is provided, thereby improving the interaction efficiency and accuracy of the entire system; (2) Improve operational production efficiency: By introducing professional models such as sentiment analysis, natural language processing, and knowledge base, the behavior of operators is analyzed and understood in real time, thereby providing more accurate feedback and suggestions, further improving operational production efficiency and operational safety.

[0331] 2. Model Integration Processing

[0332] Within a framework supporting voice interaction, this invention integrates various intelligent models to enhance the overall flexibility of the system and support operators in better operation and management. In terms of model access, a unified layering system allows for hot-swapping of models, improving the overall system's scalability. The system currently primarily incorporates technological innovations in three main areas: knowledge base, sentiment analysis, and intelligent report generation. Technically, it integrates open-source AI, and uses "fine-tuning" to perform specialized training on basic models to meet specific application scenarios. OpenAI provides a series of AI models and tools, the most famous being the GPT (Generative Pre-trained Transformer) model series, including the latest GPT-3 model. These models can be used for various tasks such as natural language processing, dialogue systems, semantic understanding, and text generation. Figure 8 The flowchart for integrating Open AI artificial intelligence models is shown below.

[0333] The model fine-tuning steps are as follows:

[0334] • Dataset Preparation: Includes input text and the corresponding target output. Ensure the dataset complies with OpenAI's usage policies and relevant laws and regulations. OpenAI is an artificial intelligence research company founded in the United States. API stands for Application Programming Interface. APIs are predefined interfaces (such as functions or HTTP interfaces) or refer to the conventions for connecting different components of a software system.

[0335] • Create a fine-tuning environment: Create a fine-tuning environment on the OpenAI console. Select a suitable base model (e.g., GPT-3.5-turbo), and then select the fine-tuning options.

[0336] • Set fine-tuning parameters: Choose appropriate parameters for the fine-tuning task. These parameters include the number of fine-tuning steps (usually hundreds or thousands of steps), the learning rate (which controls the update speed of model parameters), the batch size (the number of samples used in each mini-batch for training), etc.

[0337] • Set up fine-tuning data: Upload the prepared dataset to the fine-tuning environment. Ensure that the dataset is formatted according to OpenAI's guidelines and that the data upload instructions are followed.

[0338] • Start Fine-Tuning: To initiate the fine-tuning process, select Start Fine-Tuning on the OpenAI console. This will trigger the fine-tuning process, and OpenAI will train the model using the provided dataset.

[0339] • Monitor fine-tuning progress: Track the progress and performance of fine-tuning. The OpenAI console provides the status and metrics of the fine-tuning task, such as training steps and training loss. You can make appropriate adjustments to the fine-tuning as needed.

[0340] • Use the new fine-tuned model: After fine-tuning, select the created model from the playground models in the OpenAI console, try inputting it, and observe whether the output matches the expectations. Afterward, you can directly call the model via the API.

[0341] The specific applications of this invention are as follows:

[0342] 1) Knowledge base fine-tuned based on GPT-3.5-turbo model

[0343] GPT-3.5-turbo: GPT-3.5-turbo uses the Transformer architecture, a neural network architecture based on a self-attention mechanism. The Transformer architecture performs exceptionally well when processing long text sequences, capturing the context and semantic relationships within the text. In this invention, pre-collected maintenance, fault, and operational information, including common problems, expert diagnostics, and emergency plans, is input and trained to effectively support the operational process. Furthermore, through extensive learning from internet knowledge, it can provide effective parameter references for certain process flows. Additionally, after specialized training based on sentiment analysis, the model can determine whether user-issued commands contain negative emotions or aggressive tendencies, thus enabling it to respond accordingly, rather than relying on traditional command pass-through.

[0344] For example, you can ask questions about operational or professional issues in the production process, such as "What is the normal outlet pressure of a 30KW suspended fan?" The model call should use the Golang language and the go-openai (v1.7.0) library. Send an HTTP request to Openai with the key obtained from the Openai platform and the question content (all model calls below will use this format).

[0345] The model response states, "Under normal circumstances, the outlet pressure of a 30kW suspended fan will vary depending on the specific design and adjustments. Generally, its outlet pressure should be within the range of 1500 Pa to 4000 Pa. However, the specific outlet pressure will also be affected by factors such as the fan's operating status, installation height, and pipe resistance. In application, adjustments and optimizations may be necessary based on the actual situation. To accurately determine the outlet pressure of the suspended fan, it is recommended to refer to the specific fan model and relevant technical data, or to have a professional conduct on-site testing and evaluation." Operators can refer to this response and the actual situation of the equipment on-site to determine whether the equipment is functioning normally or to set certain threshold monitoring parameters.

[0346] Furthermore, the knowledge base can be used to determine the compliance of data collected by SCADA. If the data is found to be non-compliant, it can provide possible causes and corrective measures, thus enabling "intelligent early warning" and "intelligent diagnosis." This greatly simplifies the operational complexity for operators and improves operational efficiency. For example, "Is a methane concentration of 4.5 in the storage tank normal? What measures should be taken?" After calling the OpenAI model interface, professional suggestions will be provided.

[0347] 2) Sentiment analysis based on fine-tuning of the GPT-3.5-turbo model

[0348] Natural Language Models (NLPs): NLPs are models suitable for conversational interactions, used to generate reports and answer related questions. They excel at generating natural language dialogues. This invention uses data collected by SCADA, combined with a pre-provided prompt template, to enable the model to automatically generate and summarize reports. Furthermore, it can provide professional and constructive suggestions based on the generated reports and historical data.

[0349] For example, when an operator, in a highly emotional and negative tone, tells SCADA via voice interaction, "Turn on this lousy fan immediately! Ignore any warnings," the system uses sentiment analysis to determine whether to proceed. The code calls the model and responds, "Based on the provided utterances, we can determine that the speaker's tone is negative," thus preventing execution and avoiding extreme actions caused by personal emotional instability. Furthermore, in such situations, the system can trigger an alarm and notify management, simultaneously locking SCADA and temporarily halting all operations to prevent human error and damage.

[0350] 3) Automatic report generation based on fine-tuning of natural language models and the Dall·E model

[0351] DALL·E: DALL·E is a model for image generation that can transform text descriptions into corresponding images. While it's not directly used for report generation, it can be used to generate relevant image materials when report-related image support is needed, making reports more diverse and easier for managers to understand. Examples include generating various statistical charts based on data, automatically generating images reflecting on-site issues based on report descriptions, and the model also supports editing existing images with descriptions, greatly increasing productivity.

[0352] For example, when inspectors need to write a report after completing an on-site inspection, but forget to take photos and need some materials, such as "a bunch of white steel pipes", the model will generate some selectable images based on the prompts. The model will generate corresponding images based on the prompts and parameters, and can generate multiple images at the same time for the user to choose from. In this example, it is configured to generate 3 images with a resolution of 1024*1024.

[0353] In addition to these scenario diagrams, data from the production process can be provided to generate statistical charts to support reporting materials. Based on this, and combined with data from SCADA, the model can automatically analyze whether various indicators are normal and provide rectification suggestions, including knowledge base support, which greatly facilitates the writing of reporting materials.

[0354] 3. Data Acquisition and Control

[0355] This invention uses Ifix+IGS+PLC to achieve data communication between the software platform and the hardware platform, as well as downward control of the equipment.

[0356] iFix is ​​an automation hardware and software solution provided by General Electric Company of the United States. It is the world's leading HMI / SCADA automation monitoring and configuration software.

[0357] IGS (Industrial Gateway Server) is a server software used to connect different industrial control and monitoring devices. It is typically used in conjunction with HMI (Human Machine Interface) software (such as iFIX) to enable data communication, device control, and monitoring.

[0358] The Chinese meaning of PLC (Programmable Logic Controller) is: Programmable Logic Controller.

[0359] 1) Data Acquisition

[0360] The overall data flow architecture of the system is as follows: Figure 9 The data flow architecture diagram is shown.

[0361] • Process Hardware: Hardware used in the production process; the actual equipment used in the production process.

[0362] • IO / Driver: I / O driver layer. The IGS in the project belongs to this layer. This invention uses an IGS gateway to integrate various PLCs, instruments and other protocols, and interact directly with the hardware.

[0363] • DIT: Driver image table, where the I / O driver temporarily stores the data it reads back in this memory block;

[0364] • SAC: Responsible for periodic scanning of data, alarm processing and data control. Therefore, the SAC service needs to be configured and run in the ifix task (for example, AR and DR blocks in the process database do not go through the alarm service, but directly perform alarms at this layer. This layer is a memory operation, which is highly efficient).

[0365] • PDB: ifix process database. I / O drive addresses are configured in the PDB data block, data read from DIT is stored, and real-time values ​​of all points are recorded.

[0366] Historian: A historical database that periodically collects data from the PDB and records history, supporting data sources such as historical curves and reports. It also serves as a data transfer hub in projects, synchronizing some data to relational databases through the historical database.

[0367] SQL Server: A relational database. It uses a self-developed service to synchronize historical database data to SQL Server to support functions such as reporting. At the same time, some alarm information is also synchronized to SQL Server through the historical alarm service started by ifix. The historical alarms and operation logs currently used in the project all use this as the data source.

[0368] • Scada Workspace: The Scada engineer station is responsible for the display and control of the configuration interface. It mainly includes graphical objects, controls, schedulers, chart objects, VBA, etc. Each station in the project has an independent workstation, while Jinzhai is the main station, capable of operating all substations. Figure 9 The data flow is shown in the configuration screen, which only shows the Huiyuan and Raocheng substations and the Jinzhai main station. Other substations are not shown, but the data topology is consistent with it.

[0369] ·MySQL: A relational database used in the SCADA system to store various device data synchronized from the historical database and to distribute the data to different tables through various triggers to support business operations;

[0370] Platform: A comprehensive monitoring platform that calculates, analyzes, and ultimately presents all data.

[0371] 2) Equipment control

[0372] In terms of equipment control, this invention introduces an IoT Gateway based on IGS. The main function of the IoT Gateway is to enable communication and data transmission between industrial equipment and IoT devices. It can connect various sensors, actuators, controllers, and other IoT devices, and provide bidirectional communication with these devices. Through the IoT Gateway, the system can collect real-time data, monitor equipment status, perform remote operations, and integrate and collaborate with other systems.

[0373] The IoT Gateway primarily plays the following roles in this invention:

[0374] Connectivity and Communication: The IoT Gateway provides the ability to connect different IoT devices and communicates with them through various communication protocols (such as MQTT, OPC UA, Modbus, etc.) to collect and transmit data.

[0375] Data Acquisition and Processing: The IoT Gateway can collect real-time data from connected devices and process and parse the data. It can filter, aggregate, transform, and verify the data to ensure its accuracy and integrity.

[0376] Security and Authentication: The IoT Gateway provides security features, including device authentication, data encryption, and access control, to ensure the security and confidentiality of IoT connections.

[0377] Protocol conversion and integration: The IoT Gateway has the capability of protocol conversion, which can convert the format of data from different devices, enabling it to be integrated and interact with other systems.

[0378] Local processing and decision-making: The IoT Gateway can perform local data processing and decision-making, reducing communication latency with the cloud and supporting real-time response and control. Figure 10 A schematic diagram of IoT Gateway operation; the workflow of the IoT Gateway plugin is as follows:

[0379] • Device Configuration: First, you need to configure the device parameters in the IoT Gateway plugin. This includes the device's IP address, port number, routing address, etc.

[0380] • Data Acquisition: After configuring the device, the IoT Gateway plugin can begin collecting data from the device. This data may include the device's operating status and sensor readings.

[0381] • Data processing: The IoT Gateway plugin converts the collected raw data into a format that the SCADA system can understand, such as OPC UA, Modbus, etc., and finally transmits the digital / analog quantities through these protocols.

[0382] • Data transmission: The processed data is then sent to the SCADA system. This allows the SCADA system to perform data analysis, equipment monitoring, and fault diagnosis.

[0383] • Command Sending: In addition to data acquisition, the IoT Gateway plugin can also send control commands to devices. For example, a SCADA system can send a command through the IoT Gateway plugin to start a water pump. The start signal digital quantity is sent to the PLC's CPU through a preset point and the S7 protocol. After receiving the signal, the PLC sets the output terminal to 1, and the device is powered on.

[0384] Through the IoT Gateway, the system can achieve device interconnection and data integration, providing infrastructure and functional support for industrial automation and IoT applications. It facilitates industrial applications such as equipment management, remote monitoring, data analysis, and optimization. More importantly, it unifies the HTTP REST and MQTT interfaces for upper layers, which are essential for functions such as model integration and remote control.

[0385] Through real-time monitoring and intelligent analysis, coupled with a comprehensive knowledge base, the system can promptly provide operators with effective suggestions for corrective actions when problems are detected, such as equipment malfunctions or abnormal operations, preventing the spread of accidents and improving system security. With the continuous development of IoT technology, more new devices and protocols may emerge in the future. This invention has considered this scalability, allowing for easy integration with new devices and protocols, thus ensuring the system's long-term competitiveness. This system improves energy efficiency and reduces energy consumption to a certain extent by optimizing operations, enhancing production accuracy, and facilitating operation. For example, in the power sector, the system optimizes the operation of power generation and distribution, reducing carbon emissions; in the field of water environmental protection, it analyzes water quality data in a timely manner and combines it with historical data to provide constructive suggestions on key drug dosages and equipment operating times, promoting sustainable development.

[0386] This invention provides a method for data control of industrial production processes using an AI-based SCADA system. It addresses current AI-based SCADA systems by addressing issues such as poor user interactivity, inaccurate command issuance, cumbersome platform operation, insufficient intelligence, limited interaction, and limited model scalability. This invention collects and recognizes voice data from human-computer interaction, establishes a conversational interaction model based on a knowledge base fine-tuned using the GPT-3 model and sentiment analysis, and processes the raw electrical signal data from various types of equipment. The digital control signal is then sent to the equipment via pre-defined points and the S7 communication protocol to enable or disable power. By introducing an IoT Gateway on top of the OPC Server, this invention unifies external protocols and standards, improving system scalability and integration. Combined with a mature IGS gateway, this invention enhances the system's data processing efficiency. This is particularly important for fields requiring large amounts of data and high real-time performance, as they need to process and analyze data quickly and accurately to make timely and correct decisions, thus improving data processing efficiency. Through AI-automated control and decision-making, human intervention can be significantly reduced, thereby lowering labor costs. In special environments, such as dangerous or remote areas, AI’s automated control and decision-making can ensure continuous and stable operation and reduce personnel’s risk exposure; at the same time, automatically generated reports also greatly reduce the workload of operations personnel to a certain extent.

Claims

1. A method for data control of industrial production processes using an AI-based SCADA system, characterized in that, The method includes the following steps: Step 1. Voice Interaction: Collect and recognize voice data from human-computer interaction. Encapsulate the collected audio data into an audio container format to generate a voice packet. Preprocess the voice packet using the Mel-frequency cepstral coefficient (MFCC) method. Transform the features into a sequence transcription output using a recurrent neural network combined with the CTC loss function to obtain the recognized text. Perform Chinese word segmentation using Jieba to obtain a valid instruction vocabulary. The Chinese word segmentation process includes constructing a prefix dictionary, word segmentation based on dynamic programming, and processing out-of-vocabulary words using an Hidden Markov Model (HMM). Then, perform instruction matching in the vocabulary. The instruction matching adopts three modes: full matching, pinyin matching, and similarity calculation matching. If there is a deviation in the matching result, the user is provided with instruction correction. The system establishes a mapping relationship between the matched words and the corrected words and automatically maintains it in the instruction vocabulary. Step 2. Model Integration Processing: Based on the GPT-3 model, fine-tune the industrial knowledge base and combine it with sentiment analysis to establish a conversational interaction model. The fine-tuning is based on the input text and the corresponding target dataset output. The dataset includes a process vocabulary and an equipment vocabulary. The conversational interaction model judges the emotional tendency of the user's command text based on the sentiment analysis model. If the emotion is negative, it will block or issue an alarm. Step 3. Data Acquisition and Control: The SCADA system acquires raw electrical signal data from various types of equipment to form a raw dataset. It then converts the raw electrical signal data into digital or analog quantities to form target data. The target data is transmitted via Siemens' S7 communication protocol or Modbus protocol. The SCADA system receives the target data and sends commands to perform data control. It sends the digital quantities of the equipment control signals to the equipment through preset points and the S7 communication protocol to enable or disable power-on or power-off.

2. The method for industrial production process data control based on AI in a SCADA system according to claim 1, characterized in that, In step 1, the collected audio data is encapsulated into different audio container formats to generate speech packets. The speech packets are then transformed into feature representations through hidden layers. Time-series modeling of these features is performed using recursive layers, and finally, a fully connected layer and a Softmax normalization function are used to convert the features into a transcribed output sequence. A loss function is used to train the network without detailed alignment information. The audio-to-feature representation transformation involves preprocessing the collected audio data. This preprocessing divides the audio data into small time segments to form frames, and the audio signal within each frame is converted into spectral information. Short-time Fourier transform is used for spectral analysis, converting the time-domain waveform signal into a frequency-domain representation to obtain the spectrogram of each frame. The data in the spectrogram is then used to construct the feature representation. One type of recurrent neural network is used to process sequential data. In speech recognition, it is used to model audio signals using recurrent neural networks (RNNs) to establish a sequence model of audio signals and their time dependencies. Feature-to-sequence transcription output refers to converting the results of feature extraction and RNN sequence modeling into text transcription output corresponding to the input audio through a fully connected layer and a Softmax normalization function; in speech recognition tasks, it maps audio data to corresponding text transcription. The loss function is represented by the CTC loss function; the CTC loss function is used to train the model for sequence transcription without detailed alignment information; the goal of the CTC loss is to maximize the probability that the model generates the correct output sequence for a given input sequence. Where X represents the input sequence and Y represents the target sequence; L(X,Y) is the CTC loss corresponding to the input sequence X and the target sequence Y; It is the set of all corresponding paths to the output label sequence Y; π is a path that represents the correspondence from the input sequence to the output label sequence; P(π|X) is the conditional probability of path π given the input sequence X.

3. The method for industrial production process data control based on AI in a SCADA system according to claim 2, characterized in that, The method of using a loss function to train the network without detailed alignment information includes the following steps: Step 1-101 Preprocessing The Mel-frequency cepstral coefficient (MFCC) method is used to convert the raw audio data into training data suitable for RNN model processing, taking the input speech packet as the audio signal. The processing includes noise reduction, segmentation, sampling rate conversion, and feature extraction. The MFCC method utilizes human auditory characteristics, first mapping the linear spectrum to the Mel-frequency nonlinear spectrum based on auditory perception, and then converting it to the cepstral spectrum. The formula for converting ordinary frequencies to Mel frequencies is as follows: mel(f) represents the Mel frequency, which is the Mel scale value corresponding to the linear frequency f and is used to simulate the human ear's perception of frequency. Step 1-102: Construct a Recurrent Neural Network (RNN) model A Recurrent Neural Network (RNN) model for processing sequential data is constructed using an RNN. The RNN model includes an input layer, one or more hidden layers, and an output layer. The hidden layers can be either Long Short-Term Memory (LSTM) units or Gated Recurrent Units (GRUs). By passing information through the recurrent structure of the RNN, the temporal relationships in the sequential data are obtained. The mathematical expression of the RNN model is as follows: In an RNN unit, assuming the input is at time synchronization t, The output in the hidden state is The update process of RNN is as follows: in: It is the output of the hidden state of time synchronization t; It is the input for time synchronization t; It is the weight matrix input to the hidden state: It is the weight matrix from the hidden state of the previous time step to the hidden state of the current time step; It is the bias vector of the hidden state; It is an activation function; Step 1-103: Train the model The model is trained using labeled audio files and corresponding transcribed training data; during training, the CTC loss function is used to allow the network to learn the mapping from audio sequences to text sequences even when the input training data is misaligned. Decoding Steps 1-104 The raw output of the trained model is decoded to obtain the final text result; the output of the CTC loss function is a probability distribution, which gives the probability of all output characters for each time step; the output sequence is selected from the probability distribution using beam search; Steps 1-105: Evaluation and Optimization The model's performance is evaluated using pre-prepared training, development, and test sets. Model parameters or structure are adjusted as needed. The development and test sets are derived from the partitioning and processing of the original dataset. The dataset is divided proportionally: 80% training set, 10% development set, and 10% test set. During actual evaluation, the original development and test datasets are input into the RNN model, and the RNN model's prediction results are recorded and compared with preset metrics to measure device performance. These expected metrics are empirical values ​​of device performance in real-world scenarios. The comparison results determine whether further optimization is needed.

4. The method for industrial production process data control based on AI in a SCADA system according to claim 1, characterized in that, Step 1, which involves using Jieba for Chinese word segmentation, also includes the following steps: Step 1-201: Construct a prefix dictionary Based on the provided default dictionary library, Jieba segmentation will build a prefix dictionary, and the prefix of each word will be stored in the dictionary; Step 1-202 Dynamic Programming Word Segmentation Jieba segmentation uses a dynamic programming algorithm based on a prefix dictionary to find the optimal word segmentation method. For a sentence to be segmented, Jieba segmentation will try to find a segmentation method that maximizes the sum of the frequencies of the word sequences corresponding to this segmentation method. Step 1-203: Processing Unregistered Keywords Hidden Markov Model (HMM) is used to process out-of-vocabulary (OV) words. For words that are not in the prefix dictionary, Jieba segmentation uses an OV word recognition technique based on the Hidden Markov Model. This model infers OV words based on the transition probabilities between characters. Hidden Markov Models (HMMs) are models that use four labels and are trained using existing manually labeled word data. The formula for an HMM mainly consists of three parts: 1) State transition probability matrix A: This indicates the state at time t. Under the condition that the state transitions to state at time t+1 The probability of; 2) Observation probability matrix B: This indicates the state at time t. Generate observations under the conditions The probability of; 3) Initial state probability vector π: This indicates that the state is at time t=1. The probability of; The four tags include B: word beginning, M: word middle, E: word ending, and S: single word; Step 1-204: Add a custom dictionary Adding a user-defined dictionary: Jieba segmentation allows users to add custom dictionaries, overriding the default dictionary and handling special segmentation needs to obtain valid instruction phrases; obtaining valid instruction phrases refers to one or more instruction words that can be matched in the existing vocabulary.

5. The method for industrial production process data control based on AI in a SCADA system according to claim 1, characterized in that, Step 1, which involves performing command matching in the dictionary, employs three modes: full matching, pinyin matching, and similarity calculation matching. If the matching results are inaccurate, the system provides users with the option to correct the command. The system establishes a mapping relationship between the matched words and the corrected words and automatically maintains this mapping in the command vocabulary. The steps also include the following: Step 1-301: Loading the dictionary Add words to the word segmentation dictionary, separated by \t. The dictionary format is the same as dict.txt, with one word per line. Each line consists of three parts: word, word frequency, and part of speech, separated by spaces. The order cannot be reversed and the words are added to the word tree. Load the dictionary at the specified custom dictionary path. The dictionary can be dynamically modified in the program using add_word and del_word. The word frequency of a single word can be adjusted using suggest_freq. The custom dictionary is added to the model using jieba.load_userdict. Step 1-302 Command Matching For matching words after speech word segmentation in the instruction library, the SCADA system searches for target words layer by layer from the process word library, equipment word library, and function word library of the instruction vocabulary library. After matching the target word in the process word library, it further matches the equipment, and so on. Three modes are used respectively: full matching, pinyin matching, and similarity calculation matching. Step 1-303 Instruction Correction After the instruction matching is completed, if the pronunciation is inaccurate or the similarity calculation is off, or if the instruction is not what the user actually wants to execute, the user can further modify the recognized instruction. The user will then make corrections and the corrected instruction mapping will be further maintained in the SCADA system instruction vocabulary library. Step 1-304 Instruction Dictionary Maintenance After word segmentation and recognition, instruction matching, and instruction correction are completed, the following four situations may occur: a) If all matches are successful, proceed directly; b. If the pinyin matching or similarity calculation is successful and the user has not corrected the settings, proceed directly. c. Execute after user correction following pinyin matching or similarity calculation matching; d. If the deviation after matching is too large and no match is found, and the user does not correct it, the execution is abandoned; For cases a and d, steps 1-304 are not processed. For cases b and c, a mapping relationship is established between the matched words and the corrected words, and this relationship is maintained in the instruction vocabulary. Step 1-305 Action Execution After matching the specific actions and devices, the system sends an HTTP request to the SCADA server, and the SCADA system executes the on-site actions upon receiving the request.

6. The method for industrial production process data control based on AI in a SCADA system according to claim 5, characterized in that, In step 1-302, the pinyin matching and similarity calculation in instruction matching also use pinyin matching to further improve the accuracy of speech recognition. The go-pinyin library is used to realize the pinyin conversion. After dictionary mapping, tone processing, polyphonic character processing and non-Chinese character processing are performed. Similarity calculation uses the Levenshtein edit distance algorithm, where the Levenshtein distance between two strings A and B is defined as: Where i and j represent the index positions of strings A and B respectively, and a and b are the lengths of strings A and B respectively; Once a matching instruction is received and the user confirms its issuance, if the vocabulary for that instruction does not appear in the dictionary, it will be automatically added to the dictionary.

7. The method for industrial production process data control based on AI in a SCADA system according to claim 1, characterized in that, Step 2, the fine-tuning based on the input text and the corresponding target dataset output, includes the following steps: Step 201 Dataset Preparation The input text and the corresponding target dataset output comply with OpenAI's usage policies and relevant laws and regulations; the data is a JSON document, where each line is a suggestion completion pair; The prompt-complete pair refers to the format of the training data used to fine-tune OpenAI's models. When fine-tuning the model, a dataset containing prompts and corresponding target outputs is provided. The prompt-complete pair is used to prompt the model to complete or complete a task given the prompt. A prompt-complete pair consists of the following two main parts: Prompt: This refers to an input text, which can be a sentence, question, or task description used to guide the model in generating relevant output; a prompt is a description of the model's answer to an input sentence or question, or the task it completes. Target Output: This is the text or answer generated by the expected model in association with the prompt; it is the correct answer or text generated by the expected model given the prompt. Step 202: Create a fine-tuning environment Create a fine-tuning environment on the OpenAI console, select the base model, and then select the fine-tuning task option. The base model includes the fine-tuning functional model GPT-3.5-turbo, the chatbot program natural language model, the image generation system DALL·E, and the command-line interface CLI. Step 203: Set fine-tuning parameters The fine-tuning task selects appropriate parameters, including the number of fine-tuning steps, learning rate, and batch size. Step 204: Set fine-tuning data Upload the prepared dataset to the fine-tuning environment, format the dataset according to OpenAI's guidelines, and follow the data upload instructions; Step 205 begins fine-tuning To start the fine-tuning process, select "Start using API for fine-tuning" on the OpenAI console. This will trigger the fine-tuning process, and OpenAI will train the model using the provided dataset. Step 206: Monitor the fine-tuning progress. To track the progress and performance of fine-tuning, the OpenAI console will provide the status and metrics of the fine-tuning task, including training steps and training loss, allowing for appropriate adjustments to the fine-tuning as needed. Step 207 uses the new fine-tuning model After fine-tuning, select the already created model from the playground model in the OpenAI console, and determine whether the output result matches the expected metric. If the result is yes, call the good model directly through the API.

8. The method for industrial production process data control based on AI in a SCADA system according to claim 1, characterized in that, Step 3, data collection, also includes the following steps: Step 3-101: The SCADA system constructs an I / O driver layer, using an IGS gateway to integrate the protocols corresponding to various programmable logic controllers (PLCs) and instruments, and directly interacts with the hardware. The I / O driver temporarily stores the data read back in the DriverImage Table (DIT). Step 3-102 Configure the SAC (Scan, Alarm, and Control) service in the IFIS configuration software task. The SAC service will handle the periodic scanning of data, alarm processing, and data control. Step 3-103 configures the I / O drive address in the ifix process database PDB data block, stores the data read from the DIT, and records the real-time final acquired digital / analog values ​​of all points; the historical database periodically collects data from the process database PDB and records historical data, serving as the data source supporting historical curves and reports. Simultaneously, it synchronizes device status data to the relational database via the historical database; the relational database synchronizes historical database data to the SQL Server server through the AccessClient data synchronization service to obtain report data. Simultaneously, alarm information is synchronized to the SQL Server server through the historical alarm service started by ifix; Here, digital quantities are a data type representing discrete states. Acquire switch status or binary information; analog signals are a continuously changing data type. The system acquires values ​​within a continuous range, which are the values ​​of physical quantities such as voltage, current, temperature, and pressure that change continuously. Raw data acquisition refers to the raw electrical signal data directly obtained from sensors or devices. The raw signal data is either continuously changing analog data or discrete digital data. The SCADA system converts the raw signal data into digital or analog quantities to form target data. Step 3-104: The SCADA engineering station includes the display and control of the configuration interface. The display and control of the configuration interface mainly includes graphical objects, controls, schedulers, and chart objects. Each station in the SCADA engineering station has an independent workstation. The SCADA deployment adopts a distributed deployment architecture. A master station is defined on the upper layer of each substation, and the data of each substation can be directly monitored through the master station.

9. A method for industrial production process data control based on AI in a SCADA system according to claim 1, characterized in that, Step 3, obtaining the raw dataset and sending commands for data control by the SCADA system, refers to communicating and exchanging data with the device through the IoT Gateway plugin. This enables data collection and transmission, as well as device authentication, data encryption, and access control. Data from different devices is converted into different formats. The SCADA system collects real-time data, monitors device status, performs remote operations, and integrates and collaborates with other systems. The specific working steps of the IoT Gateway plugin are as follows: Step 3-201 Equipment Configuration Configure device parameters in the IoT Gateway plugin. Device parameters include the device's IP address, port number, and communication protocol. Step 3-202 Data Acquisition After the device is configured, the IoT Gateway plugin begins to collect data from the device through the OPC Server. The collected data includes the device's operating status and sensor readings. Step 3-203 Data Processing The IoT Gateway plugin collects raw binary data, converts the raw binary data and message data encapsulated through various protocols into target data in analog and digital form, and finally reports the collected target data values ​​through the TCP protocol. Step 3-204 Data Transmission The processed target data is transmitted to the SCADA system, which performs data analysis, equipment monitoring, and fault diagnosis. Step 3-205 Command Sending In addition to acquiring target data, the IoT Gateway plugin also sends control commands to the device. It transmits the start signal digital value to the PLC's central processing unit (CPU) via preset points and the S7 communication protocol. Upon receiving the signal, the PLC powers on the device. A point refers to a data acquisition point or a data input / output point, representing a specific monitoring or control data item used to indicate various parameters, states, or device interfaces that need to be monitored or controlled. The S7 communication protocol is a customized communication protocol developed by Siemens for its PLC series, used to transmit data and control commands between the computer and the PLC. In the SCADA system, the operator uses controls on the user interface to send control commands to preset points.

10. A method for industrial production process data control based on an AI-based SCADA system according to claim 9, characterized in that, The IoT Gateway plugin also integrates and collaborates with other systems. Through the access of the IoT Gateway, the system realizes the interconnection of devices and the integration of data, and performs device management, remote monitoring, data analysis and optimization. It uses a unified RESTful HTTP and message queue telemetry transmission (MQTT) interface for the upper layer. The upper layer model integration and remote control rely on the HTTP REST and MQTT standard interfaces. When the IoT Gateway plugin provides an HTTP interface and the upper-layer service integrates the IoT Gateway plugin, after selecting the network adapter, specifying the port and the routing URL, the IoT Gateway plugin will start a cluster search REST server according to the above-configured parameters. The upper-layer service can then directly call the API to obtain and send data. Among them, data acquisition refers to the upper-layer service acting as the caller, obtaining the device's operating status, faults, and parameter data through the HTTP interface, and using this data as the basic data input for the integration of the upper-layer model; data delivery refers to sending the command data for controlling the device to start and stop to the IoT Gateway service through the HTTP interface, thereby achieving the final remote control.

Images