Method and system for identifying and acquiring real-time data of industrial control system

Abstract

The invention discloses a method and a system for identifying and acquiring real-time data of an industrial control system, and particularly relates to a method for acquiring real-time data of an industrial control system deployed in an internal network under the condition of internal and external network isolation. Real-time data in an industrial control system interface is extracted and stored only by receiving a video stream unidirectionally output by an industrial control host and utilizing an image processing technology. Based on the characteristic that the positions of different display data in an industrial control system interface are fixed, the industrial data are accurately collected by configuring position information of the data needing to be collected. The method comprises the following specific steps: 1) configuring a position where data needs to be acquired in an industrial control interface; 2) intercepting an interface image from a video stream unidirectionally output by the industrial control system according to a specified time interval; 3) extracting a screenshot corresponding to the data to be collected according to the position configuration; 4) preprocessing the data screenshot and performing optical character recognition; and 5) verifying the correctness of the identification result and storing the identification result. The method is efficient, accurate, real-time and safe.

Description

Technical Field

[0001] The present invention relates to data acquisition technology in industrial control systems, and specifically to a method and system for real-time identification and acquisition of industrial control system data under network isolation conditions. It belongs to the industrial control technology direction in the computer field and also covers research and applications related to data acquisition and management. Background Art

[0002] The widespread application of industrial control systems (hereinafter referred to as industrial control systems) has brought many advantages to enterprises in improving production efficiency and optimizing management. By displaying the real-time data of production equipment in various links of production through the human-machine interface, operators can quickly obtain key information, make timely decisions and feedback operations, avoid production failures and safety hazards, and thus significantly improve the stability and safety of the production process. At the same time, the long-term storage of real-time data in the industrial control system provides enterprises with a rich historical record, helping enterprise management to achieve trend analysis, equipment maintenance prediction, and rapid tracing of problems. With the help of these industrial control data, enterprises can optimize production processes, improve the utilization of resources required for production, and meet the compliance requirements of regulatory agencies at all levels, laying the foundation for future technology upgrades and system improvements for enterprises.

[0003] Although most of the industrial control software used by existing enterprises already has the function of automatic data collection and meter reading and uploading data to remote storage, most enterprises, for security reasons, keep the industrial control system in a closed network environment and ensure the security of the industrial control system by physically isolating the industrial control system from the external service network system. Once the industrial control system is connected to the external Internet, the key information in the industrial production process directly controlled by the industrial control system may be subject to various threats, including virus invasion, data tampering, and illegal access and control. External network attacks may cause the industrial control system to paralyze and production to stagnate, and then feedback to the production process through the industrial control system to cause equipment damage, or even cause serious safety accidents. This not only threatens the production efficiency and safety of the enterprise, but may also cause huge economic losses. Faced with these complex network security issues, enterprises usually need to invest high costs for security maintenance, including the deployment of protection systems, monitoring of network interfaces, and training of relevant personnel. On the one hand, maintaining the necessary security requires continuous investment of a large amount of funds and technical forces to implement comprehensive network security protection measures, which greatly increases the production costs of enterprises; on the other hand, even if enterprises have made sufficient investments in network security, absolute security cannot be guaranteed. Therefore, in order to avoid high maintenance costs and potential risks caused by network security issues, most small and medium-sized enterprises choose to deploy industrial control systems on internal networks to achieve physical isolation from external networks, but this also brings new problems.

[0004] Due to the limitations of the isolated network environment, small and medium-sized enterprises can only rely on the traditional manual meter reading method to record industrial control data. This method is inefficient, the collected data can only be used for historical records, and it is prone to human errors, which cannot meet the real-time and accuracy requirements of modern industry for production process management. At the same time, it is difficult for enterprises to automatically integrate data into the enterprise resource planning (Enterprise Resource Planning, hereinafter referred to as ERP) system, and it is impossible to achieve comprehensive data analysis and optimization management, which directly hinders the automation, intelligent development and digital transformation of the production and management of enterprises, and makes it difficult for industrial enterprises to apply modern information technology to help enterprises gain competitive advantages. In this context, how to use industrial control systems to improve efficiency and optimize management while minimizing their potential risks has become a topic that needs to be urgently solved in the current field of industrial informatization. It is particularly important to provide a technology that can safely, efficiently and accurately collect enterprise industrial control system data under network isolation conditions.

[0005] The present invention proposes an innovative solution that can collect and store real-time data of the interactive interface in the intranet industrial control system while isolating the external network. This method ensures the integrity of the data by optimizing the data collection and storage process, while keeping the enterprise industrial control system inaccessible to the external network, reducing the security maintenance costs of small and medium-sized enterprises. In this way, it not only helps enterprises reduce security costs and overcome the shortcomings of traditional technologies, but also provides new impetus for the digital transformation and sustainable development of enterprises. Summary of the invention

[0006] The purpose of the present invention is to overcome the network security risk problem that may be caused by the application of existing industrial control data acquisition technology, and to provide an industrial control data acquisition method and system based on network isolation conditions. After the method obtains the video stream output by the industrial control host through hardware (one-way output), the image processing method is used to safely obtain the real-time data displayed in the human-computer interaction interface of the enterprise industrial control system. Since the data acquisition of this method is not performed through the network port, the risk of the industrial control system deployed in the internal network being attacked by the external network is fundamentally eliminated.

[0007] In order to achieve the purpose of the present invention, the technical solution adopted is summarized as follows:

[0008] A method for acquiring industrial control data based on network isolation conditions, which obtains the output video stream of the industrial control system through hardware facilities and inputs it into the working host responsible for data collection for data identification, storage and other operations. The main process includes the following steps:

[0009] (1) Obtain image data of the human-machine interface regularly through the one-way video stream output by the industrial control system;

[0010] (2) Obtain screenshots of each data area based on the interface screenshots in step (1);

[0011] (3) preprocessing the data area screenshot obtained in step (2);

[0012] (4) performing optical character recognition on the preprocessed data screenshot obtained in step (3);

[0013] (5) Check the data text results obtained in step (4) and record the status;

[0014] (6) Check the result of step (5) that needs secondary identification and then proceed to step (4);

[0015] (7) Store the data text results obtained through step (5).

[0016] The step (1) specifically comprises the following steps:

[0017] (1-1) Under the condition of isolating the network, the method of periodically obtaining the human-machine interface is as follows:

[0018] If the video output graphics card of the industrial control host has multiple output interfaces, the video output mode can be set to screen copy mode to distribute the video signal at the same time, with one end connected to the original display and the other end connected to the video capture card of the working host; if the industrial control host has only one output interface, the video signal can be processed in two with the help of a video splitter, with one end connected to the original display and the other end connected to the video capture card of the working host. The above method ensures that the normal display function of the industrial control host is not affected and data collection requirements can be met. The working host receives the video stream of the industrial control system human-machine interface from the video capture card and takes a screenshot of the video in the video stream to obtain the human-machine interface image data.

[0019] The entire acquisition process relies on the split screen device and video capture card hardware, and through one-way data transmission, the industrial control system can collect data while isolating the external network. Since this method only involves one-way video data transmission on the industrial control system interface, it eliminates the possibility of external network attacks on the industrial control system and effectively avoids network security threats.

[0020] In order to flexibly set the time interval for collecting and storing data, define the time T for the nth screenshot n It can be expressed as T n =t0+n·t.

[0021] Where t0 represents the starting time (the time of the first screenshot), t represents the time interval (in minutes), and n represents the sequence number of the screenshot (n=1, 2, 3, ...).

[0022] The step (2) specifically comprises the following steps:

[0023] (2-1) The method for obtaining screenshots of each data area based on the interface screenshot is as follows:

[0024] In the human-machine interface of the industrial control system, the production data that the enterprise is concerned about are all displayed in a fixed frame (hereinafter referred to as data block), and the corresponding production data name is marked above each data block, for example: the reading of the hydraulic gauge in a pipeline. Since the position of the data block is fixed during the operation of the industrial control host, the present invention uses this feature to preset the position information (including coordinates and size) of each data block through the configuration file, so as to obtain a screenshot of each data block (hereinafter referred to as data screenshot, PIC) according to the position information.

[0025] The position information of the data block in the data screenshot includes: (x, y) represents the coordinates of the upper left corner, w and h represent the width and height of the data block respectively, and the screenshot area of ​​the data block can be expressed as a rectangular range: Rect = (x, y, w, h).

[0026] For the process of taking a screenshot every time t, at time point T n After taking the screenshot of the interface, continue taking screenshots of each data block. When taking the nth screenshot of the interface, extract the set P obtained from all the data block areas. n It can be expressed as:

[0027] P n ={(x i ,y i, ,w i ,h i )|i=1,2,…,k}.

[0028] Among them, i is the number of the data block, there are k data blocks in total, (x i ,y i, ,w i ,h i ) is the location information of the i-th data block.

[0029] The step (3) specifically comprises the following steps:

[0030] (3-1) Get data screenshot P i After that, it is subjected to adaptive binarization processing, that is, P i The pixel point I(x,y) in the image is converted to the pixel point B(x,y) of the binary image. The specific method is as follows:

[0031]

[0032] Where V *The optimal threshold is a dynamic value that is determined dynamically for each pixel based on the local characteristics of the image, rather than using a global fixed threshold for the entire image. The processing steps include: first converting the image to grayscale and denoising it (such as applying Gaussian blur); then selecting the local mean or Gaussian weighted mean as the threshold calculation method; finally, dynamically calculating the optimal threshold for each pixel by setting the local window size and adjusting parameters.

[0033] (3-2) Due to the differences in background and font colors of the data blocks in the human-machine interface of the industrial control software, the adaptive binarization process may lead to inconsistent image styles. Therefore, for the image with inconsistent binary image styles obtained in (3-1), a method to unify the style information is proposed. The specific method is as follows:

[0034] Preset the style information S of each data block in the configuration file i ,include:

[0035] Background color distribution BG i : For example: pure color (green) or mixed color (multiple colors exist in the same data block).

[0036] Target style: unified as black background and white font.

[0037] The background color of the mixed color data block after binarization is still mixed color, and the background color needs to be unified. Therefore, for the binary image B(x, y) processed in step (3-1), perform the following steps to obtain a pure color binary image B′(x, y):

[0038] i. Start from the left edge of the image and scan the pixels column by column.

[0039] ii. For column c, count the number of black pixels N c .

[0040] iii. Compare N c and N c-1 (the number of black pixels in the previous column), if N c ≠N c-1 (there is a change), record the column index c stop , continue scanning until the second change, and get the column index c end .

[0041] iv. For all column indices c≤c end Replace the pixels by changing the black pixels to white to form the pure white background area on the left.

[0042] v. Start scanning from the right edge of the image and repeat the above steps to obtain a binary image with a white background and black font color.

[0043] To achieve the target style, it is necessary to invert the binary image B′(x, y) with a uniform background color to obtain B″(x, y):

[0044]

[0045] This operation adjusts the background of the image to black and the font to white to meet the unified target style.

[0046] (3-3) In a single human-machine interface screenshot, there are usually multiple data blocks, each of which is small in size, resulting in low accuracy when performing text recognition directly. According to experiments, under the same conditions and quantity, if the image is enlarged by 200%, the recognition accuracy can be increased from 81.23% to 99.66%. Therefore, the following processing is performed on the binary image B(x, y) (or B″(x, y)) returned by step (3-1) or step (3-2) to obtain B end (x,y):

[0047] Resize the image according to the magnification ratio PE to obtain the enlarged image size:

[0048] W′=PE×W, H′=PE×H, where PE=200%.

[0049] Where W is the original width of the image, H is the original height of the image, and the enlargement process uses a bilinear interpolation method to ensure that the image quality is preserved as much as possible.

[0050] The step (4) specifically comprises the following steps:

[0051] (4-1) The method for obtaining text results based on the preprocessed data screenshot is as follows:

[0052] The optical character recognition (OCR) engine is used to process the binary image B obtained by step 3. end (x, y) is recognized to obtain the text result TX and text status TS:

[0053] <TX,TS> =OCR(B end (x,y)).

[0054] The step (5) specifically comprises the following steps:

[0055] (5-1) Check the text results obtained in step (4) and record the status as follows:

[0056] After obtaining the preliminary recognition result TX, the history record of each data block is maintained by maintaining a history space H. For each human-machine interface, a history space H is defined, which is a data block set {H1, H2, …, H k}, where H i Represents the history of the i-th data block:

[0057]

[0058] in, is the jth recognition result of the i-th data block, for Corresponding state, state Maintained by the state comparison table (see Example 2 for an example), m represents the maximum number of times recorded in the history space, which can be set according to actual needs.

[0059] According to the preset status comparison table, TX i TS i To make a judgment, assume that the state comparison table presets the required secondary recognition state code as τ:

[0060]

[0061] After the status is determined, the historical space is updated and the current recognition result and status (TX i ,TS i ) is added to the corresponding data block history H i middle:

[0062] H i ←H i ∪{(TX i ,TS i )}.

[0063] If H i If the maximum number of records m is exceeded, the earliest record is deleted to keep the capacity of the history space fixed.

[0064] The data that has been verified to be correct and the historical space has been updated enters step (7) for further processing.

[0065] The step (6) specifically comprises the following steps:

[0066] (6-1) The operation method for checking the data that needs secondary identification in step (5) is as follows:

[0067] When the initial recognition result is judged to require secondary recognition, the system will start the second OCR engine to process image B. end (x,y) is re-recognized. This engine will re-recognize the processed image and obtain a new text result Tx′ iand state TS′ i :

[0068] TX′ i ,TS′ i =OCR2(B end (x,y)).

[0069] The result will be considered final, and the system will log the process and update the historical space for subsequent analysis and tracking.

[0070] The step (7) specifically comprises the following steps:

[0071] (7-1) The method for storing the single text result and data status returned by step 5 or step 6 is as follows:

[0072] For each data block, the system records its final result and status For each data block P i , organizing its records into tuples R i :

[0073]

[0074] (7-2) The method for obtaining and storing all data blocks and status of the interface screenshot is as follows:

[0075] Repeat steps (3) to (7-1) to form a set R of all data block record tuples of the interface screenshot. final :

[0076]

[0077] Where k is the number of data blocks in the current interface; d is the date when it is stored, in the form of xxxx-xx-xx; t is the time when it is stored, in the form of xx:xx:xx.

[0078] The set is stored as a complete record in the database table DBT corresponding to the human-machine interface:

[0079] DBT←DBT∪{R final}.

[0080] The above steps summarize the technical solution of the present invention, wherein the configuration file is mentioned in step (2) and step (3). The configuration file is a key tool for the system to achieve accurate positioning and style adjustment during data acquisition and processing. It records the location information, style characteristics and other necessary auxiliary information of the data block in the human-computer interface to ensure the efficiency and accuracy of the entire process.

[0081] The present invention proposes an interactive generation tool to help users quickly and accurately generate configuration file information for each human-machine interface. The main process includes the following steps:

[0082] (1) Human-machine interface selection;

[0083] (2) Zooming and moving the interface;

[0084] (3) Acquisition of single data block information;

[0085] (4) Checking data block configuration information;

[0086] (5) Generation of complete configuration files;

[0087] (6) Creation of database tables.

[0088] The step (1) specifically comprises the following steps:

[0089] (1-1) The method to open the tool selection human-machine interface is as follows:

[0090] The user selects a human-machine interface image in the tool as the starting point for generating or loading a configuration file. The image can be obtained by reading it locally or directly intercepting it. The system will verify and save the image name entered by the user. The specific description is as follows:

[0091] The user can browse the file system and select the human-machine interface image PS of the industrial control work host that has been obtained locally. Alternatively, the user can also initiate a capture operation through the tool to obtain the current human-machine interface screenshot PS from the industrial control work host in real time. Before taking a screenshot, you need to enter the target name PName of the screenshot, and PName must use English characters. Special symbols and spaces are prohibited. The system will perform a duplicate name check on the PName entered by the user. If a file named PName already exists in the target directory, the user will be prompted to re-enter the name; if the name PName is unique, the screenshot PS is allowed to be saved as an image file PS saved , the path is the specified subdirectory of the tool.

[0092] The step (2) specifically comprises the following steps:

[0093] (2-1) The method for moving and scaling the interface is as follows:

[0094] After entering the image display page, since the data block is small, the user can use the scroll wheel or the "Zoom in" and "Zoom out" buttons to adjust the image size. After adjusting to the appropriate size, the user can hold down the Ctrl key on the keyboard and hold down the left mouse button to drag the image to the position containing the target data block.

[0095] The step (3) specifically comprises the following steps:

[0096] (3-1) The method for obtaining information of a single data block is as follows:

[0097] The user clicks the left button and drags the mouse to draw a red rectangle on the image to select the target data block area. The system captures the coordinate information of the rectangle drawn by the user in real time:

[0098] Rect=(x,y,w,h),

[0099] Where (x, y) is the coordinate of the upper left corner of the rectangle, and w and h are the width and height respectively.

[0100] (3-2) The method for users to fill in data information is as follows:

[0101] When the user clicks the "Get Information" button, the system will capture and fill in the coordinate information R and the identifier Mix∈{0,1} of whether the background color is mixed. The system will automatically judge based on the color features in the area.

[0102] The user needs to manually enter the data block P i Name to completely fill in the data block information.

[0103] (3-3) The method for the user to correct the data information is as follows:

[0104] The user clicks the "Correction" button, and the system automatically adjusts the coordinates and size of the data block R = (x', y', w', h'), and a pop-up window displays the screenshot of the data block selected by the user. R . Among them, x′,y′,w′,h′∈Z + , Z + is a set of positive integers.

[0105] The user checks the contents in the data block information window, confirms the completeness and accuracy of the coordinate information R, background color mark Mix and data block name Name, and then clicks the "Save" button to add the current data block information to the configuration file.

[0106] The step (4) specifically comprises the following steps:

[0107] (4-1) The method by which the system checks the data block configuration information is as follows:

[0108] When the user clicks the "Save" button to create the configuration information of the current data block, the system will automatically check the information, including the following: whether the data block name Name is duplicated, because it is stored as a column name in the database and must be unique; whether the elements of the coordinate information R are integer data. The configuration information is checked automatically. If the data does not meet the requirements, the tool will pop up a prompt window to ask the user to re-enter. If and only if the information format is correct, the tool will add the current data block configuration information to the configuration file.

[0109] The step (5) specifically comprises the following steps:

[0110] (5-1) The method to generate a complete configuration file is as follows:

[0111] The user repeats steps (2), (3), and (4) on this page to complete the configuration of all data blocks in the human-machine interface and form a complete configuration file.

[0112] (5-2) The method for dynamic display and review during configuration is as follows:

[0113] When the user clicks the "Show Data" button, the system will mark all the data blocks of the saved configuration information with a blue rectangle. When the user right-clicks inside the blue rectangle, an information window will pop up. i blue Display the configuration information of the data block (including: rectangular area, data block image, name):

[0114] W i blue =<(x i ,y i ,,w i ,h i ),P i ,Name>

[0115] For the information to be modified, the user can repeat steps (2), (3), and (4) to update the configuration and save it again. If no modification is required, the information window can be closed directly.

[0116] When the user clicks the "Hide Data" button, all blue rectangular box marks will be removed without affecting the saved configuration information.

[0117] (5-3) The method of exiting the system and restoring operations is as follows:

[0118] The user can exit the operation at any time during the configuration process, and the tool will automatically save the currently configured data block information to temporary storage.

[0119] To operate the same video screenshot image PIC again, the user needs to load the image from the local file and re-enter step (1) to start the operation. The tool will automatically load the saved configuration information.

[0120] The step (6) specifically comprises the following steps:

[0121] (6-1) The method for establishing the database table corresponding to the current human-machine interface is as follows:

[0122] When the user confirms to create the table, click the "Finish" button, and the system will return the user to the pop-up window before selecting the picture. After selecting the "Create Table" button, the table creation operation will be executed.

[0123] The system uses the picture name PName as the database table name T PName , and checks whether the table name exists. If the table does not exist, the system creates a new table with the following initial structure:

[0124] Table TPName ={Name i ,NameState i |i∈[1,k],i∈Z}

[0125] Where k is the number of data block information saved in the configuration file corresponding to the current image, and NameState corresponds to the collection state of the data block named Name.

[0126] If the table already exists, add data block information and check the data block field name and status set in the configuration file Field collection with existing table For the newly added data block name Add a new column (Nmae new ,NameState new ).

[0127] After the table is created or updated, the system prompts that the operation is successful.

[0128] Through this generation tool, the generation of configuration files corresponding to the human-machine interface and the creation of its database table can be integrated into one project, significantly improving production efficiency. In the initial experiments of the present invention, it took up to 4 hours to manually generate a human-machine interface configuration file containing 83 data blocks, and there were errors in recording data block names and repeated addition of the same data block configuration information. After using this tool, under uninterrupted operation, the generation time was shortened to about 20 minutes, and no errors occurred. These practical results fully demonstrate the usability and practicality of the supporting tool, and provide indispensable support for achieving the goals of the present invention.

[0129] Compared with the prior art, the method and system for real-time data identification and acquisition of industrial control systems under isolated network conditions provided by the present invention have the following advantages:

[0130] 1. Enhanced Data Security:

[0131] By using a screen splitter and a working host of a video capture card, the present invention can obtain the human-machine interface data of the industrial control system deployed in the intranet in real time without the need for the industrial control system to be connected to the Internet. When the industrial control system is isolated from the external network, it can not only obtain and record the industrial control data in real time and efficiently, but also avoid various network security threats brought by the access to the Internet.

[0132] 2. Efficient data collection:

[0133] The present invention provides an automated way to collect real-time data from industrial control systems, solving the problems of low efficiency and error-proneness in the traditional manual meter reading method that enterprises have to choose, and improving the efficiency and accuracy of data recording. The present invention provides a flexible storage solution that supports users to choose to store data in a local database or a cloud database according to their needs, and supports seamless integration with the ERP system. This enables enterprises to achieve centralized management and comprehensive analysis of data, and facilitates comprehensive monitoring and management of the production process.

[0134] 3. Reduce security maintenance costs:

[0135] The present invention saves enterprises from the high costs of purchasing and maintaining network equipment, managing network traffic, and hiring and training professional network security personnel, thereby significantly reducing their operating expenses. It reduces the investment of enterprises in network security and maintenance, allowing limited resources to be used more efficiently in other key business areas.

[0136] 4. Supporting enterprise digital transformation:

[0137] The present invention provides strong support for modern production and management for enterprises, promotes enterprises to quickly respond to market changes and production needs, and helps them realize automation and optimization of production processes through efficient and secure data processing and management. It supports enterprises to conduct comprehensive data analysis and optimize production plans, promotes digital transformation and sustainable development of enterprises, and enhances their competitiveness in the market.

[0138] 5. Compliance and Issue Tracking:

[0139] The stored historical data can meet the regulatory authorities’ requirements for data recording and reporting, ensuring that the company’s operations are legal and compliant. In addition, by storing and analyzing historical data, companies can conduct trend analysis, predict equipment maintenance needs, and quickly trace back to find the root cause of problems when they occur, thereby improving operational efficiency and safety.

[0140] Through the above-mentioned innovative technical solutions and auxiliary tools, the present invention successfully solves the problem that general means cannot collect industrial control system data in real time under the condition of network isolation. Through network isolation and intelligent identification technology, the present invention greatly improves the efficiency of data processing and management while ensuring the security, efficiency and accuracy of data acquisition, providing strong support for the production management and digital development of small and medium-sized enterprises, and solving their urgent need for a method and system that can safely and efficiently acquire and store industrial control system data when disconnected from the network, providing solid technical support and innovative solutions for enterprises to achieve the strategic goals of safe production and cost reduction and efficiency improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0141] Figure 1 The human-machine interface data recognition and archiving flow chart of the method of the present invention is

[0142] Figure 2 The data screenshot preprocessing flow chart in the embodiment of the method of the present invention is

[0143] Figure 3 The interactive tool of the method of the present invention generates a flow chart of a configuration file DETAILED DESCRIPTION

[0144] The technical solution of the present invention is described in detail below through specific embodiments.

[0145] Example 1: Generation of human-machine interface configuration files and creation of database tables

[0146] In order to achieve efficient acquisition and management of human-machine interface data, the present invention first needs to generate a configuration file for the human-machine interface and create a corresponding database table. The specific steps are as follows:

[0147] Step 1: Human-machine interface selection

[0148] After running the tool, a pop-up window will be displayed. At this time, the "Create Table" button is gray and cannot be clicked. The user needs to click the "Collect" button to select the corresponding human-computer interface image.

[0149] In this embodiment, the user chooses to read an image file named "general_control" from a local file and successfully loads it into the main interface for generating configuration information.

[0150] If the user chooses to load by screenshot, it is necessary to connect the capture card and obtain the current interface of the video source, and enter a unique English file name to save the screenshot to the folder "OCR_image".

[0151] Step 2: Zoom and move the interface

[0152] After entering the image display page, due to the small size of the data block, the user can zoom in on the image by scrolling up with the wheel or clicking the "Zoom In" button. Conversely, the user can zoom out on the image by scrolling down with the wheel or clicking the "Zoom Out" button.

[0153] In addition, users can hold down the Ctrl key and drag the left mouse button to move the image position, making it easier to view and operate data blocks.

[0154] Step 3: Obtaining information about a single data block

[0155] (3-1) Get single data block information

[0156] By left-clicking and dragging the mouse, draw a red rectangle on the graph to select the target data block. During the drawing process, the system tracks and records the coordinate information of the rectangle in real time.

[0157] (3-2) Fill in data information

[0158] After the user clicks the "Get Information" button, the system automatically captures and fills in the coordinate information of the selected rectangle. Except for the data block name (Name), other information has been pre-filled by the system. The user needs to manually enter the data block name, in this case "LI_0200", and check whether the data block contains a white background (background mixed color). At this time, the user needs to verify whether the mixed color identification automatically filled by the system is accurate, where "1" indicates background mixed color and "0" indicates background pure color.

[0159] (3-3) Correction data information

[0160] The user must click the "Correction" button, and the system will automatically adjust the coordinates and width and height information of the rectangular box to positive integer form. At the same time, a window will pop up to display a screenshot of the adjusted rectangular box area.

[0161] The user needs to check whether the screenshot content meets the needs and confirm the coordinate information of the rectangular frame, the background color mixing mark, and the accuracy of the data block name. After confirmation, click the "Save" button to complete the configuration information saving operation of a single data block.

[0162] Step 4: Check the data block configuration information

[0163] When the user clicks the "Save" button to create the configuration information of the current data block, the tool will automatically check the data block configuration information, including the following: whether the data block name is duplicated, whether the coordinates and width and height information are integer data. If the check finds that the data does not meet the requirements, the system will pop up a window to prompt the user to modify and re-enter. After checking that all information is correct, the system will save the configuration information of the data block to a configuration file named after the image.

[0164] Step 5: Generate a complete configuration file

[0165] After clicking the "Show Data" button, the system will mark the data block with saved configuration information with a blue rectangular box. The user can gradually repeat steps 2, 3 and 4 to generate the configuration information of all data blocks of the entire human-machine interface in sequence until all data block information is generated or the user decides to end the current operation. The user can view the "general_control.ini" file saved in this embodiment in the folder "image_ini". If you need to modify the generated data block information, the user only needs to right-click anywhere in the blue rectangular box to enter the update interface. The update operation must also follow the configuration information check requirements of step 4 to ensure the integrity and accuracy of the data.

[0166] Step 6: Create database tables

[0167] When all data block information is saved, the user clicks the "Finish" button to end the operation of the current interface and return to the initial pop-up window interface. Then, click the "Create Table" button, the tool will create a database table with the image name as the database table name, and prompt that the table creation is successful. The system will automatically check whether the database table already exists: if the table already exists, only the data block name and status column that have not been recorded will be added, and there is no need to create it again.

[0168] Through the above steps, a configuration file named "general_control" is successfully generated. This configuration file contains detailed information about 83 data blocks related to the human-machine interface. The record of each data block includes the following 6 items of information: the x and y coordinates of the upper left corner of the data block, the width and height of the data block, the name of the data block, and whether the background color of the data block contains white. This information provides complete data support for subsequent database storage and processing.

[0169] Example 2: Real-time data recognition, acquisition and archiving of human-machine interface

[0170] Step 1: Get the image data of the human-machine interface regularly

[0171] This system achieves safe acquisition of the industrial control system human-machine interface by building a PCIE video capture card into the working host. By connecting a screen splitter, the output signal of the industrial control host is divided into two: one end is connected to the original monitor to ensure the normal operation of the system, and the other end is connected to the PCIE video capture card of the working host. Under the premise of confirming network isolation, the working host receives the video signal of the industrial control system human-machine interface and successfully obtains the content of the human-machine interface.

[0172] This embodiment sets a frame of human-machine interface image to be collected every 5 minutes. The data block configuration information of the interface is stored in the "general_control.ini" file generated in Embodiment 1, providing a basis for subsequent data analysis and management

[0173] Step 2: Get screenshots of each data area based on the interface screenshots in step 1

[0174] Through the data block coordinates and width and height information preset in "general_control.ini", this system can accurately obtain screenshots of each data block in the human-machine interface. The system reads the 83 data block information in the configuration file in turn and obtains the original image of the data block based on this information.

[0175] Step 3: Preprocess the data area screenshot obtained in step 2

[0176] (3-1) Binarization

[0177] The data screenshot is adaptively binarized to obtain a binarized image. At this time, the obtained binarized style may not meet the target style.

[0178] (3-2) Unified Binarization Image Style

[0179] The "is_white_background" in the data block configuration information indicates whether the background of the data block contains a white background, which may cause the background color of the image after binarization to be inconsistent (black and white mixed). In order to achieve a unified style, the system applies a method to replace the continuous black column pixels on the left and right sides of the binary image with white, thereby obtaining a binary image with white as the background and black as the font color. Subsequently, the entire image is pixel-inverted, and the goal of unifying the binary image style is finally achieved, that is, the processing result with black as the background and white as the font color.

[0180] (3-3) Scale up the binary image

[0181] The current image is enlarged by 200%. The enlargement operation uses a bilinear interpolation method to ensure that the image details are preserved.

[0182] Step 4: Perform optical character recognition on the pre-processed data screenshot obtained in step 3

[0183] Mobilize the OCR engine to recognize the image and obtain the text result corresponding to the image.

[0184] Step 5: Check the text results obtained in step 4 and record the status

[0185] After obtaining the preliminary recognition result, the system first makes a state judgment on the result, and searches for the historical space data of the corresponding state according to the judgment result. In this embodiment, the maximum number of historical space records m is 20 times. In order to make a state judgment, the present invention maintains the following state comparison table, where the judgment basis "1" indicates that no secondary recognition is required, and "0" indicates that secondary recognition is required, see Table 5-1.

[0186] Table 5-1 Status comparison table

[0187]

[0188] It is particularly important to note that for the state where the recognition result is a floating point number, even if the state itself is correct, it is still necessary to meet the condition that the difference between the current result and the average value of the historical results does not exceed ±20% to be considered correct; otherwise, it is considered to require secondary recognition. Data that requires secondary recognition will enter step 6 for further processing, and data that does not require secondary recognition will enter step 7.

[0189] Step 6: Check the results of step 5 for secondary identification

[0190] When the preliminary recognition result needs to be recognized again, the second OCR engine object maintained by the system is started to recognize the processed image, obtain the recognition result and its text status, and use this result as the final result, while recording logs and status.

[0191] Step 7: Store the text results and status of step 5

[0192] (7-1) Get the result and status information of the current data block

[0193] Record and temporarily store the text results and status returned by step 5 or step 6. For example, the record tuple of the current data block is (85.56,0).

[0194] (7-2) Get the results and status information of all data blocks in the current interface screenshot

[0195] Repeat step 3 to step (7-1) to gradually obtain the final results and status of all data blocks of the current human-machine interface to form a tuple set. In this embodiment, the tuple set corresponding to the human-machine interface has 83 elements. The system organizes these results, status and recording time into a record and stores it in the database table "general_control". For example, the recording time is 2024-11-28, 15:20:32, and then the corresponding data block results and status are stored in sequence.

[0196] In order to verify the accuracy of the recognition efficiency, a long-term test was conducted by obtaining the human-machine interface video source in the embodiment. The specific method of one of the tests was: Figure 1 A total of 235 screenshots were taken. Each screenshot contains 81 valid data blocks, and a total of 19013 data blocks were generated. The test results show that there are 64 data blocks with recognition errors, including 38 data blocks that cannot recognize the numbers after or before the decimal point, 24 data blocks with empty recognition results, and 2 data blocks that mistakenly recognize numbers as letters. The overall recognition accuracy rate reached 99.66%. This result shows that the data acquisition device of the present invention has achieved a high recognition accuracy rate under the restricted condition of isolated network, ensuring the security and reliability of data, thereby demonstrating its effectiveness and feasibility in practical applications.

[0197] In summary, the present invention proposes a method for collecting human-machine interface data of industrial control systems under isolated network conditions, which provides an efficient and reliable solution for enterprises to realize data collection and management under the premise of ensuring network security. By using a video capture card to achieve real-time acquisition of human-machine interface images, and combining the configuration file preset and the dynamic update mechanism of data block information, the present invention can help enterprises ensure accurate collection and safe storage of industrial control data without the need for additional complex network protection investment.

[0198] The above is only a preferred example to illustrate the present invention, which does not limit the scope of the present invention. Therefore, without departing from the concept of the present invention, any equivalent changes made by using the contents of the description and drawings of the present invention are also included in the scope of the claims of the present invention.

Claims

1. A method for real-time data recognition and acquisition in an industrial control system, the method taking a video stream unidirectionally output by an industrial control host computer received through a video capture card as input, and storing a data record consisting of date, time, data result and data status as output, Its characteristics include the following steps: (1) Obtain image data of the human-machine interface regularly through the one-way video stream output by the industrial control system; (2) Obtain screenshots of each data area based on the interface screenshots in step (1); (3) preprocessing the data area screenshot obtained in step (2); (4) performing optical character recognition on the preprocessed data screenshot obtained in step (3); (5) Check the data text results obtained in step (4) and record the status; (6) If the result of step (5) is unqualified, perform step (4) again; (7) Store the data text results obtained through step (5).

2. The method according to claim 1, characterized in that The step (1) is to obtain only the unidirectional output video stream under the network isolation condition and intercept it at a fixed time, and specifically comprises the following steps: (2-1) If the video output graphics card of the industrial control host has multiple output interfaces, the video output mode can be set to screen copy mode, and the video signal can be distributed simultaneously, with one end connected to the original display and the other end connected to the video capture card of the working host; if the industrial control host has only one output interface, the video signal can be split into two with the help of a video splitter, with one end connected to the original display and the other end connected to the video capture card of the working host. The above method ensures that the normal display function of the industrial control host is not affected and data collection requirements can be met; (2-2) The working host receives the video stream of the human-machine interface of the industrial control system from the video capture card, and takes regular screenshots of the video in the video stream to obtain image data of the human-machine interaction interface.

3. The method according to claim 1, characterized in that The step (2) obtains screenshots of each data area through a preset data block configuration file, wherein the configuration file includes coordinates, width, height, background color and name information of the data block; the configuration file is generated by an interactive tool and specifically includes the following six steps: (1) Human-machine interface selection; (2) Zooming and moving the interface; (3) Acquisition of single data block information; (4) Checking data block configuration information; (5) Generation of complete configuration files; (6) Creation of database tables.

4. The method according to claim 1, characterized in that The step (3), preprocessing aims to unify the screenshot style of the data area into a target style with a white background color and a black font color, and specifically includes the following steps: (4-1) After obtaining the data screenshot, it is subjected to adaptive binarization processing. i The pixel point U(x,y) in the image is converted to the pixel point B(x,y) of the binary image. The specific method is as follows: Where V * is the optimal threshold, which is a dynamic value that is dynamically determined for each pixel based on the local characteristics of the image, rather than using a global fixed threshold for the entire image; the processing steps include: first converting the image to a grayscale image and denoising it (such as applying Gaussian blur); then selecting the local mean or Gaussian weighted mean as the threshold calculation method; finally, by setting the local window size and adjusting the parameters, dynamically calculating the optimal threshold for each pixel; (4-2) For the image with inconsistent binary image style obtained in (4-1), a method for unifying style information is proposed. The specific method is as follows: Preset the style information S of each data block in the configuration file i ,include: Background color distribution BG i : Pure color or mixed color; Target style: unified as black background and white font; The background color of the mixed color data block after binarization is still mixed color, and the background color needs to be unified; therefore, for the binary image B(x, y) processed in step (4-1), the following steps are performed to obtain a pure color binary image B′(x, y): i. Starting from the left edge of the image, scan the pixels column by column; ii. For column c, count the number of black pixels N c ; iii. Compare N c and N c-1 That is, the number of black pixels in the previous column. If N c ≠N c-1 , record the column index c stop , continue scanning until the second change, and get the column index c end ; iv. For all column indices c≤c end Replace the pixels of the image, changing the black pixels to white to form a pure white background area on the left; v. Start scanning from the right edge of the image and repeat the above steps to obtain a binary image with a white background and black fonts; To achieve the target style, it is necessary to invert the binary image B′(x, y) with a uniform background color to obtain B″(x, y): This operation adjusts the image background to black and the font to white to meet the unified target style; (4-3) The binary image B(x, y) (or B″(x, y)) returned by step (4-1) or step (4-2) is resized according to the enlargement ratio PE to obtain an enlarged image.

5. The method according to claim 1, characterized in that The step (5) verifies the data text result status through the status comparison table to determine whether to perform secondary recognition, and at the same time maintains a history space for each interactive interface to maintain the history record of each data block, specifically including the following steps: (5-1) For each human-computer interface, define a history space, which consists of a set of data blocks. Each data block set consists of a text result and a state pair. The state is maintained by a state comparison table. The maximum number of times the history space is recorded can be set according to actual needs. (5-2) According to the preset status comparison table, the status of the data text result is judged; for floating-point type text, if the fluctuation between the current result and the average value of the historical space data is within the range of ±20%, the data is temporarily stored and the historical space is updated; if the fluctuation exceeds 20% or the data is not of floating-point type, enter step (6) for further processing.

6. The method according to claim 1, characterized in that The step (3) obtains the configuration information of a single data block through an interactive tool. The method combines human-computer interaction to accurately locate the area of ​​the target data block and extract detailed information including coordinates, size, background color and name. Specifically, the method includes the following steps: (6-1) The method for obtaining information of a single data block is as follows: The user draws a red rectangle on the image by left-clicking and dragging the mouse to select the target data block area; the system captures the coordinate information of the rectangle drawn by the user in real time: Rect=(x,y,w,h), Where (x, y) is the coordinate of the upper left corner of the rectangle, w and h are the width and height respectively; (6-2) The method for users to fill in data information is as follows: When the user clicks the "Get Information" button, the system will capture and fill in the coordinate information R and the identification of whether the background color is mixed color Mix∈{0,1}. The system will automatically judge based on the color characteristics in the area; The user needs to manually enter the data block P i Name, to completely fill in the data block information; (6-3) The method for the user to calibrate data information is as follows: When the user clicks the "Correction" button, the system automatically adjusts the coordinates and size of the data block R = (x', y', w', h'), and a pop-up window displays the screenshot of the data block selected by the user. R ; where x′,y′,w′,h′∈Z + , Z + is a set of positive integers; The user checks the contents in the data block information window, confirms the completeness and accuracy of the coordinate information R, background color mark Mix and data block name Name, and clicks the "Save" button to add the current data block information to the configuration file.

Images