Flowchart creation method, model training method, device, equipment and medium

Abstract

The present application provides a method for creating a flowchart, a method for training a model, an apparatus, a device, and a medium, which relate to the field of computer technology. The method for creating a flowchart includes: obtaining an industrial control flowchart picture of a first industrial control system; performing text recognition on the industrial control flowchart picture based on a pre-trained flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content; and generating an industrial control flowchart of a second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content. The present application can save the drawing time of engineering personnel and improve work efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a method for creating a flowchart, a method for training a model, a device, a device and a medium. Background Art

[0002] Industrial control systems act as the industrial brains in the industrial system. With the emergence of new projects or renovation projects, or due to reasons such as the lifespan of the control system, newly developed control systems or updated versions of control systems are used to replace the old control systems to meet the requirements of new projects or renovation projects.

[0003] After the control system is replaced, engineers need to re-edit various industrial control flowcharts in the old control system in the new control system. This work is cumbersome and mechanical, and it takes a lot of time for engineers. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for creating a flowchart, a method for training a model, a device, a device and a medium for the above-mentioned deficiencies in the prior art, so as to quickly create a corresponding industrial control flowchart in the industrial control system based on the industrial control flowchart picture and save the time for manually creating the flowchart.

[0005] To achieve the above purpose, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for creating a flowchart, and the method includes:

[0007] Obtain an industrial control flowchart picture of a first industrial control system;

[0008] Perform text recognition on the industrial control flowchart picture based on a pre-trained flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content;

[0009] Generate an industrial control flowchart of the second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content.

[0010] Optionally, the flowchart text recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier; the performing text recognition on the industrial control flowchart picture based on the pre-trained text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content includes:

[0011] Use the convolutional neural network to identify the industrial control flow chart picture, and obtain the coordinate information of multiple text regions in the industrial control flow chart picture and the semantic information of the text in each text region;

[0012] Use the recurrent neural network to process the semantic information of the text in the text region, and obtain the sequential relationship between the texts in the text region;

[0013] Use the classifier to process the semantic information of the text in the text region and the sequential relationship, and obtain the text content in the text region. The coordinate information of the text content is the coordinate information of the text region.

[0014] Optionally, before generating the industrial control flow chart of the second industrial control system by using the flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content, the method further includes:

[0015] Store the text content and the coordinate information of the text content as an information file;

[0016] Import the information file into the flowchart editing software.

[0017] Optionally, the step of using the pre-trained flowchart text recognition model to perform text recognition on the industrial control flow chart picture to obtain the text content in the industrial control flow chart picture and the coordinate information of the text content includes:

[0018] In response to the text recognition operation input by the user, call the flowchart text recognition model through a preset encapsulation interface, so as to perform text recognition on the industrial control flow chart picture based on the flowchart text recognition model, and obtain the text content in the industrial control flow chart picture and the coordinate information of the text content;

[0019] Through the preset encapsulation interface, call a file saving program to store the text content and the coordinate information of the text content as an information file.

[0020] Optionally, before calling the file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as an information file, the method further includes:

[0021] In response to the file saving path selection operation input by the user, use the target path selected by the file saving path selection operation as the storage path;

[0022] Calling the file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as an information file, including:

[0023] Calling the file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as the information file under the storage path.

[0024] In a second aspect, an embodiment of the present application further provides a method for training a flowchart text recognition model, including:

[0025] Obtaining a plurality of sample industrial control flowcharts;

[0026] Converting each sample industrial control flowchart into a picture to obtain a flowchart sample picture corresponding to each sample industrial control flowchart;

[0027] Performing model training based on the plurality of sample industrial control flowcharts and the flowchart sample pictures corresponding to each sample industrial control flowchart to obtain the flowchart text recognition model.

[0028] Optionally, the obtaining a plurality of sample industrial control flowcharts includes:

[0029] Collecting real flowcharts of a plurality of process flows;

[0030] Processing the real flowcharts of the plurality of process flows to obtain a plurality of synthetic flowcharts; the plurality of sample industrial control flowcharts include: the real flowcharts of the plurality of process flows and the plurality of synthetic flowcharts.

[0031] In a third aspect, an embodiment of the present application further provides a flowchart creation device, and the device includes:

[0032] A picture acquisition module, configured to acquire an industrial control flowchart picture of a first industrial control system;

[0033] An identification module, configured to perform text recognition on the industrial control flowchart picture based on a pre-trained flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content;

[0034] A flowchart generation module, configured to generate an industrial control flowchart of the second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content.

[0035] Optionally, the flowchart text recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier; the recognition module includes:

[0036] A convolution recognition unit, which is used to recognize the industrial control flow chart picture by using the convolution neural network, and obtain the coordinate information of multiple text regions in the industrial control flow chart picture and the semantic information of the text in each text region;

[0037] A cyclic recognition unit, which is used to process the semantic information of the text in the text region by using the cyclic neural network, and obtain the sequential relationship between the texts in the text region;

[0038] A classifier recognition unit, which is used to process the semantic information of the text in the text region and the sequential relationship by using the classifier, and obtain the text content in the text region, and the coordinate information of the text content is the coordinate information of the text region.

[0039] Optionally, before the flow chart generation module, the device further includes:

[0040] A storage module, which is used to store the text content and the coordinate information of the text content as an information file;

[0041] An import module, which is used to import the information file into the flow chart editing software.

[0042] Optionally, the recognition module includes:

[0043] A recognition model calling unit, which is used to respond to the text recognition operation input by the user, call the flow chart text recognition model through a preset encapsulation interface, and perform text recognition on the industrial control flow chart picture based on the flow chart text recognition model, so as to obtain the text content in the industrial control flow chart picture and the coordinate information of the text content;

[0044] A save program calling unit, which is used to call a file save program through the preset encapsulation interface, and store the text content and the coordinate information of the text content as an information file.

[0045] Optionally, before the save program calling unit, the device further includes:

[0046] A path selection unit, which is used to respond to the file save path selection operation input by the user, and use the target path selected by the file save path selection operation as the storage path;

[0047] The save program calling unit is specifically used to call the file save program through the preset encapsulation interface, and store the text content and the coordinate information of the text content as the information file under the storage path.

[0048] Fourthly, an embodiment of the present application further provides a computer device, including: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the computer device runs, the processor communicates with the storage medium through the bus. The processor executes the program instructions to perform the steps of the flowchart creation method as described in any of the above embodiments and the steps of the training method of the flowchart text recognition model as described in any of the above embodiments.

[0049] Fifthly, an embodiment of the present application further provides a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is run by a processor, it performs the steps of the flowchart creation method as described in any of the above embodiments and the steps of the training method of the flowchart text recognition model as described in any of the above embodiments.

[0050] The beneficial effects of the present application are:

[0051] The present application provides a flowchart creation method, a model training method, a device, a device, and a medium. Among them, the flowchart creation method includes: obtaining an industrial control flowchart picture of a first industrial control system; performing text recognition on the industrial control flowchart picture based on a pre-trained flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content; and generating an industrial control flowchart of a second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content. The solution of the present application can generate the industrial control flowchart in the second industrial control system by identifying the text content in the industrial control flowchart picture of the first industrial control system and the coordinate information of the text content, without the need for engineers to manually create the industrial control flowchart in the first industrial control system in the second industrial control system, enabling the complex and mechanical work to be automatically completed, saving the drawing time of engineers, and improving work efficiency. Description of the Drawings

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0053] Figure 1 It is a schematic flowchart of a flowchart creation method provided by an embodiment of the present application;

[0054] Figure 2 It is a schematic flowchart of another flowchart creation method provided by an embodiment of the present application;

[0055] Figure 3 It is a schematic flowchart of yet another flowchart creation method provided by an embodiment of the present application;

[0056] Figure 4 It is a schematic flowchart of still another flowchart creation method provided by an embodiment of the present application;

[0057] Figure 5 It is a schematic diagram of an interface of a flowchart text recognition operation interface provided by an embodiment of the present application;

[0058] Figure 6 It is a schematic flowchart of a training method of a flowchart text recognition model provided by an embodiment of the present application;

[0059] Figure 7 It is a schematic structural diagram of a flowchart creation device provided by an embodiment of the present application;

[0060] Figure 8 It is a schematic diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0061] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0063] In addition, the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0064] It should be noted that, without conflict, the features in the embodiments of the present application may be combined with each other.

[0065] In the existing technology, after the industrial control system is replaced or upgraded, engineers need to re-edit various industrial control flowcharts in the old control system in the new control system. This work is complicated and mechanical, and it takes a lot of time of engineers.

[0066] Based on this, the present application intends to provide a flowchart creation method. By using a flowchart text recognition model to perform text recognition on various industrial control flowchart pictures in the old control system, the text content and the coordinate information of the text content in the industrial control flowchart pictures are obtained, and according to the text content and the coordinate information of the text content, corresponding industrial control flowcharts are generated in the new control system, without engineers having to create them manually, saving the time of engineers.

[0067] The following will detail the flowchart creation method and the training method of the flowchart text recognition model provided by the present application.

[0068] Please refer to Figure 1 , which is a schematic flowchart of a flowchart creation method provided by an embodiment of the present application. As Figure 1 shown, the method includes:

[0069] S10: Obtain an industrial control flowchart picture of a first industrial control system.

[0070] In this embodiment, the industrial control system is a system that controls the automation of various industrial production processes in the field of industrial automation. The industrial control system targeted by this embodiment may be a distributed control system (DCS), also known as a distributed control system, which is based on a microprocessor and adopts the control idea of decentralized control, centralized operation, hierarchical management, flexible configuration, and convenient configuration.

[0071] The first industrial control system is an existing system used in the industrial production process. There are industrial control flowcharts for realizing the automation of various industrial production processes in the first industrial control system. The method for obtaining the corresponding industrial control flowchart picture based on the industrial control flowchart in the first industrial control system may be: directly taking a screenshot of the industrial control flowchart, or exporting the industrial control flowchart to a picture format.

[0072] S20: Based on a pre-trained flowchart text recognition model, perform text recognition on the industrial control flowchart picture to obtain the text content and the coordinate information of the text content in the industrial control flowchart picture.

[0073] In this embodiment, the text recognition model is a model for recognizing text information in a picture, and the flowchart text recognition model can recognize the text information in a flowchart picture. Based on the pre-trained flowchart text recognition model, the text content in the industrial control flowchart picture and the coordinate information of the text content in the picture can be recognized.

[0074] S30: According to the text content and the coordinate information of the text content, use the flowchart editing software in the second industrial control system to generate the industrial control flowchart of the second industrial control system.

[0075] In this embodiment, the second industrial control system is the latest system to be adopted in the industrial production process. The second industrial control system can be a completely different system after replacing the first industrial control system, or a new version system after updating the version of the first industrial control system.

[0076] The second industrial control system has flowchart editing software, and the flowchart editing software can edit the corresponding industrial control flowchart for various industrial production processes. The flowchart editing software in this embodiment can directly obtain the text content and the coordinate information of the text content obtained in S20 above, so as to display the corresponding text content at the corresponding position in the editing interface of the flowchart editing software based on the coordinate information of the text content.

[0077] After the text content in the industrial control flowchart picture is displayed on the editing interface of the flowchart editing software, the text boxes of each text content in the industrial control flowchart picture and the connection relationship between each text box can be referred to. Then, the text content can be re-edited with text boxes on the editing interface of the flowchart editing software, and each text box can be connected to generate the industrial control flowchart. Of course, the flowchart line recognition model can also be used to recognize each text box and connection line in the flowchart, obtain each text box and connection line through the flowchart editing software, and combine the text content and the coordinate information of the text content recognized by the flowchart text recognition model to generate the industrial control flowchart of the second industrial control system in the flowchart editing software.

[0078] The embodiment of the present application provides a method for creating a flowchart, including: obtaining an industrial control flowchart picture of a first industrial control system; performing character recognition on the industrial control flowchart picture based on a pre-trained flowchart character recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content; and generating an industrial control flowchart of a second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content. The solution of the embodiment of the present application can generate the industrial control flowchart in the second industrial control system by identifying the text content in the industrial control flowchart picture of the first industrial control system and the coordinate information of the text content, without the need for engineers to manually create the industrial control flowchart in the first industrial control system in the second industrial control system, automating the complex and mechanical work, saving the drawing time of engineers, and improving work efficiency.

[0079] Based on the above embodiment, the flowchart character recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier. The embodiment of the present application will elaborate on the recognition process of the flowchart character recognition model.

[0080] Please refer to Figure 2 , which is a schematic flowchart of another method for creating a flowchart provided by the embodiment of the present application. As Figure 2 shown, in the embodiment of the present application, the method for creating a flowchart includes S10, S21-S23, and S30, where S10 and S30 are the same as those in the above embodiment and will not be elaborated here.

[0081] Specifically, the method for creating a flowchart in the embodiment of the present application includes:

[0082] S10: Obtain an industrial control flowchart picture of a first industrial control system.

[0083] S21: Use a convolutional neural network to recognize the industrial control flowchart picture to obtain the coordinate information of multiple text regions in the industrial control flowchart picture and the semantic information of the text in each text region.

[0084] In this embodiment, after obtaining the industrial control flowchart picture, the industrial control flowchart picture to be recognized is input into a trained convolutional neural network (Convolutional Neural Networks, CNN) to recognize the coordinate information of multiple text regions in the industrial control flowchart picture in the industrial control flowchart picture, and to extract the features of the text in each text region to extract the semantic information of the text. For example, the convolutional neural network in this embodiment can be implemented using the Faster R-CNN model, or the R-CNN or YOLO v3 model.

[0085] S22: Process the semantic information of the text in the text region using a recurrent neural network to obtain the sequential relationship between the texts in the text region.

[0086] In this embodiment, after obtaining the semantic information of the text in each text region, the semantic information of the text in each text region is input into a trained Recurrent Neural Network (RNN) to learn the sequential relationship between the texts, so as to obtain the sequential relationship between the texts in each text region. For example, the recurrent neural network in this embodiment can adopt BLSTM (Bi-directional Long Short-Term Memory, bidirectional long short-term memory neural network). Since there is a strong correlation between the front and back content of the texts, using BLSTM can make the learned sequential relationship between the texts more accurate. Of course, a Seq2Seq model can also be used, and this application does not limit this.

[0087] S23: Use a classifier to process the semantic information of the text and the sequential relationship in the text region to obtain the text content in the text region, and the coordinate information of the text content is the coordinate information of the text region.

[0088] In this embodiment, after obtaining the semantic information of the text in each text region and the sequential relationship between the texts, the semantic information of the text and the sequential relationship between the texts are input into a classifier. The classifier can determine the probabilities of multiple alternative texts corresponding to each text to be recognized based on the semantic information of the text, and combine the sequential relationship between the texts to select the target text from the multiple alternative texts corresponding to each text to be recognized, combine the text sequences of the multiple target texts to obtain the text content in the text region, and use the coordinate information of the text region as the coordinate information of the text content. The classifier outputs the text content of the text region and the coordinate information of the text content.

[0089] S30: According to the text content and the coordinate information of the text content, use the flowchart editing software in the second industrial control system to generate the industrial control flowchart of the second industrial control system.

[0090] The flowchart creation method provided by the embodiments of the present application obtains an industrial control flowchart picture of a first industrial control system, uses a convolutional neural network to identify the industrial control flowchart picture, obtains the coordinate information of multiple text regions in the industrial control flowchart picture and the text semantic information in each text region, uses a recurrent neural network to process the text semantic information in the text regions, obtains the sequential relationship between the texts in the text regions, uses a classifier to process the text semantic information and the sequential relationship in the text regions, obtains the text content in the text regions, the coordinate information of the text content is the coordinate information of the text regions, and according to the text content and the coordinate information of the text content, uses the flowchart editing software in the second industrial control system to generate an industrial control flowchart of the second industrial control system. The solution of the embodiments of the present application can improve the accuracy of text content recognition in industrial control flowchart pictures through a convolutional neural network, a recurrent neural network, and a classifier.

[0091] Based on the above embodiments, the embodiments of the present application further provide a flowchart creation method. Please refer to Figure 3 which is a schematic flowchart of another flowchart creation method provided by the embodiments of the present application. As Figure 3 shown, in the embodiments of the present application, before the above S30, the method further includes:

[0092] S41: Store the text content and the coordinate information of the text content as an information file.

[0093] In the embodiments of the present application, after the flowchart text recognition model recognizes the text content and the coordinate information of the text content, in order to facilitate the flowchart editing software to obtain the text content and the coordinate information of the text content, the text content and the coordinate information of the text content can be stored as an information file in a preset format, and the format of the information file is a format that can be recognized by the flowchart editing software. For example, it can be an information file in csv format.

[0094] S42: Import the information file into the flowchart editing software.

[0095] In the embodiments of the present application, the information file is obtained through the import interface of the flowchart editing software. The flowchart editing software parses the content in the information file to obtain the text content and the coordinate information of the text content in the information file, and according to the coordinate information of the text content, displays the corresponding text content at the corresponding position in the editing interface of the flowchart editing software.

[0096] The flowchart creation method provided by the embodiments of the present application stores the text content and the coordinate information of the text content as an information file, and imports the information file into the flowchart editing software. The solution provided by this embodiment stores the text content and the coordinate information of the text content as an information file, which is convenient for unified management of the text content and the coordinate information of the text content, and is also convenient for the flowchart editing software to import, avoiding errors during the import process, and improving the accuracy and efficiency of creating a flowchart.

[0097] Based on the above embodiments, the embodiments of the present application further provide a flowchart creation method. Please refer to Figure 4 , which is a schematic flowchart of still another flowchart creation method provided by the embodiments of the present application. As Figure 4 shown, in the embodiments of the present application, the above S20 includes:

[0098] S24: In response to the text recognition operation input by the user, call the flowchart text recognition model through a preset encapsulation interface, so as to perform text recognition on the industrial control flowchart picture based on the flowchart text recognition model, and obtain the text content and the coordinate information of the text content in the industrial control flowchart picture.

[0099] In this embodiment, for the convenience of the operation of engineering personnel, a programming language can be used to write an operation interface to generate a flowchart text recognition operation interface. On the flowchart text recognition operation interface, the pre-trained flowchart text recognition model and the file saving program are encapsulated as interfaces, which is convenient for calling the flowchart text recognition model and the file saving program through the interface based on the operation on the flowchart text recognition operation interface.

[0100] Exemplarily, please refer to Figure 5 , which is an interface schematic diagram of a flowchart text recognition operation interface provided by the embodiments of the present application. Taking Figure 5 as an example, the flowchart text recognition process of the embodiments of the present application is described, but Figure 5 the flowchart text recognition operation interface shown cannot be the only limitation of the flowchart text recognition operation interface of the embodiments of the present application.

[0101] As Figure 7 shown, in response to the picture acquisition path selection operation input by the user, obtain the industrial control flowchart picture of the first industrial control system from the picture acquisition path. After obtaining the industrial control flowchart picture to be recognized, the user inputs a text recognition operation on the flowchart text recognition operation interface. In response to this text recognition operation, call the flowchart text recognition model through a preset encapsulation interface to recognize the text content and the coordinate information of the text content in the industrial control flowchart picture. The specific recognition process can refer to the above S21 - S23, which will not be elaborated here.

[0102] S25: Call a file saving program through a preset encapsulation interface to store the text content and the coordinate information of the text content as an information file.

[0103] In this embodiment, after the flowchart text recognition model finishes recognition, the flowchart text recognition operation interface calls a file saving program through a preset encapsulation interface to store the text content and the coordinate information of the text content as an information file.

[0104] In an alternative embodiment, in response to a file saving path selection operation input by the user, use the target path selected by the file saving path selection operation as the storage path, and call a file saving program through a preset encapsulation interface to store the text content and the coordinate information of the text content as an information file under the storage path.

[0105] In this embodiment, the user can input a file saving path selection operation in the flowchart text recognition operation interface before or after the recognition starts. This file saving path selection operation is used to select the storage location of the information file and generate a corresponding storage path based on this storage location. After determining the storage path, call the file saving path to store the text content and the coordinate information of the text content as an information file under the storage path.

[0106] The flowchart creation method provided by the embodiments of this application responds to a text recognition operation input by the user, calls a flowchart text recognition model through a preset encapsulation interface to perform text recognition on an industrial control flowchart picture based on the flowchart text recognition model, obtains the text content and the coordinate information of the text content in the industrial control flowchart picture, and calls a file saving program through a preset encapsulation interface to store the text content and the coordinate information of the text content as an information file. The solution of the embodiments of this application can realize the automatic recognition of industrial control flowchart pictures and the storage of information files through the flowchart text recognition operation interface, which is convenient for engineers to operate.

[0107] Based on the above flowchart creation method, the embodiments of this application also provide a training method for a flowchart text recognition model. Please refer to Figure 6 , which is a schematic flowchart of a training method for a flowchart text recognition model provided by the embodiments of this application. As Figure 6 shown, this method includes:

[0108] S50: Obtain multiple sample industrial control flowcharts.

[0109] In this embodiment, in order to train the initial flowchart text recognition model, it is necessary to collect multiple sample industrial control flowcharts. The sample industrial control flowcharts can include flowcharts with text information of different process flows in different industrial fields such as the stone or chemical industry.

[0110] In an alternative embodiment, true flowcharts of multiple process flows can be collected and processed to obtain multiple synthetic flowcharts; the multiple sample industrial control flowcharts include: the true flowcharts of multiple process flows and multiple synthetic flowcharts.

[0111] Specifically, the true flowchart is a flowchart with text information of different process flows in different industrial fields. By performing operations such as font transformation, text area deformation, blurring, adding noise, and / or adding a background on the true flowchart, a synthetic flowchart is obtained. Obtaining a synthetic flowchart by processing the true flowchart can enrich the quantity and types of the sample industrial control flowcharts, making the flowchart text recognition model trained more robust.

[0112] S60: Convert each sample industrial control flowchart into a picture to obtain a flowchart sample picture corresponding to each sample industrial control flowchart.

[0113] In this embodiment, to train the text recognition function of the model for pictures, it is necessary to convert each sample industrial control flowchart into a picture. The method for obtaining the flowchart sample picture corresponding to each sample industrial control flowchart can be: directly taking a screenshot of each sample industrial control flowchart, or exporting each sample industrial control flowchart into a picture format, and this application does not limit this here.

[0114] S70: Perform model training based on the multiple sample industrial control flowcharts and the flowchart sample pictures corresponding to each sample industrial control flowchart to obtain a flowchart text recognition model.

[0115] In this embodiment, input the multiple sample industrial control flowcharts and the flowchart sample pictures corresponding to each sample industrial control flowchart into an initial flowchart text recognition model, and adjust the parameters of the model according to the training results of the model until the training results meet the requirements to obtain a flowchart text recognition model.

[0116] The training method of the flowchart text recognition model provided by the embodiments of this application obtains multiple sample industrial control flowcharts, converts each sample industrial control flowchart into a picture to obtain a flowchart sample picture corresponding to each sample industrial control flowchart, and performs model training based on the multiple sample industrial control flowcharts and the flowchart sample pictures corresponding to each sample industrial control flowchart to obtain a flowchart text recognition model. The solution of the embodiments of this application trains the model to obtain a flowchart text recognition model for application in the flowchart creation method to improve the creation efficiency of the flowchart.

[0117] Based on the above embodiments, the embodiments of this application also disclose a virtual device for a flowchart creation method. Please refer to Figure 7, which is a schematic structural diagram of a flowchart creation device provided by an embodiment of the present application. As Figure 7 shown, the device includes:

[0118] An image acquisition module 10, configured to acquire an industrial control flowchart image of a first industrial control system;

[0119] An identification module 20, configured to perform character recognition on the industrial control flowchart image based on a pre-trained flowchart character recognition model, and obtain the text content in the industrial control flowchart image and the coordinate information of the text content;

[0120] A flowchart generation module 30, configured to generate an industrial control flowchart of a second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content.

[0121] Optionally, the flowchart character recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier; the identification module 20 includes:

[0122] A convolutional recognition unit, configured to recognize the industrial control flowchart image by using a convolutional neural network, and obtain the coordinate information of multiple text regions in the industrial control flowchart image and the text semantic information in each text region;

[0123] A recurrent recognition unit, configured to process the text semantic information in the text region by using a recurrent neural network, and obtain the sequential relationship between the characters in the text region;

[0124] A classifier recognition unit, configured to process the text semantic information and the sequential relationship in the text region by using a classifier, and obtain the text content in the text region, and the coordinate information of the text content is the coordinate information of the text region.

[0125] Optionally, the device further includes:

[0126] A storage module, configured to store the text content and the coordinate information of the text content as an information file;

[0127] An import module, configured to import the information file into the flowchart editing software.

[0128] Optionally, the identification module 20 includes:

[0129] An identification model calling unit, configured to respond to a character recognition operation input by a user, call the flowchart character recognition model through a preset encapsulation interface, and perform character recognition on the industrial control flowchart image based on the flowchart character recognition model, so as to obtain the text content in the industrial control flowchart image and the coordinate information of the text content;

[0130] A save program calling unit, configured to call a file saving program through a preset encapsulation interface to store the text content and the coordinate information of the text content as an information file.

[0131] Optionally, the apparatus further includes:

[0132] A path selection unit, configured to respond to a file saving path selection operation input by a user, and use the target path selected by the file saving path selection operation as the storage path;

[0133] The save program calling unit is specifically configured to call a file saving program through a preset encapsulation interface to store the text content and the coordinate information of the text content as an information file under the storage path.

[0134] The above apparatus is used to execute the method provided in the foregoing embodiment, and its implementation principle and technical effects are similar, and will not be elaborated here.

[0135] The above modules may be one or more integrated circuits configured to implement the above method, for example: one or more Application Specific Integrated Circuits (ASICs), or, one or more microprocessors, or, one or more Field Programmable Gate Arrays (FPGAs), etc. Again, when a certain module above is implemented in the form of a processing element dispatching program code, the processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processors that can call program code. Again, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0136] Please refer to Figure 8 , which is a schematic diagram of a computer device provided by an embodiment of the present application. As Figure 8 shown, the computer device 100 includes: a processor 101, a storage medium 102, and a bus. The storage medium 102 stores program instructions executable by the processor 101. When the computer device 100 runs, the processor 101 communicates with the storage medium 102 through the bus, and the processor 101 executes the program instructions to execute the foregoing method embodiment. The specific implementation manner and technical effects are similar and will not be elaborated here.

[0137] Optionally, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the foregoing method embodiment. The specific implementation manner and technical effects are similar and will not be elaborated here.

[0138] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0139] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware, or in the form of hardware plus software functional units.

[0141] The above integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above software functional units stored in a storage medium include several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to execute some steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (English: Read-Only Memory, abbreviated as: ROM), random access memories (English: Random Access Memory, abbreviated as: RAM), magnetic disks or optical discs that can store program codes.

[0142] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for creating a flowchart, characterized in that, The method includes: Obtain an industrial control flowchart picture of a first industrial control system; Perform text recognition on the industrial control flowchart picture based on a pre-trained flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content; the flowchart text recognition model is trained and obtained according to multiple sample industrial control flowcharts and the flowchart sample pictures corresponding to the sample industrial control flowcharts, and the sample industrial control flowcharts include: real flowcharts of multiple process flows and multiple synthetic flowcharts; wherein, the real flowcharts are flowcharts with text information of different process flows in different industrial fields; According to the text content and the coordinate information of the text content, use the flowchart editing software in the second industrial control system to generate an industrial control flowchart of the second industrial control system; The flowchart text recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier; the performing text recognition on the industrial control flowchart picture based on the pre-trained text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content includes: Use the convolutional neural network to recognize the industrial control flowchart picture to obtain the coordinate information of multiple text regions in the industrial control flowchart picture and the text semantic information in each text region; Use the recurrent neural network to process the text semantic information in the text region to obtain the sequential relationship between the words in the text region; Use the classifier to process the text semantic information and the sequential relationship in the text region to obtain the text content in the text region, and the coordinate information of the text content is the coordinate information of the text region; The classifier determines the probabilities of multiple candidate words corresponding to each word to be recognized based on the text semantic information, and combines the sequential relationship between the words to select the target word from the multiple candidate words corresponding to each word to be recognized, and combines the word sequences of the multiple target words to obtain the text content in the text region.

2. The method according to claim 1, characterized in that Before the using the flowchart editing software in the second industrial control system to generate an industrial control flowchart of the second industrial control system according to the text content and the coordinate information of the text content, the method further includes: Store the text content and the coordinate information of the text content as an information file; Import the information file into the flowchart editing software.

3. The method according to claim 1, wherein The performing text recognition on the industrial control flowchart picture based on the pre-trained flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content includes: In response to a text recognition operation input by a user, call the flowchart text recognition model through a preset encapsulation interface to perform text recognition on the industrial control flowchart picture based on the flowchart text recognition model to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content; Call a file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as an information file.

4. The method according to claim 3, wherein Before calling a file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as an information file, the method further includes: In response to a file saving path selection operation input by a user, use the target path selected by the file saving path selection operation as the storage path; The step of calling a file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as an information file includes: Call the file saving program through the preset encapsulation interface to store the text content and the coordinate information of the text content as the information file under the storage path.

5. A training method for a flowchart text recognition model, characterized in that, Includes: Obtain a plurality of sample industrial control flowcharts; Convert each sample industrial control flowchart into a picture to obtain a flowchart sample picture corresponding to each sample industrial control flowchart; Perform model training based on the plurality of sample industrial control flowcharts and the flowchart sample pictures corresponding to the sample industrial control flowcharts to obtain the flowchart text recognition model. The flowchart text recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier. After an industrial control flowchart picture is input into the flowchart text recognition model, the convolutional neural network is used to recognize the industrial control flowchart picture to obtain the coordinate information of multiple text regions in the industrial control flowchart picture and the text semantic information in each text region; the recurrent neural network is used to process the text semantic information in the text region to obtain the sequential relationship between the words in the text region; the classifier is used to process the text semantic information and the sequential relationship in the text region to obtain the text content in the text region, and the coordinate information of the text content is the coordinate information of the text region; determine the probabilities of multiple alternative words corresponding to each word to be recognized based on the text semantic information, and combine the sequential relationship between the words to select a target word from the multiple alternative words corresponding to each word to be recognized, and perform a combination of the word sequences of the multiple target words to obtain the text content in the text region; The step of obtaining a plurality of sample industrial control flowcharts includes: Collect real flowcharts of a plurality of process flows, where the real flowcharts are flowcharts containing text information of different process flows in different industrial fields; Process the real flowcharts of the plurality of process flows to obtain a plurality of synthesized flowcharts. The plurality of sample industrial control flowcharts include: the real flowcharts of the plurality of process flows and the plurality of synthesized flowcharts.

6. A flowchart creation device, characterized in that, The device includes: A picture acquisition module, configured to acquire an industrial control flowchart picture of a first industrial control system; An identification module, configured to perform text recognition on the industrial control flowchart picture based on a pre-trained flowchart text recognition model, to obtain the text content in the industrial control flowchart picture and the coordinate information of the text content; the flowchart text recognition model is obtained by training according to a plurality of sample industrial control flowcharts and flowchart sample pictures corresponding to the sample industrial control flowcharts, and the sample industrial control flowcharts include: real flowcharts of a plurality of process flows, and a plurality of synthetic flowcharts; wherein, the real flowcharts are flowcharts containing text information of different process flows in different industrial fields; A flowchart generation module, configured to generate an industrial control flowchart of the second industrial control system by using a flowchart editing software in the second industrial control system according to the text content and the coordinate information of the text content; The flowchart text recognition model includes: a convolutional neural network, a recurrent neural network, and a classifier; The identification module is further configured to use the convolutional neural network to recognize the industrial control flowchart picture, to obtain the coordinate information of a plurality of text regions in the industrial control flowchart picture and the word semantic information in each text region; use the recurrent neural network to process the word semantic information in the text region, to obtain the sequential relationship between the words in the text region; use the classifier to process the word semantic information and the sequential relationship in the text region, to obtain the text content in the text region, and the coordinate information of the text content is the coordinate information of the text region; the classifier determines the probabilities of a plurality of alternative words corresponding to each word to be recognized based on the word semantic information, and combines the sequential relationship between the words, selects the target word from the plurality of alternative words corresponding to each word to be recognized, and combines the word sequences of the plurality of target words to obtain the text content in the text region.

7. A computer device, characterized in that, Including: A processor, a storage medium, and a bus, the storage medium stores program instructions executable by the processor, when the computer device runs, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to perform the steps of the flowchart creation method according to any one of claims 1 to 4 and the steps of the training method of the flowchart text recognition model according to any one of claims 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is run by the processor, it performs the steps of the flowchart creation method according to any one of claims 1 to 4 and the steps of the training method of the flowchart text recognition model according to any one of claims 5.

Images