Multimodal industrial control system and method

Abstract

This invention provides a multimodal industrial control system and method, belonging to the field of industrial control technology. The multimodal industrial control system includes: a first controller group, a second controller group, a data acquisition module, and industrial field devices. The data acquisition module is used to acquire multiple types of signals from the industrial field devices using various communication methods. The first controller group is used to process first-type signals transmitted from the data acquisition module to generate first control commands. The second controller group is used to process second-type signals transmitted from the data acquisition module to generate second control commands, and transmits the second control commands to the first controller group. The first controller group is also used to generate control command signals based on the first and second control commands. This invention classifies the interactive signals of industrial field devices into first-type signals and second-type signals. By having the first controller group process the first-type signals and the second controller group process the second-type signals, it achieves classified processing of multiple types of signals, improving controller processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of industrial control technology, and more specifically to a multimodal industrial control system and a multimodal industrial control method. Background Technology

[0002] With the development of industrial automation and intelligence, the signals that industrial control systems need to access and process are becoming increasingly complex. Controllers not only need to access and process traditional industrial standard input and output signals, but also need to access and process signals from wireless industrial sensors, as well as multi-dimensional signals such as audio, video, and vibration from the industrial field.

[0003] Existing industrial control systems often process multiple types of signals through a single controller, and these signals are input to the controller irregularly, which can easily lead to low processing efficiency. Furthermore, existing industrial control systems can only process traditional industrial standard input and output signals, and there is no systematic processing method for signals from wireless industrial sensors, as well as audio, video, and vibration signals from industrial sites. Summary of the Invention

[0004] To address the aforementioned technical deficiencies, this invention proposes a multimodal industrial control system and method. The multimodal industrial control system classifies the interactive signals from industrial field equipment into a first type of signal and a second type of signal. The first type of signal is processed by a first controller group, and the second type of signal is processed by a second controller group, achieving classified processing of multiple signal types and improving controller processing efficiency. The multimodal industrial control system can acquire and process industrial input and output signals from industrial field equipment, as well as video, audio, and vibration signals, thereby enhancing the system's ability and efficiency in handling industrial production processes.

[0005] The first aspect of the present invention provides a multimodal industrial control system, comprising: a first controller group, a second controller group, a data acquisition module, and industrial field equipment;

[0006] The acquisition module is used to acquire various signals from industrial field equipment using multiple communication methods;

[0007] The first controller group is used to process a first type of signal transmitted from the acquisition module and generate a first control command based on the first type of signal;

[0008] The second controller group is used to process the second type of signal transmitted from the acquisition module, generate a second control command based on the second type of signal, and transmit the second control command to the first controller group;

[0009] The first controller group is also used to generate control command signals according to the first control instruction and the second control instruction;

[0010] The first type of signal includes industrial input signals and industrial output signals of industrial field equipment; the second type of signal includes, but is not limited to, video signals, audio signals and vibration signals of industrial field equipment.

[0011] In this embodiment of the invention, the acquisition module includes a first acquisition unit, a second acquisition unit, and a third acquisition unit;

[0012] The first acquisition unit is connected to the first controller group and the industrial field equipment via wired communication, and is used to acquire the first type of signal from the industrial field equipment and send the first type of signal to the first controller group.

[0013] The second acquisition unit is connected to the second controller group and the industrial field equipment via wireless communication, and is used to acquire the second type of signals from the industrial field equipment and send the second type of signals to the second controller group;

[0014] The third acquisition unit is connected to the second controller group and the industrial field equipment via wired or wireless communication, and is used to acquire the second type of signals from the industrial field equipment and send the second type of signals to the second controller group.

[0015] The second controller group is also used to send the received first type of signal to the first controller group.

[0016] In this embodiment of the invention, the system further includes:

[0017] The execution module is used to receive control command signals from the first controller group and control the industrial field equipment to perform control actions according to the control command signals.

[0018] In this embodiment of the invention, the system further includes:

[0019] A mobile terminal is used to acquire status information of industrial field equipment via wireless communication and generate mobile control commands based on the status information of the industrial field equipment.

[0020] In this embodiment of the invention, the second controller group receives a mobile control command from the mobile terminal and transmits the mobile control command to the first controller group;

[0021] The first controller group generates control command signals based on the movement control command, the first control command, and the second control command.

[0022] In this embodiment of the invention, the mobile terminal communicates with the second controller group via near-field communication to obtain the status information of the industrial field equipment stored in the second controller group, and generates mobile control commands based on the obtained status information of the industrial field equipment.

[0023] In this embodiment of the invention, the mobile terminal identifies the identification information of industrial field equipment through radio frequency technology and sends the identified identification information to the second controller group;

[0024] The second controller group determines the status information of the industrial field device based on the identification information and sends the status information of the industrial field device to the mobile terminal.

[0025] The mobile terminal generates mobile control commands based on the status information of the industrial field equipment.

[0026] In this embodiment of the invention, the mobile terminal is also used to collect video signals from industrial field equipment and send the collected video signals to the second controller group;

[0027] The second controller group determines the status information of the industrial field equipment based on the video signal and sends the status information of the industrial field equipment to the mobile terminal.

[0028] The mobile terminal generates mobile control commands based on the status information of the industrial field equipment.

[0029] In this embodiment of the invention, the system further includes: a server terminal, the server terminal comprising:

[0030] The device mapping unit is used to map industrial field devices and their corresponding identification information and video signals to form and store an industrial field device mapping relationship table.

[0031] The second controller group determines the industrial field device corresponding to the flag information or video signal sent by the mobile terminal in the industrial field device mapping table, and obtains the status information of the determined industrial field device.

[0032] In this embodiment of the invention, the server terminal further includes:

[0033] The neural network model training unit is used to provide trained algorithm parameters for the first controller group and the second controller group.

[0034] The first controller group performs logical operations on the first type of signal based on the algorithm parameters output by the neural network model training unit to generate the first control command;

[0035] The second controller group performs logical operations on the second type of signal based on the algorithm parameters output by the neural network model training unit, and generates the second control command.

[0036] In this embodiment of the invention, the second controller group is provided with a data processing neural network model, which is used to preprocess the second type of signal acquired by the acquisition module;

[0037] The neural network model training unit is also used to provide the second controller group with the model parameters of the trained data processing neural network model;

[0038] The data processing neural network model performs data preprocessing on the acquired second type of signal based on the model parameters output by the neural network model training unit.

[0039] In this embodiment of the invention, the system further includes: a wired terminal and a network switch group;

[0040] The network switch group is used to connect the first controller group, the second controller group, wired terminals, mobile terminals and server terminals to the network.

[0041] The wired terminal includes:

[0042] The monitoring and control unit is used to monitor the first controller group and the second controller group in real time.

[0043] The configuration unit is used to assign tasks to the first controller group and the second controller group.

[0044] A second aspect of the present invention provides a multimodal industrial control method, comprising:

[0045] The acquisition module collects various signals from industrial field equipment using multiple communication methods;

[0046] The first controller group processes the first type of signal transmitted from the acquisition module and generates a first control command based on the first type of signal.

[0047] The second controller group processes the second type of signal transmitted from the acquisition module, generates a second control command based on the second type of signal, and transmits the second control command to the first controller group.

[0048] The first controller group generates control command signals based on the first control command and the second control command.

[0049] The multimodal industrial control system classifies the interactive signals of industrial field equipment into a first type of signal and a second type of signal. The first type of signal is processed by a first controller group and the second type of signal is processed by a second controller group, thereby realizing the classification and processing of multiple types of signals and improving the processing efficiency of the controller.

[0050] Other features and advantages of the technical solution of the present invention will be described in detail in the following detailed embodiments section. Attached Figure Description

[0051] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0052] Figure 1 This is a structural block diagram of the multimodal industrial control system provided in the embodiments of the present invention;

[0053] Figure 2 This is a structural block diagram of a multimodal industrial control system for processing the first type of signal provided in an embodiment of the present invention;

[0054] Figure 3 This is a structural block diagram of a multimodal industrial control system for processing the second type of signal provided in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the task allocation structure of the multimodal industrial control system provided in an embodiment of the present invention;

[0056] Figure 5 This is a flowchart of the multimodal industrial control method provided in the embodiments of the present invention.

[0057] Explanation of reference numerals in the attached figures

[0058] 1-First controller group, 2-Second controller group, 3-First acquisition unit, 4-Second acquisition unit, 5-Third acquisition unit, 6-Industrial field equipment, 7-Execution module, 8-Mobile terminal, 9-Network switch group, 10-Wired terminal, 11-Monitoring and control unit, 12-Configuration unit, 13-Electronic tag, 14-Camera, 15-Server terminal, 16-Neural network model training unit, 17-Device mapping unit. Detailed Implementation

[0059] To make the technical solutions and advantages of the embodiments of the present invention clearer, the exemplary embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0060] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0062] In this invention, unless otherwise explicitly specified and limited, terms such as "installation," "connection," "linking," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0063] In the process of realizing this invention, the inventors discovered that with the development of industrial automation and intelligence, the signals that industrial control systems need to access and process are becoming increasingly complex. The controller not only needs to access and process traditional industrial standard input and output signals, but also needs to access and process signals from wireless industrial sensors, as well as multi-dimensional signals such as audio, video and vibration from the industrial site.

[0064] Existing industrial control systems often process multiple types of signals through a single controller, and these signals are input to the controller irregularly, which can easily lead to low processing efficiency. Furthermore, existing industrial control systems can only process traditional industrial standard input and output signals, and there is no systematic processing method for signals from wireless industrial sensors, as well as audio, video, and vibration signals from industrial sites.

[0065] To address the aforementioned problems, this invention provides a multimodal industrial control system, comprising: a first controller group, a second controller group, a data acquisition module, and industrial field devices. The data acquisition module is used to acquire multiple types of signals from the industrial field devices using various communication methods. The first controller group is used to process a first type of signal transmitted from the data acquisition module and generate a first control command based on the first type of signal. The second controller group is used to process a second type of signal transmitted from the data acquisition module, generate a second control command based on the second type of signal, and transmit the second control command to the first controller group. The first controller group is also used to generate a control command signal based on the first and second control commands. The first type of signal includes: industrial input signals and industrial output signals from the industrial field devices. The second type of signal includes, but is not limited to: video signals, audio signals, and vibration signals from the industrial field devices. The multimodal industrial control system provided by this invention classifies the interactive signals of the industrial field devices into first and second types of signals. By processing the first type of signal through the first controller group and the second type of signal through the second controller group, it achieves classified processing of multiple types of signals, improving the controller processing efficiency. The multimodal industrial control system can collect and process industrial input and output signals from industrial field equipment, as well as video, audio, and vibration signals. It also features a mobile monitoring method that can quickly identify, locate, and monitor industrial equipment, thereby improving the system's ability and efficiency in handling production processes in the industrial field.

[0066] This invention improves information processing and control levels in industrial settings by accessing and analyzing multimodal signals. The "touch-to-use" function enables screenless devices and systems to have screens and small screens to be enlarged, enhancing the human-machine interface of the control system. Video recognition facilitates convenient monitoring of field equipment, improving control efficiency in industrial settings.

[0067] Figure 1 This is a structural block diagram of the multimodal industrial control system provided in an embodiment of the present invention. Figure 1 As shown, the first aspect of the present invention provided in this embodiment provides a multimodal industrial control system, including: a first controller group 1, a second controller group 2, a data acquisition module, and industrial field equipment 6;

[0068] The acquisition module is used to acquire various signals from industrial field equipment 6 using multiple communication methods;

[0069] The first controller group 1 is used to process the first type of signal transmitted from the acquisition module and generate a first control command based on the first type of signal;

[0070] The second controller group 2 is used to process the second type of signal transmitted from the acquisition module, generate a second control command based on the second type of signal, and transmit the second control command to the first controller group 1;

[0071] The first controller group 1 is also used to generate control command signals according to the first control instruction and the second control instruction.

[0072] In this embodiment, the first type of signal includes industrial input signals and industrial output signals of industrial field equipment, and the second type of signal includes video signals, audio signals and vibration signals of industrial field equipment.

[0073] Specifically, industrial input signals and industrial output signals include 24V / 220V digital input / output signals, 4~20mA / 0~10V analog input / output signals, RTD (Resistor Tolerance Device) signals, etc.

[0074] Figure 2 This is a structural block diagram of a multimodal industrial control system for processing the first type of signal provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of a multimodal industrial control system for processing the second type of signal, provided in an embodiment of the present invention; as shown below. Figure 2-3 As shown:

[0075] In this embodiment, the acquisition module includes a first acquisition unit 3, a second acquisition unit 4, and a third acquisition unit 5;

[0076] The first acquisition unit 3 is connected to the first controller group 1 and the industrial field device 6 via wired communication, and is used to acquire the first type of signal from the industrial field device 6 and send the first type of signal to the first controller group 1.

[0077] The second acquisition unit 4 is connected to the second controller group 2 and the industrial field device 6 via wireless communication, and is used to acquire the first type of signal and the second type of signal of the industrial field device 6, and send the first type of signal and the second type of signal to the second controller group 2.

[0078] The third acquisition unit 5 is connected to the second controller group 2 and the industrial field device 6 via wired or wireless communication, and is used to acquire the second type of signal from the industrial field device 6 and send the second type of signal to the second controller group 2.

[0079] The second controller group 2 is also used to send the received first type of signal to the first controller group 1.

[0080] Furthermore, the first acquisition unit 3 is connected to the industrial field equipment 6 and the first controller group 1 via wired communication, and is used to acquire the first type of signal and transmit it to the first controller group 1.

[0081] The second acquisition unit 4 is connected to the industrial field equipment 6 and the second controller group 2 via wireless communication. It is used to acquire the vibration signal in the first type of signal and the second type of signal, and to transmit the acquired first type of signal and vibration signal to the second controller group 2.

[0082] The third acquisition unit 5 is connected to the industrial field equipment 6 and the second controller group 2 via wired or wireless communication, and is used to acquire the second type of signal and transmit it to the second controller group 2.

[0083] Specifically, the first acquisition unit 3 connects to the traditional industrial sensors of the industrial field equipment 6, and acquires the first type of signals transmitted by the traditional industrial sensors. The second acquisition module connects to the wireless sensors of the industrial field equipment 6, and the second acquisition unit 4 acquires the first type of signals and vibration signals transmitted by the wireless sensors. The third acquisition unit 5 acquires audio, video, and vibration signals from the industrial field transmitted by devices such as industrial cameras and acoustic radar.

[0084] In this embodiment, the system further includes:

[0085] The execution module 7 is used to receive control command signals from the first controller group 1 and control the industrial field equipment 6 to perform control actions according to the control command signals.

[0086] The execution module 7 receives the control command signal from the first controller group 1 and outputs industrial input / output signals to the industrial field equipment 6 via wired communication.

[0087] Furthermore, the first controller group 1 may consist of one or more industrial controllers, responsible for processing industrial input and output signals and performing related logical operations and outputs. The second controller group 2 may consist of one or more industrial controllers, responsible for processing audio, video, and vibration signals and performing related logical operations and outputs. Data interaction occurs within the first controller group 1, within the second controller group 2, and between the first controller group 1 and the second controller group 2; the results of their logical operations can be mutually accessed.

[0088] In this embodiment, the system further includes:

[0089] Mobile terminal 8 is used to acquire the status information of industrial field equipment 6 via wireless communication and generate mobile control commands based on the status information of industrial field equipment 6.

[0090] Specifically, the mobile terminal 8 is connected to the second controller group 2 via wireless communication, and the mobile terminal 8 obtains the status information of the industrial field equipment 6 from the second controller group 2.

[0091] In this embodiment, the mobile terminal 8 and the second controller group 2 are connected via near-field communication.

[0092] In this embodiment, the second controller group 2 receives a mobile control command from the mobile terminal 8 and transmits the mobile control command to the first controller group 1;

[0093] The first controller group 1 generates a control command signal based on the movement control command, the first control command, and the second control command.

[0094] In this embodiment, the mobile terminal 8 communicates with the second controller group 2 via near-field communication to obtain the status information of the industrial field device 6 stored in the second controller group 2, and generates mobile control commands based on the obtained status information of the industrial field device 6.

[0095] In this embodiment, the mobile terminal 8 identifies the identification information of the industrial field equipment 6 through radio frequency technology and sends the identified identification information of the industrial field equipment 6 to the second controller group 2;

[0096] The second controller group 2 determines the status information of the industrial field device 6 based on the identification information of the industrial field device 6, and sends the status information of the industrial field device 6 to the mobile terminal 8;

[0097] The mobile terminal 8 generates a mobile control command based on the status information of the industrial field equipment 6.

[0098] In this embodiment, the mobile terminal 8 is also used to collect video signals from the industrial field equipment 6 and send the collected video signals from the industrial field equipment 6 to the second controller group 2;

[0099] The second controller group 2 identifies the corresponding industrial field device 6 based on the video signal of the industrial field device 6, determines the status information of the industrial field device 6, and sends the status information of the industrial field device 6 to the mobile terminal 8.

[0100] The mobile terminal 8 generates a mobile control command based on the status information of the industrial field equipment 6.

[0101] Specifically, the mobile terminal 8 integrates a near-field communication chip and a wireless communication chip. In this embodiment, the mobile terminal 8 uses radio frequency identification (RFID) technology for near-field communication. The mobile terminal 8 achieves real-time monitoring of the first controller group 1 and the second controller group 2 through a wireless network, and realizes the "tap-to-click" function through near-field communication. The mobile terminal 8 is equipped with sensors such as a camera 14, and can realize relevant monitoring functions through video recognition of the industrial site.

[0102] Furthermore, the mobile terminal 8 "touches" the industrial controller, and the second controller group 2 identifies the corresponding industrial controller that is in contact with the mobile terminal 8, obtaining the status information of the equipment and system controlled by that controller. The second controller group 2 transmits the obtained information to the mobile terminal 8 via near-field communication. The control commands from the mobile terminal 8 are transmitted to the first controller group 1 via the second controller group 2. The first controller group 1 performs relevant logical operations and outputs control commands to the execution module 7 to complete system-level control.

[0103] The specific workflow of the mobile terminal 8 "tap-tap" industrial controller is as follows:

[0104] The second controller group 2 acquires the status information of the equipment or system controlled by the corresponding industrial controller;

[0105] The second controller group 2 transmits the acquired information to the mobile terminal 8 via near-field communication;

[0106] Mobile terminal 8 implements system-level monitoring functions. Mobile control commands issued by mobile terminal 8 are transmitted to the second controller group 2 via near-field communication or wireless communication.

[0107] The second controller group 2 transmits the movement control command to the first controller group 1 in a data interaction manner, and the first controller group 1 performs logical operations.

[0108] The first controller group 1 outputs control command signals to the execution module 7, and the execution module 7 outputs industrial input / output signals to the relevant field equipment.

[0109] The mobile terminal 8 "tap" onto the field device, and the second controller group 2 identifies the industrial field device 6 and obtains its status information. The second controller group 2 transmits the acquired information to the mobile terminal 8 via near-field communication. The mobile terminal 8's movement control commands are transmitted to the first controller group 1 via the second controller group 2. The first controller group 1 performs relevant logic operations and outputs control commands to the execution module 7 to complete device-level control.

[0110] The specific workflow of the mobile terminal 8 "tap-tap" to industrial field equipment 6 is as follows:

[0111] Mobile terminal 8 can "tap" industrial field device 6 to identify the identification information of the field device through radio frequency technology;

[0112] The second controller group 2 receives the identification information of the industrial field device 6 sent from the mobile terminal 8, interacts with the first controller group 1, and searches for and obtains the status information of the field device's identification.

[0113] The second controller group 2 transmits the status information of the industrial field device 6 to the mobile terminal 8 via near-field communication;

[0114] Mobile terminal 8 implements device-level monitoring functions. Mobile control commands issued by mobile terminal 8 are transmitted to the second controller group 2 through near-field communication or wireless communication.

[0115] The second controller group 2 transmits the movement control command to the first controller group 1 in a data interaction manner, and the first controller group 1 performs logical operations.

[0116] The first controller group 1 outputs control command signals to the execution module 7, and the execution module 7 outputs industrial input / output signals to the relevant field equipment.

[0117] The mobile terminal 8 integrates a camera 14 and other video acquisition devices, capable of acquiring video signals from the industrial field equipment 6. The mobile terminal 8 transmits the acquired video signals to the second controller group 2 via wireless communication. The second controller group 2 processes and analyzes the received video signals, identifies the industrial field equipment 6 corresponding to the video signals, and obtains the status information of the industrial field equipment 6.

[0118] The second controller group 2 sends the acquired status information of the industrial field device 6 to the mobile terminal 8. The mobile terminal 8 issues a mobile control command based on the status information of the industrial field device 6, which is then transmitted via the second controller group 2 to the first controller group 1. The first controller group 1 performs logical operations and outputs control command signals to the execution module 7.

[0119] Electronic tags 13 are installed on the second controller group 2 and the industrial field equipment 6 in the industrial site. The unique identification information of the device can be obtained through the electronic tag 13, which is used to locate the device.

[0120] In this embodiment, the system further includes: a server terminal 15, the server terminal 15 comprising:

[0121] The device mapping unit 17 is used to map the industrial field device 6 and the identification information and video signal corresponding to the industrial field device 6 to form a mapping relationship table of industrial field device 6 and store it.

[0122] The second controller group 2 determines the industrial field device 6 corresponding to the flag information or video signal sent by the mobile terminal 8 in the mapping table of industrial field devices 6, and obtains the status information of the determined industrial field device 6.

[0123] In this embodiment, the server terminal 15 further includes:

[0124] The neural network model training unit 16 is used to provide trained algorithm parameters for the first controller group 1 and the second controller group 2.

[0125] The first controller group 1 performs logical operations on the first type of signal according to the algorithm parameters output by the neural network model training unit 16 to generate the first control command;

[0126] The second controller group 2 performs logical operations on the second type of signal based on the algorithm parameters output by the neural network model training unit 16, and generates the second control command.

[0127] In this embodiment, the second controller group 2 is equipped with a data processing neural network model, which is used to preprocess the second type of signal acquired by the acquisition module;

[0128] Specifically, the acquisition module transmits the second type of signal to the second controller group 2, inputs the second type of signal into the data processing neural network model, and the data processing neural network model outputs the preprocessed second type of signal.

[0129] The neural network model training unit 16 is also used to provide the trained data processing neural network model with model parameters for the second controller group 2;

[0130] The data processing neural network model performs data preprocessing on the acquired second type of signal based on the model parameters output by the neural network model training unit 16.

[0131] Figure 4 This is a schematic diagram of the task allocation structure of the multimodal industrial control system provided in an embodiment of the present invention, as shown below. Figure 4 As shown, in this embodiment, the system further includes a wired terminal 10 and a network switch group 9. The wired terminal 10 is communicatively connected to the first controller group 1 and the second controller group 2, the wired terminal 10, the mobile terminal 8, and the server terminal 15 through the network switch group 9.

[0132] The wired terminal 10 includes:

[0133] The monitoring and control unit 11 is used to monitor the first controller group 1 and the second controller group 2 in real time.

[0134] Configuration unit 12 is used to assign tasks to the first controller group 1 and the second controller group 2.

[0135] In this embodiment, the network switch group 9 supports various network topologies for industrial control networks, including star, ring, bus, and hybrid topologies.

[0136] Figure 5 This is a flowchart of the multimodal industrial control method provided in an embodiment of the present invention. For example... Figure 5 As shown, the second aspect of the present invention provided in this embodiment offers a multimodal industrial control method, comprising:

[0137] S1. Acquire various signals from industrial field equipment 6 through the acquisition module using multiple communication methods;

[0138] S2. Process the first type of signal transmitted from the acquisition module through the first controller group 1, and generate a first control command based on the first type of signal;

[0139] S3. The second type of signal transmitted from the acquisition module is processed by the second controller group 2, a second control command is generated based on the second type of signal, and the second control command is transmitted to the first controller group 1;

[0140] S4. The first controller group 1 generates a control command signal based on the first control instruction and the second control instruction;

[0141] S5. The execution module 7 receives control command signals from the first controller group 1 and controls the industrial field equipment 6 to perform control actions according to the control command signals.

[0142] In this embodiment, the method further includes:

[0143] The acquisition module includes a first acquisition unit 3, a second acquisition unit 4, and a third acquisition unit 5;

[0144] The first acquisition unit 3 is connected to the first controller group 1 and the industrial field device 6 via wired communication, and is used to acquire the first type of signal from the industrial field device 6 and send the first type of signal to the first controller group 1.

[0145] The second acquisition unit 4 is connected to the second controller group 2 and the industrial field device 6 via wireless communication, and is used to acquire the first type of signal and the second type of signal of the industrial field device 6, and send the first type of signal and the second type of signal to the second controller group 2.

[0146] The third acquisition unit 5 is connected to the second controller group 2 and the industrial field device 6 via wired or wireless communication, and is used to acquire the second type of signal from the industrial field device 6 and send the second type of signal to the second controller group 2.

[0147] The second controller group 2 is also used to send the received first type of signal to the first controller group 1.

[0148] In this embodiment, the method further includes:

[0149] The mobile terminal 8 communicates with the second controller group 2 via near-field communication to obtain the status information of the industrial field equipment 6 controlled by the second controller group 2 stored in the second controller group 2; and generates mobile control commands based on the obtained status information.

[0150] The second controller group 2 receives the mobile control command output from the mobile terminal 8 and outputs the mobile control command to the first controller group 1.

[0151] The first controller group 1 is also used to generate control command signals based on the movement control command, the first control command, and the second control command.

[0152] In this embodiment, the method further includes:

[0153] The mobile terminal 8 uses radio frequency technology to identify the industrial field device 6 and sends the identification information of the identified industrial field device 6 to the second controller group 2.

[0154] The second controller group 2 obtains the status information of the industrial field device 6 based on the received identification information and sends the status information of the industrial field device 6 to the mobile terminal 8.

[0155] The mobile terminal 8 generates mobile control commands based on the received status information of the industrial field device 6.

[0156] In this embodiment, the method further includes:

[0157] The mobile terminal 8 collects video signals from the industrial field equipment 6 and sends the collected video signals to the second controller group 2.

[0158] The second controller group 2 identifies the corresponding industrial field device 6 through the video signal, obtains the status information of the industrial field device 6, and sends the status information of the industrial field device 6 to the mobile terminal 8.

[0159] The mobile terminal 8 generates mobile control commands based on the received status information of the industrial field device 6.

[0160] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0161] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0162] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. Furthermore, it should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. As long as such combination does not violate the spirit of the embodiments of the present invention, it should also be considered as the content disclosed by the embodiments of the present invention.

Claims

1. A multimodal industrial control system, characterized in that, include: The system comprises: a first controller group, a second controller group, a data acquisition module, industrial field equipment, and a mobile terminal. The acquisition module is used to acquire various signals from industrial field equipment using multiple communication methods; The first controller group is used to process a first type of signal transmitted from the acquisition module and generate a first control command based on the first type of signal. The first type of signal includes: industrial input signals and industrial output signals of industrial field equipment. The second controller group is used to process the second type of signals transmitted from the acquisition module, generate a second control command based on the second type of signals, and transmit the second control command to the first controller group. The second type of signals includes: video signals, audio signals, and vibration signals of industrial field equipment. The mobile terminal integrates a near-field communication chip and a wireless communication chip. The mobile terminal communicates with industrial field equipment via the near-field communication chip and communicates with the second controller group via the wireless communication chip or the near-field communication chip. The mobile terminal identifies the identification information of industrial field equipment through near-field communication and sends the identification information to the second controller group through wireless communication or near-field communication. The second controller group obtains the status information of the industrial field equipment from the first controller group based on the received identification information, and sends the status information of the industrial field equipment to the mobile terminal via wireless communication or near-field communication. The mobile terminal generates mobile control commands based on the status information of the industrial field equipment. The first controller group is also configured to generate control command signals based on the first control instruction, the second control instruction from the second controller group, and the mobile control instruction from the mobile terminal.

2. The multimodal industrial control system according to claim 1, characterized in that, The acquisition module includes a first acquisition unit, a second acquisition unit, and a third acquisition unit; The first acquisition unit is connected to the first controller group and the industrial field equipment via wired communication, and is used to acquire the first type of signal from the industrial field equipment and send the first type of signal to the first controller group. The second acquisition unit is connected to the second controller group and the industrial field equipment via wireless communication, and is used to acquire the first type of signal and the second type of signal from the industrial field equipment, and send the first type of signal and the second type of signal to the second controller group; The third acquisition unit is connected to the second controller group and the industrial field equipment via wired or wireless communication, and is used to acquire the second type of signals from the industrial field equipment and send the second type of signals to the second controller group. The second controller group is also used to send the received first type of signal to the first controller group.

3. The multimodal industrial control system according to claim 1, characterized in that, The system also includes: The execution module is used to receive control command signals from the first controller group and control the industrial field equipment to perform control actions according to the control command signals.

4. The multimodal industrial control system according to claim 1, characterized in that, The second controller group receives a mobile control command from the mobile terminal and transmits the mobile control command to the first controller group.

5. The multimodal industrial control system according to claim 1, characterized in that, The mobile terminal is also used to collect video signals from industrial field equipment and send the collected video signals to the second controller group; The second controller group determines the status information of the industrial field equipment based on the video signal and sends the status information of the industrial field equipment to the mobile terminal. The mobile terminal generates mobile control commands based on the status information of the industrial field equipment.

6. The multimodal industrial control system according to claim 5, characterized in that, The system further includes: a server terminal, the server terminal comprising: The device mapping unit is used to map industrial field devices and their corresponding identification information and video signals to form and store an industrial field device mapping relationship table. The second controller group determines the industrial field device corresponding to the flag information or video signal sent by the mobile terminal in the industrial field device mapping table, and obtains the status information of the determined industrial field device.

7. The multimodal industrial control system according to claim 6, characterized in that, The server terminal also includes: The neural network model training unit is used to provide trained algorithm parameters for the first controller group and the second controller group. The first controller group performs logical operations on the first type of signal based on the algorithm parameters output by the neural network model training unit to generate the first control command; The second controller group performs logical operations on the second type of signal based on the algorithm parameters output by the neural network model training unit, and generates the second control command.

8. The multimodal industrial control system according to claim 7, characterized in that, The second controller group is equipped with a data processing neural network model, which is used to preprocess the second type of signal acquired by the acquisition module; The neural network model training unit is also used to provide the second controller group with the model parameters of the trained data processing neural network model; The data processing neural network model performs data preprocessing on the acquired second type of signal based on the model parameters output by the neural network model training unit.

9. The multimodal industrial control system according to claim 7, characterized in that, The system also includes: wired terminals and network switch groups; The network switch group is used to connect the first controller group, the second controller group, wired terminals, mobile terminals and server terminals to the network. The wired terminal includes: The monitoring and control unit is used to monitor the first controller group and the second controller group in real time. The configuration unit is used to assign tasks to the first controller group and the second controller group.

10. A multimodal industrial control method, characterized in that, include: The acquisition module collects various signals from industrial field equipment using multiple communication methods; The first controller group processes the first type of signals transmitted from the acquisition module and generates a first control command based on the first type of signals. The first type of signals includes: industrial input signals and industrial output signals of industrial field equipment. The second controller group processes the second type of signals transmitted from the acquisition module, generates a second control command based on the second type of signals, and transmits the second control command to the first controller group. The second type of signals includes: video signals, audio signals, and vibration signals of industrial field equipment. The identification information of industrial field equipment is identified by a mobile terminal using near-field communication, and the identification information is sent to the second controller group using wireless communication or near-field communication. The second controller group obtains the status information of the industrial field equipment from the first controller group based on the received identification information, and sends the status information of the industrial field equipment to the mobile terminal through wireless communication or near-field communication. The mobile terminal generates mobile control commands based on the status information of the industrial field equipment. The first controller group generates control command signals based on the first control command, the second control command from the second controller group, and the mobile control command from the mobile terminal.

Images