Perception-transmission-control fusion-oriented industrial semantic communication method, system, equipment and medium

Abstract

The invention relates to a perception-transmission-control fusion-oriented industrial semantic communication method, system and device and a medium, and the system comprises a to-be-observed system which builds a cloud side-end collaborative industrial Internet of Things system and is used for collecting multi-modal data in an industrial scene; the industrial semantic communication sending end and the industrial semantic communication receiving end are used for realizing semantic feature extraction, identification and efficient transmission of industrial data, namely multi-modal data in an industrial scene; and the feedback controller is used for generating a control signal according to the optimized feedback control logic and acting on the industrial semantic communication system. The invention provides a new mode to ensure efficient communication between machines and improve the semantic interoperation capability of an industrial heterogeneous system, so as to support intelligent man-machine-object interaction of M2M, H2M, H2H and the like.

Description

Technical Field

[0001] The present invention relates to an industrial semantic communication method, system, equipment and medium oriented to perception-transmission-control fusion, and belongs to the technical field of industrial semantic communication. Background Art

[0002] With the deployment of 5G / 5G-A cellular communication systems, artificial intelligence (AI) models, and cloud-edge-end collaborative network architectures in industrial scenarios, the convergence of communication, computing, and control has become a core driver for the digital, networked, and intelligent transformation and upgrade of next-generation industrial network systems. However, the separate design of perception, transmission, and control poses significant challenges to industrial network systems, resulting in poor interoperability and inefficient collaboration among heterogeneous industrial systems. In particular, for a host of new scenarios such as predictive maintenance, product quality, intelligent control of production processes and process parameters, intelligent decision-making for production scheduling, and virtual fusion of digital twins, the need for comprehensive and accurate perception of system status, robust and efficient data transmission, and agile feedback control of systems has become extremely urgent. The integrated design of perception, transmission, and control has become key to advancing industrial intelligence.

[0003] Semantic communication and efficient, task-oriented semantic data transmission have become key technologies for future 6G cellular communications. In the industrial sector, a host of factors, including the extremely fragmented nature of industrial scenarios and specific requirements, the interdependencies of machine communication tasks, and the difficulty in accurately identifying the semantics of massive amounts of industrial data, pose significant challenges to efficient industrial semantic transmission. Furthermore, machine-to-machine communication in industrial scenarios differs from human communication. Its multi-source, heterogeneous, unstructured, and complex and sparse nature makes it difficult to accurately and quantitatively extract semantic features from high-dimensional industrial data. Furthermore, existing industrial datasets are often unavailable for training deep neural networks for industrial semantic communication systems, and most industrial data is time series data, requiring complex data analysis techniques for semantic extraction and labeling. Summary of the Invention

[0004] To address the shortcomings of the existing technology, the present invention provides an Intelligent Sensing, Transmission, and Control with Industrial Semantic Communications (ISTC-ISC) method for the fusion of perception, transmission and control to support efficient industrial data transmission and agile control of industrial Internet of Things devices and networks.

[0005] The present invention focuses on solving the problems of semantic recognition, efficient transmission, and joint design of feedback control of complex industrial data, aiming to provide an industrial semantic communication method, system, equipment and medium for perception-transmission-control fusion, so as to provide a new way to ensure efficient communication between machines and improve the semantic interoperability of industrial heterogeneous systems, thereby supporting intelligent interaction between people, machines and objects such as M2M (Machine-to-Machine), H2M (Human-to-Machine), and H2H (Human-to-Human).

[0006] Explanation of terms:

[0007] 1. CBAM Module: The Convolutional Block Attention Module (CBAM) is a lightweight, pluggable, and universal module designed to enhance the representation capabilities of Convolutional Neural Networks (CNN) models. It uses an attention mechanism to enhance important features and suppress unimportant ones. The CBAM module consists of two parts: the Channel Attention module and the Spatial Attention module. It improves model performance by adaptively focusing on information in the spatial and channel dimensions.

[0008] 2. ResNet model: ResNet (Residual Network) is a deep convolutional neural network architecture that introduces residual blocks and directly adds input data to the convolution output through skip connections. It learns the "residual" between input and output instead of directly learning complex function mappings, effectively improving the feature learning ability of data.

[0009] The technical solution of the present invention is:

[0010] Industrial semantic communication system for perception-transmission-control integration, including:

[0011] The system to be observed builds a cloud-edge-end collaborative industrial IoT system to collect multimodal data in industrial scenarios;

[0012] Industrial semantic communication transmitter and receiver, used to extract, identify and efficiently transmit the semantic features of industrial data, i.e., multimodal data in industrial scenarios;

[0013] The feedback controller is used to generate a control signal according to an optimized feedback control logic and act on the industrial semantic communication system.

[0014] According to the present invention, the transmitting end preferably includes an industrial semantic recognition network and a deep compressed sensing encoder;

[0015] Industrial semantic recognition network, used to identify input data samples x k Perform semantic recognition and classification; including:

[0016] Extract fixed-length input data samples x from real-time industrial data streams k , as the data input for a single process; use the industrial semantic recognition network to identify each input data sample x k Classify the data into different categories based on the classification results. The data is divided into 0-n categories according to the importance from low to high. For the data of high importance, that is, the data of categories 1-n, it is transmitted through deep compressed sensing codec; for the data of low importance, that is, the data of category 0, a semantic flag S is sent to the receiving end;

[0017] The deep compressed sensing encoder semantically extracts and compresses the data of high importance to obtain the compressed data, and the sender transmits the compressed data through the channel.

[0018] Further preferably, the semantic recognition and classification process includes: data collection, data preprocessing, semantic classification and data transmission;

[0019] Data acquisition: extract fixed-length input data samples x from real-time industrial data streams k ;

[0020] Data preprocessing: input data sample x k Perform standardization, denoising, and completion processing in sequence;

[0021] Semantic classification: The pre-processed data is fed into the industrial semantic recognition network for semantic recognition and classification into different categories.

[0022] The industrial semantic recognition network includes a residual module, a CBAM module, and a fully connected layer. The residual module includes a primary convolutional layer, a multi-level convolutional layer, and a residual block. The primary convolutional layer includes a convolutional layer and a ReLU activation function, and the multi-level convolutional layer includes a convolutional layer, a batch normalization layer, and a ReLU activation function. The CBAM module includes channel attention and spatial attention.

[0023] The preprocessed data is passed through a primary convolutional layer and two multi-level convolutional layers for convolution and ReLU activation operations, and then enters the CBAM module through a residual block. The feature map extracted by the residual block is mapped to the CBAM module. Each CBAM module uses maximum pooling and average pooling for feature compression in turn, and then calculates the channel and spatial importance through convolution and ReLU activation functions. Finally, the most important part of the feature map is enhanced through weighted operations. The feature output after weighted operation is convolved and the feature expression is further optimized through the Gamma weighting method. The weights of each part of the feature are adjusted to form the final feature representation. The final feature representation enters the fully connected layer, and the feature map is finally classified.

[0024] Data transmission: For data of high importance, i.e., data of categories 1-n, it is transmitted through deep compressed sensing encoding and decoding; for data of low importance, i.e., data of category 0, a semantic flag S is sent to the receiving end.

[0025] Further preferably, semantic extraction and compression are performed on the data of high importance; including: data input, feature extraction, compressed sensing, and compressed data output;

[0026] Data input: Input the important data into the deep compressed sensing encoder;

[0027] Feature extraction: Using deep learning models to extract high-level semantic features from important data, and learning the complex patterns and semantic information of the data through stacked neural networks;

[0028] Compressed sensing: Compress data by applying compressed sensing technology and convert it into a low-dimensional feature vector, i.e. compressed data;

[0029] Compressed data output: Send the compressed data to the receiving end.

[0030] Preferably, according to the present invention, the receiving end includes a deep compressed sensing decoder and an industrial AI model:

[0031] The deep compressed sensing decoder maps the compressed data received from the transmitter into reconstructed data and reconstructs the semantic information of the original data;

[0032] Industrial AI models perform intelligent manufacturing, predictive maintenance, intelligent decision-making, and other cloud-based industrial applications based on the semantic information of the recovered original data. They transmit control information to the feedback controller through the semantic / data recovery quality or AI prediction and reasoning results to perform intelligent control and decision-making on the industrial system.

[0033] Further preferably, the deep compressed sensing decoder maps the compressed data received from the transmitter into reconstructed data, and reconstructs the semantic information of the original data; including: data input, sparse reconstruction and feature reconstruction, and data output;

[0034] Data input: The compressed data is input to the deep compressed sensing decoder;

[0035] Sparse reconstruction and feature reconstruction: Apply compressed sensing technology to reconstruct the coefficients of compressed data;

[0036] Use deep neural networks to further recover high-order semantic information in the data and reconstruct the semantic information of the original data;

[0037] Data output: The deep compressed sensing decoder outputs the semantic information of the reconstructed original data.

[0038] Further preferably, the industrial AI model includes a prediction model, intelligent decision-making, and control information output;

[0039] Prediction model: The industrial AI model analyzes the semantic information of the raw data to build a prediction model, which is used to predict equipment failures and monitor health. It uses supervised or unsupervised learning methods to analyze whether the equipment has potential failures and predict the remaining service life of the equipment.

[0040] Intelligent decision-making: providing optimized decisions based on prediction results;

[0041] Control information output: The control information is transmitted to the industrial control system, and automatic control is achieved through the feedback of the control information.

[0042] According to the preferred embodiment of the present invention, the feedback controller includes a controller and an actuator; the controller generates an adaptive control signal through feedback control logic and optimizes the actuator input signal in combination with the control gain factor; the actuator receives the control signal and acts on the semantic / data transmission processing process.

[0043] Further preferably, the optimized control signal is sent to the actuator to adjust the state of the industrial semantic communication system; including:

[0044] Dynamically adjust the weights of reconstruction error, bandwidth allocation, and computing resource consumption based on the needs and real-time requirements of different tasks. By optimizing the compression ratio, bandwidth allocation strategy, and the activation matrix of the perception process, the system ensures that the optimal balance between resource consumption and reconstruction error is achieved.

[0045] Alternatively, by generating control instructions, key parameters in the industrial semantic communication network, including data transmission rate, bandwidth allocation, transmission delay and computing overhead, are adjusted to adjust the execution of the industrial semantic communication network, optimize the data transmission process, and achieve the best performance of the system.

[0046] Preferably, according to the present invention, the multimodal semantic knowledge graph provides a comprehensive and queryable knowledge base by fusing, aligning and unifying data and semantics of different modalities, supporting the analysis, reasoning and decision-making of multimodal data;

[0047] Construct a multimodal semantic knowledge graph: First, collect multimodal data; then, extract semantic features from the multimodal data; fuse the semantic features from the multimodal data and transform them into a unified representation; finally, organize the extracted multimodal semantic features and relationships into a knowledge graph structure; the nodes of the graph represent entities, and the edges represent the relationships between entities.

[0048] The industrial semantic communication method for perception-transmission-control fusion runs on an industrial semantic communication system for perception-transmission-control fusion, including:

[0049] Step 101: Build a cloud-edge-end collaborative industrial IoT system to support state perception, data collection, transmission, and feedback control in an industrial network environment.

[0050] Step 102: Training an industrial semantic recognition network for extracting, identifying, and classifying complex industrial data semantics;

[0051] Step 103: Train a deep compressed sensing encoder and deploy it at the transmitter and receiver of the industrial semantic communication system to complete the transmission and reconstruction of complex industrial data and semantics;

[0052] Step 104: With the joint goal of minimizing system consumption and data reconstruction error, the data compression ratio and bandwidth allocation strategy of the perception and transmission processes are adaptively optimized and adjusted, and the data compression ratio of the encoding and decoding process is adaptively adjusted with the help of the controller and actuator.

[0053] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned industrial semantic communication method for perception-transmission-control fusion when executing the computer program.

[0054] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned industrial semantic communication method for perception-transmission-control fusion.

[0055] The beneficial effects of the present invention are:

[0056] The present invention provides an industrial semantic communication method, system, equipment and medium for the integration of perception, transmission and control, which is used to support the integration of intelligent perception, transmission and control in cloud-edge collaborative industrial networks. A cloud-edge collaborative network architecture is constructed for the deployment of industrial semantic communication systems to support state perception, data acquisition, transmission and feedback control in industrial network environments; an industrial semantic recognition network is trained for the extraction, recognition and classification of complex industrial data semantics; a deep compressed perception codec network is trained and deployed at the transmitting and receiving ends of the industrial semantic communication system to complete the transmission and reconstruction of complex industrial data and semantics. With the joint goal of minimizing system consumption and data reconstruction errors, key factors such as the data compression ratio and bandwidth allocation strategy of the perception and transmission processes are adaptively optimized and adjusted. With the help of controllers and actuators, the data compression ratio of the coding and decoding process is adaptively adjusted, effectively reducing the amount of data transmitted in the system, optimizing communication and computing resources, and improving the efficiency of the intelligent perception-transmission-control closed-loop system in the industrial Internet of Things network.

[0057] This paper proposes an industrial semantic communication method, system, device, and medium for perception-transmission-control fusion, supporting state perception, data collection, transmission, and feedback control in industrial network environments. The key points are:

[0058] 1. Different from the basic mode of direct data collection and transmission or direct semantic extraction and transmission of general industrial data transmission systems or semantic communication systems, the present invention adds an industrial semantic recognition network before semantic transmission, trains the industrial semantic recognition network for semantic recognition and classification of complex industrial data, and assists the semantic extraction, transmission and reconstruction process of the data based on the semantic information of the recognition and classification results.

[0059] 2. Unlike general industrial data transmission systems or semantic communication systems that use conventional data compression or deep learning methods to reduce data transmission volume, the present invention constructs a cloud-edge collaborative network architecture for the deployment of industrial semantic communication methods and systems, trains a deep compressed sensing codec network based on deep learning and compressed sensing, and deploys it at the sending and receiving ends as the core module of the industrial semantic communication system to complete the extraction, transmission and reconstruction of complex industrial data and semantics.

[0060] 3. Unlike traditional data compression transmission systems or semantic communication systems with fixed compression ratios, the industrial semantic communication method and system of the present invention adapts to the complex needs of different industrial scenarios, with the joint goal of minimizing system consumption and data reconstruction errors. With the help of controllers and actuators, the data compression ratios of the perception and transmission processes are adaptively optimized and adjusted to form a closed-loop architecture of perception-transmission-control, effectively reducing system resource consumption and ensuring the efficiency of data semantic transmission and reconstruction quality.

[0061] 4. Unlike traditional industrial data transmission systems that rely solely on a single data transmission model for data processing and transmission, the present invention uses the object-oriented mechanism of the OPC UA unified information model to comprehensively model industrial data in typical scenarios, sort out methods, variables and events related to industrial semantic communication, and use technologies such as entity-relationship-entity triple tables to construct a multimodal semantic knowledge graph for establishing a semantic knowledge base shared by the sending and receiving ends of industrial semantic communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flowchart of the industrial semantic communication method for perception-transmission-control fusion of the present invention;

[0063] Figure 2 This is an architectural diagram of the industrial semantic communication system for perception-transmission-control fusion according to the present invention;

[0064] Figure 3 This is an architectural diagram of the industrial semantic recognition network of the present invention;

[0065] Figure 4 Architectural diagram for the design of the robust and interpretable deep compressed sensing codec and the construction of the industrial semantic communication system. DETAILED DESCRIPTION

[0066] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0067] Example 1

[0068] Industrial semantic communication systems for perception-transmission-control fusion, such as Figure 2 As shown, including:

[0069] Aiming at the systems to be observed in actual industrial scenarios, we build a cloud-edge-end collaborative industrial IoT system to collect multimodal data in industrial scenarios, such as text, images, videos, sounds, and various sensors.

[0070] Industrial semantic communication transmitter and receiver, used to extract, identify and efficiently transmit the semantic features of industrial data, i.e., multimodal data in industrial scenarios;

[0071] The industrial semantic communication system collects observed system status information via the IIoT network. Using the industrial semantic recognition network, it identifies distinct semantic information within the industrial data. This information is then encoded using a semantic encoder and transmitted via a wireless channel to a semantic decoder at the receiving end. This perception of the observed system enables instant judgment of system status, assisting in decision-making during subsequent transmission. By understanding the semantic characteristics of the transmitted data, less important data does not need to be transmitted to the receiving end, saving valuable communication and computing resources. Deep compressed sensing methods are used to transmit valid data in a compressed state, minimizing the amount of data exchanged between the sender and receiver. This approach not only reduces transmission latency but also optimizes network resource utilization.

[0072] The process of building an industrial semantic communication system includes the design of a robust and interpretable deep compressed sensing semantic codec, training and industrial semantic knowledge graph, and the construction of a semantic annotation dataset. Figure 4 As shown, the specific process has the following characteristics:

[0073] 1. Collect multimodal data (text, images, videos, sounds, sensors, etc.) from industrial environments, comprehensively model industrial data in typical scenarios with the object-oriented mechanism of the OPCUA unified information model, sort out methods, variables, and events related to industrial semantic communication, and implement unified modeling of industrial data using an object-oriented approach.

[0074] 2. Segment the long-scale time series data that have been uniformly modeled and contain rich industrial implicit knowledge into time segments of fixed length. With the help of a dual-driven approach of physical information and data, construct and design an industrial semantic feature extraction and recognition network to realize the implicit semantic mining of industrial data and improve the interpretability of the neural network with the help of physical mechanism information.

[0075] 3. With the help of technologies such as entity-relationship-entity triple tables, build an industrial semantic knowledge graph. With the help of technologies such as graph databases and vector databases, use the industrial semantic knowledge graph to establish a semantic knowledge base shared by the sender and receiver of industrial semantic communication, form semantic labels and semantic annotation data sets, and improve the semantic information reasoning capabilities of deep compressed sensing encoding and decoding networks and industrial AI models.

[0076] 4. Using semantically annotated datasets to train deep compressed sensing codec networks in industrial semantic communication systems, semantic noise of varying intensities is artificially added to the signal source and channel to improve the robustness of the deep compressed sensing codec networks.

[0077] The feedback controller is used to generate a control signal according to an optimized feedback control logic and act on the industrial semantic communication system.

[0078] Example 2

[0079] The industrial semantic communication system for perception-transmission-control integration described in Example 1 is different in that:

[0080] The sending end includes an industrial semantic recognition network and a deep compressed sensing encoder;

[0081] Industrial semantic recognition network, used to identify input data samples x k Perform semantic recognition and classification; e.g. Figure 3 As shown, including:

[0082] Extract fixed-length (or adjustable-length) input data samples x from real-time industrial data streams k , as the data input for a single process; use the industrial semantic recognition network to identify each input data sample x k Classification is performed, and data is divided into different categories according to the classification results. Data is divided into 0-n categories according to importance from low to high (n can be adjusted according to actual data). The larger the value, the higher the importance. For data with high importance, that is, data of categories 1-n, it is transmitted through deep compressed sensing encoding and decoding; for data with low importance, that is, data of category 0, a short semantic flag S is sent to the receiving end; without the need for a deep compressed sensing model to compress and transmit the data, thereby greatly reducing the amount of data transmitted in the industrial Internet of Things network.

[0083] The deep compressed sensing encoder semantically extracts and compresses the data of high importance to obtain the compressed data, and the sender transmits the compressed data through the channel.

[0084] The semantic recognition and classification process includes: data collection, data preprocessing, semantic classification and data transmission;

[0085] Data acquisition: extract fixed-length (adjustable length) input data samples x from real-time industrial data streams k Extract fixed-length (adjustable-length) input data samples x from real-time industrial data streams k ,These data samples may include sensor data, operation status information, etc., collected from different devices (such as sensors, controllers, etc.).

[0086] Data preprocessing: input data sample x k Perform standardization, denoising, and completion processing in sequence;

[0087] Semantic classification: The pre-processed data is fed into the industrial semantic recognition network for semantic recognition and classification into different categories.

[0088] The industrial semantic recognition network includes a residual module, a CBAM module, and a fully connected layer. The residual module includes a primary convolutional layer, a multi-level convolutional layer, and a residual block. The primary convolutional layer includes a convolutional layer and a ReLU activation function, and the multi-level convolutional layer includes a convolutional layer, a batch normalization layer, and a ReLU activation function. The CBAM module includes channel attention and spatial attention.

[0089] The preprocessed data is passed through a primary convolution layer and two multi-level convolution layers for convolution and ReLU activation operations, respectively, to perform layer-by-layer feature extraction and alleviate the problem of gradient disappearance, and then enter the CBAM module through a residual block; the feature map extracted by the residual block is mapped to the CBAM module, and each CBAM module uses maximum pooling and average pooling in turn for feature compression, and then calculates the channel and spatial importance through convolution and ReLU activation functions, and finally enhances the most important part of the feature map through weighted operations; each CBAM module acts independently on different feature layers to enhance feature selectivity; the feature output after weighted operation is convolved, and the feature expression is further optimized through the Gamma weighting method, and the weights of each part of the feature are adjusted to form the final feature representation; the final feature representation enters the fully connected layer, and the feature map is classified into the final result;

[0090] Data transmission: For data of high importance, i.e., data of categories 1-n, it is transmitted through deep compressed sensing encoding and decoding; for data of low importance, i.e., data of category 0, a short semantic flag S is sent to the receiving end.

[0091] Compared to the proposed TWM-Net, the Industrial Semantic Recognition Network improves upon this by replacing the traditional multi-head attention module with the CBAM module. The CBAM module introduces a more sophisticated attention mechanism, independently learning attention in the channel and spatial dimensions to enhance the representational capabilities of feature outputs. Unlike the traditional multi-head attention mechanism, the CBAM module models channel attention through global average pooling and global maximum pooling, and uses a spatial attention mechanism to adaptively focus on more important semantic regions in the data, enhancing the model's accuracy and efficiency in industrial semantic recognition and classification tasks.

[0092] Semantic extraction and compression of highly important data; including: data input, feature extraction, compressed sensing, and compressed data output;

[0093] Data input: Input the important data into the deep compressed sensing encoder;

[0094] Feature extraction: Using deep learning models such as deep neural networks or convolutional neural networks, high-level semantic features are extracted from important data, and the complex patterns and semantic information of the data are learned through stacked neural networks. Stacked neural networks stack multiple network layers, allowing each layer to learn more complex features and semantic information based on the previous layer. This allows the network to capture important semantic information of the data and effectively model the complex patterns of the data. For example, the ResNet model enables the network to learn deep feature representations through its unique residual connections. Each layer not only learns the features input from the previous layer, but the residual connections also allow information to be directly transferred between different layers, thus avoiding the vanishing gradient problem common in deep networks.

[0095] Compressed sensing: By applying compressed sensing technology, data is compressed and converted into low-dimensional feature vectors, namely compressed data; this reduces the redundancy in the data, retains key information and reduces the data dimension.

[0096] Compressed data output: Send the compressed data to the receiving end.

[0097] The receiving end includes a deep compressed sensing decoder and an industrial AI model:

[0098] The deep compressed sensing decoder maps the compressed data received from the transmitter into reconstructed data and reconstructs the semantic information of the original data;

[0099] Industrial AI models perform intelligent manufacturing, predictive maintenance, intelligent decision-making, and other cloud-based industrial applications based on the semantic information of the recovered original data. They transmit control information to the feedback controller through the semantic / data recovery quality or AI prediction and inference results to perform intelligent control and decision-making of the industrial system.

[0100] The deep compressed sensing decoder is a system used in conjunction with the deep compressed sensing encoder. The deep compressed sensing decoder maps the compressed data received from the transmitter into reconstructed data and reconstructs the semantic information of the original data. It includes: data input, sparse reconstruction and feature reconstruction, and data output.

[0101] Data input: The compressed data is input to the deep compressed sensing decoder;

[0102] Sparse reconstruction and feature reconstruction: Apply compressed sensing technology to reconstruct the coefficients of compressed data; based on the compressed data, use deep neural networks to further recover the high-order semantic information in the data and reconstruct the semantic information of the original data;

[0103] Data output: The deep compressed sensing decoder outputs the semantic information of the reconstructed original data. The output semantic information of the reconstructed original data is sent to the industrial AI model along with the semantic flag.

[0104] Industrial AI models include prediction models, intelligent decision-making, and control information output;

[0105] Predictive model: The industrial AI model analyzes the semantic information of the raw data to build a predictive model. The predictive model uses historical equipment data, operating status, and other information to predict equipment failures and monitor health. It uses supervised or unsupervised learning methods, such as deep learning, support vector machines (SVMs), and random forests, to analyze whether equipment has potential failures and predict the remaining useful life of the equipment.

[0106] Intelligent Decision-Making: After performing predictive maintenance, industrial AI models provide optimized decisions based on the predicted results. For example, if a device predicts an impending failure, the model will provide maintenance recommendations and schedule an engineer for an inspection. Industrial AI models can also optimize production scheduling, automatically adjusting equipment operating status and process parameters based on real-time data and production needs.

[0107] Control information output: Finally, based on the decision output of the industrial AI model, the control information is transmitted to the industrial control system. Through the feedback of the control information, automated control is achieved, ensuring the stability and efficiency of the industrial operation process.

[0108] The feedback controller includes a controller and an actuator; the controller generates an adaptive control signal through feedback control logic and optimizes the actuator input signal in combination with the control gain factor; the actuator receives the control signal and acts on the semantic / data transmission processing process.

[0109] The quality of semantic information recovered from compressed data by a deep compressed sensing decoder (i.e., the magnitude of the reconstruction error) and the degree of match with the original signal serve as the basis for optimizing control signal generation. This is combined with industrial AI models to perform equipment fault prediction, performance evaluation, health monitoring, and production scheduling reasoning based on data analysis and historical information, generating prediction results and decision outputs. Based on the deviation between the system's current state and the target state, the controller generates optimized control signals, sends these optimized control signals to the actuators, and adjusts the state of the industrial semantic communication system. By adjusting key factors such as the compression ratio, bandwidth allocation strategy, and the compressed sensing activation matrix, the control signal can achieve optimal performance while minimizing the total system cost (including transmission cost, computational cost, and control error).

[0110] Send optimized control signals to actuators to adjust the state of the industrial semantic communication system; including:

[0111] Based on the needs and real-time requirements of different tasks, the system dynamically adjusts the weights of reconstruction error, bandwidth allocation, and computing resource consumption. By optimizing key factors such as the compression ratio, bandwidth allocation strategy, and the activation matrix of the perception process, the system ensures that the system finds the optimal balance between resource consumption and reconstruction error, thereby minimizing both system consumption and reconstruction error. For example, for low-precision, low-real-time tasks such as temperature sensors, a higher compression ratio can be used, accepting a larger reconstruction error, thereby saving bandwidth and computing resources. For high-precision, high-real-time tasks such as robotic control systems, the system reduces the compression ratio to reduce reconstruction error, while prioritizing more bandwidth and computing resources to ensure low latency and data accuracy. Using prior knowledge, the system assigns appropriate weights to bandwidth and computing resources, while simultaneously monitoring task requirements and network status in real time. Through a certain joint optimization problem, the system adaptively adjusts bandwidth, computing resources, and compression strategy to minimize resource consumption and find the optimal balance between reconstruction error and resource usage. This ensures that the system optimizes resource allocation while maintaining task accuracy and real-time performance, improving overall efficiency and reliability.

[0112] Alternatively, by generating precise control instructions to adjust key parameters within the industrial semantic communication network, including data rate, bandwidth allocation, transmission latency, and computational overhead, the execution of the industrial semantic communication network can be regulated, the data transmission process optimized, and optimal system performance achieved. This, in turn, improves the efficiency and reliability of the industrial semantic communication system. For example, in intelligent manufacturing, robot control systems require data with low latency and high transmission rates, while sensor data is updated less frequently. Control instructions dynamically adjust the data transmission rate based on these requirements, allocating more bandwidth to high-priority tasks and reducing latency. Furthermore, instructions can intelligently allocate computing resources based on the computing needs of different devices, avoiding network overload and improving computing efficiency. When the system detects that a device in a particular link requires more bandwidth or lower latency, it automatically adjusts network settings to ensure the smooth execution of critical tasks. Through this precise adjustment, the industrial semantic communication network can effectively avoid data congestion, latency, and packet loss, ensuring stable data transmission and improving overall system reliability and responsiveness, thereby optimizing industrial production processes, reducing downtime, and ensuring efficient production line operation.

[0113] Multimodal semantic knowledge graphs are used to assist in the extraction, fusion, and alignment of semantic features for multimodal data during data transmission, providing semantic specifications and rules. Based on the OPC UA (Open Systems Interconnection Unified Architecture) information model, unified modeling and semantic encapsulation of industrial system / network / device data and embedded information models (real-time data acquisition, historical information collection) are achieved. Leveraging cognitive intelligence, industrial multimodal semantic knowledge graphs are constructed, providing industrial knowledge bases and prior information for industrial semantic communication systems. This assists in the extraction, fusion, and semantic reasoning of multimodal data in the manufacturing process, thereby improving the "plug-and-play" and semantic interoperability capabilities of heterogeneous IT / OT systems.

[0114] Multimodal semantic knowledge graphs fuse, align, and unify data and semantics from different modalities to provide a comprehensive, queryable knowledge base that supports analysis, reasoning, and decision-making of multimodal data.

[0115] To build a multimodal semantic knowledge graph: First, collect data from multiple modalities; this data comes from sources such as industrial equipment, sensors, text information, images, sound, and historical data. Then, extract semantic features from these multimodal data. For example, extract physical features (such as temperature and pressure) from sensor data, visual features such as shape and color from image data, and semantic information such as entities, relationships, and events from text data. The method for extracting semantic features depends on the data type. For example, deep learning methods (such as convolutional neural networks (CNNs) for image features and recurrent neural networks (RNNs) for time series data) or natural language processing techniques can be used to extract semantic features. Semantic features from multiple modalities are fused and transformed into a unified representation. For example, cross-modal learning methods can be used to integrate image features, text features, and sensor data features into a common semantic space. This is called semantic alignment, which enables accurate matching of semantic information from different sources. Semantic alignment typically relies on graph mapping rules and algorithms, using a unified semantic model (such as the OPC UA information model) to ensure semantic consistency across modalities. Finally, the extracted multimodal semantic features and relationships are organized into a knowledge graph structure; the nodes of the graph represent entities (such as equipment, components, and operating procedures), while the edges represent the relationships between entities (such as dependencies and transmission relationships). In industrial scenarios, nodes can be equipment, process steps, production lines, sensors, etc., while edges can be control relationships, data flows, maintenance history, etc.

[0116] OPC UA is based on data transmission and information modeling, providing an object-oriented working mechanism. The OPC UA specification opens up the possibility of defining information models. The OPC UA information model is used in industrial semantic communication systems that integrate perception, transmission, and control, aiming to standardize the representation of complex industrial data and processes. By modeling elements such as devices, sensors, and control systems, it is possible to define attributes (such as temperature, pressure, and device status), methods (such as start, stop, and adjust), and relationships between objects (such as hierarchical relationships and communication relationships between devices) for each object. This modeling approach not only ensures semantic consistency but also enables efficient and accurate transmission and sharing of information between different devices and applications, thereby enhancing the intelligence and interoperability of industrial automation systems.

[0117] To achieve the aforementioned objectives, the present invention constructs a cloud-edge collaborative network architecture to support the integration of perception, transmission, and control with industrial semantic communication in future 6G-oriented industrial IoT networks. It also proposes an industrial semantic communication framework based on deep compressed sensing to extract industrial semantics, enabling semantic-level transmission and agile control. Within this framework, a semantic recognition network is designed to reduce redundancy in industrial data. Intelligent strategies are used to train and quantize the deep compressed sensing network to reduce industrial data transmission volume and improve the robustness of the industrial semantic communication method and system to various channel conditions.

[0118] The present invention constructs a cloud-edge collaborative network architecture for the deployment of industrial semantic communication systems, supporting state perception, data collection, transmission, and feedback control in industrial network environments. The cloud is mainly responsible for the training and optimization of global semantic models, while the edge nodes focus on real-time data processing and local semantic extraction, thereby ensuring global performance while meeting low latency requirements. Through the cloud-edge collaborative network, the system can achieve efficient resource allocation, real-time response, and global optimization, while reducing transmission load and improving data processing efficiency.

[0119] This invention trains an industrial semantic recognition network for extracting, identifying, and classifying the semantics of complex industrial data. The trained semantic recognition network can improve the automation and intelligence of industrial data processing, effectively reduce manual intervention, and enhance the accuracy and efficiency of the system. It also possesses the ability to parse complex semantic relationships.

[0120] This invention trains a deep compressed sensing codec network and deploys it at the transmitter and receiver of an industrial semantic communication system to extract, transmit, and reconstruct complex industrial data and semantics. The transmitter performs efficient data compression and semantic encoding, while the receiver implements high-quality decoding and semantic recovery, ensuring the integrity and accuracy of semantic transmission. This improves the bandwidth utilization and anti-interference capabilities of the industrial semantic communication system, making it adaptable to low-bandwidth, high-interference network environments while achieving high-quality data reconstruction.

[0121] The present invention takes minimizing system consumption and data and semantic reconstruction errors as the joint goal, adaptively optimizes and adjusts key factors such as the data compression ratio and bandwidth allocation strategy of the perception and transmission processes, and uses the controller and actuator to adaptively adjust the data compression ratio of the encoding and decoding process. This optimization strategy can achieve efficient regulation of industrial semantic communication systems, and can adaptively balance performance and cost in different industrial scenarios, reduce system resource consumption, and improve the robustness and robustness of the system to meet the sustainable development needs in the industrial automation environment.

[0122] Example 3

[0123] The industrial semantic communication method for perception-transmission-control fusion runs on the industrial semantic communication system for perception-transmission-control fusion, such as Figure 1 As shown, including:

[0124] Step 101: Build a cloud-edge-end collaborative industrial IoT system to support state perception, data collection, transmission, and feedback control in an industrial network environment.

[0125] Step 102: Training an industrial semantic recognition network for extracting, identifying, and classifying complex industrial data semantics;

[0126] Step 103: Train a deep compressed sensing encoder and deploy it at the transmitter and receiver of the industrial semantic communication system to complete the transmission and reconstruction of complex industrial data and semantics;

[0127] Step 104: With the joint goal of minimizing system consumption and data reconstruction error, the data compression ratio and bandwidth allocation strategy of the perception and transmission processes are adaptively optimized and adjusted, and the data compression ratio of the encoding and decoding process is adaptively adjusted with the help of the controller and actuator.

[0128] Example 4

[0129] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the industrial semantic communication method for perception-transmission-control fusion described in Example 3 are implemented.

[0130] Example 5

[0131] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the industrial semantic communication method for perception-transmission-control fusion described in Example 3.

Claims

1. An industrial semantic communication system for perception-transmission-control fusion, characterized by: include: The system to be observed builds a cloud-edge-end collaborative industrial IoT system to collect multimodal data in industrial scenarios; Industrial semantic communication transmitter and receiver, used to extract, identify and efficiently transmit the semantic features of industrial data, i.e., multimodal data in industrial scenarios; The feedback controller is used to generate a control signal according to an optimized feedback control logic and act on the industrial semantic communication system.

2. The industrial semantic communication system for perception-transmission-control fusion according to claim 1 is characterized in that: The sending end includes an industrial semantic recognition network and a deep compressed sensing encoder; Industrial semantic recognition network, used to identify input data samples x k Perform semantic recognition and classification; include: Extract fixed-length input data samples x from real-time industrial data streams k , as the data input for a single process; use the industrial semantic recognition network to identify each input data sample x k Classify the data into different categories based on the classification results. The data is divided into 0-n categories according to the importance from low to high. For the data of high importance, that is, the data of categories 1-n, it is transmitted through deep compressed sensing codec; for the data of low importance, that is, the data of category 0, a semantic flag S is sent to the receiving end; The deep compressed sensing encoder semantically extracts and compresses the data of high importance to obtain the compressed data, and the sender transmits the compressed data through the channel.

3. The industrial semantic communication system for perception-transmission-control fusion according to claim 2 is characterized in that: The semantic recognition and classification process includes: data collection, data preprocessing, semantic classification and data transmission; Data acquisition: extract fixed-length input data samples x from real-time industrial data streams k ; Data preprocessing: input data sample x k Perform standardization, denoising, and completion processing in sequence; Semantic classification: The pre-processed data is fed into the industrial semantic recognition network for semantic recognition and classification into different categories. The industrial semantic recognition network includes a residual module, a CBAM module, and a fully connected layer. The residual module includes a primary convolutional layer, a multi-level convolutional layer, and a residual block. The primary convolutional layer includes a convolutional layer and a ReLU activation function, and the multi-level convolutional layer includes a convolutional layer, a batch normalization layer, and a ReLU activation function. The CBAM module includes channel attention and spatial attention. The preprocessed data is passed through a primary convolutional layer and two multi-level convolutional layers for convolution and ReLU activation operations, and then enters the CBAM module through a residual block. The feature map extracted by the residual block is mapped to the CBAM module. Each CBAM module uses maximum pooling and average pooling for feature compression in turn, and then calculates the channel and spatial importance through convolution and ReLU activation functions. Finally, the most important part of the feature map is enhanced through weighted operations. The feature output after weighted operation is convolved and the feature expression is further optimized through the Gamma weighting method. The weights of each part of the feature are adjusted to form the final feature representation. The final feature representation enters the fully connected layer, and the feature map is finally classified. Data transmission: For data of high importance, i.e., data of categories 1-n, it is transmitted through deep compressed sensing encoding and decoding; for data of low importance, i.e., data of category 0, a semantic flag S is sent to the receiving end.

4. The industrial semantic communication system for perception-transmission-control fusion according to claim 2, characterized in that: Semantic extraction and compression of highly important data; including: data input, feature extraction, compressed sensing, and compressed data output; Data input: Input the important data into the deep compressed sensing encoder; Feature extraction: Using deep learning models to extract high-level semantic features from important data, and learning the complex patterns and semantic information of the data through stacked neural networks; Compressed sensing: Compress data by applying compressed sensing technology and convert it into a low-dimensional feature vector, i.e. compressed data; Compressed data output: Send the compressed data to the receiving end.

5. The industrial semantic communication system for perception-transmission-control fusion according to claim 1 is characterized in that: The receiving end includes a deep compressed sensing decoder and an industrial AI model: The deep compressed sensing decoder maps the compressed data received from the transmitter into reconstructed data and reconstructs the semantic information of the original data; Industrial AI models use the semantic information of recovered raw data to perform intelligent manufacturing, predictive maintenance, intelligent decision-making, and other cloud-based industrial applications. They transmit control information to feedback controllers based on the semantic / data recovery quality or AI prediction and inference results, enabling intelligent control and decision-making of industrial systems. Further preferably, the deep compressed sensing decoder maps the compressed data received from the transmitter into reconstructed data, and reconstructs the semantic information of the original data; including: data input, sparse reconstruction and feature reconstruction, and data output; Data input: The compressed data is input to the deep compressed sensing decoder; Sparse reconstruction and feature reconstruction: Apply compressed sensing technology to reconstruct the coefficients of compressed data; Use deep neural networks to further recover high-order semantic information in the data and reconstruct the semantic information of the original data; Data output: The deep compressed sensing decoder outputs the semantic information of the reconstructed original data; Further preferably, the industrial AI model includes a prediction model, intelligent decision-making, and control information output; Prediction model: The industrial AI model analyzes the semantic information of the raw data to build a prediction model, which is used to predict equipment failures and monitor health. It uses supervised or unsupervised learning methods to analyze whether the equipment has potential failures and predict the remaining service life of the equipment. Intelligent decision-making: providing optimized decisions based on prediction results; Control information output: The control information is transmitted to the industrial control system, and automatic control is achieved through the feedback of the control information.

6. The industrial semantic communication system for perception-transmission-control fusion according to claim 1 is characterized in that: The feedback controller includes a controller and an actuator. The controller generates an adaptive control signal through feedback control logic and optimizes the actuator input signal in combination with a control gain factor. The actuator receives the control signal and acts on the semantic / data transmission processing process. Further preferably, the optimized control signal is sent to the actuator to adjust the state of the industrial semantic communication system; including: Dynamically adjust the weights of reconstruction error, bandwidth allocation, and computing resource consumption based on the needs and real-time requirements of different tasks. By optimizing the compression ratio, bandwidth allocation strategy, and activation matrix of the perception process, the system ensures that the optimal balance between resource consumption and reconstruction error is achieved. Alternatively, by generating control instructions, key parameters in the industrial semantic communication network, including data transmission rate, bandwidth allocation, transmission delay and computing overhead, are adjusted to adjust the execution of the industrial semantic communication network, optimize the data transmission process, and achieve the best performance of the system.

7. The industrial semantic communication system for perception-transmission-control fusion according to claim 1, characterized in that: Multimodal semantic knowledge graphs fuse, align, and unify data and semantics from different modalities to provide a comprehensive, queryable knowledge base that supports analysis, reasoning, and decision-making of multimodal data. Construct a multimodal semantic knowledge graph: First, collect multimodal data; then, extract semantic features from the multimodal data; fuse the semantic features from the multimodal data and transform them into a unified representation; finally, organize the extracted multimodal semantic features and relationships into a knowledge graph structure; the nodes of the graph represent entities, and the edges represent the relationships between entities.

8. An industrial semantic communication method for perception-transmission-control fusion, running on an industrial semantic communication system for perception-transmission-control fusion as described in any one of claims 1 to 7, characterized in that: include: Step 101: Build a cloud-edge-end collaborative industrial IoT system to support state perception, data collection, transmission, and feedback control in an industrial network environment. Step 102: Training an industrial semantic recognition network for extracting, identifying, and classifying complex industrial data semantics; Step 103: Train a deep compressed sensing encoder and deploy it at the transmitter and receiver of the industrial semantic communication system to complete the transmission and reconstruction of complex industrial data and semantics; Step 104: With the joint goal of minimizing system consumption and data reconstruction error, the data compression ratio and bandwidth allocation strategy of the perception and transmission processes are adaptively optimized and adjusted, and the data compression ratio of the encoding and decoding process is adaptively adjusted with the help of the controller and actuator.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the industrial semantic communication method for perception-transmission-control fusion according to claim 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the industrial semantic communication method for perception-transmission-control fusion according to claim 8 are implemented.

Images