A semantic-driven communication-control co-design method and related device

Abstract

The application provides a semantic-driven communication-control collaborative design method and related equipment, belonging to the technical field of industrial Internet of Things wireless control. The method comprises: acquiring real-time state data, and generating a first control instruction based on the real-time state data; activating an external sensing device through an event triggering mechanism, collecting external sensing data, and performing semantic extraction on the external sensing data to obtain spatiotemporal motion semantic information; transmitting the spatiotemporal motion semantic information to a remote controller through an uplink wireless channel; the remote controller performs asynchronous fusion on the received spatiotemporal motion semantic information and the real-time state data to obtain fusion features; generating a second control instruction based on the fusion features; and performing closed-loop control on each device in the industrial Internet of Things system according to the first control instruction and / or the second control instruction. The application can improve the reliability of key semantic information delivery under adverse channel conditions such as low signal-to-noise ratio, improve the performance of closed-loop control, and maintain the task execution capability.

Description

Technical Field

[0001] This application relates to the field of wireless control technology for the Industrial Internet of Things, and in particular to a semantic-driven communication-control co-design method and related equipment. Background Technology

[0002] As the Industrial Internet of Things (IIoT) develops towards wireless and intelligent technologies, wireless network control systems are evolving from local programmable logic controller (PLC) architectures to cloud-edge collaborative control paradigms. This improves system collaboration and scalability, but also further increases the requirements of the control closed loop on wireless link latency, transmission reliability, and task execution stability.

[0003] In closed-loop control of the Industrial Internet of Things (IIoT), control decisions increasingly rely on multi-source information: on-premises sensing measurements (such as joint angles and velocities) provide high-rate, lightweight instantaneous local state updates, while external sensing observations (such as images / videos) provide rich global scene information at a lower rate. The two are naturally complementary in terms of timeliness and information completeness. However, under unfavorable channel conditions such as low signal-to-noise ratios, sensor state information and external sensing information are susceptible to latency, packet loss, and channel fading during wireless transmission. This leads to reduced freshness of state updates and unreliable delivery of critical information, resulting in uncertain state estimation at the receiver, increased control errors, and even task unreachability. In particular, high-dimensional external sensing data (such as video streams) is difficult to deliver in a timely and complete manner, leading to missing observation information and severely impacting the quality of control decisions.

[0004] Semantic communication, by transmitting task-related semantic representations, can reduce redundant data transmission and improve the efficiency of effective information transmission, providing a new approach for the wireless transmission of high-dimensional data. However, existing semantic communication methods mainly focus on data reconstruction quality, semantic similarity, or communication efficiency as optimization objectives, lacking joint consideration of closed-loop control performance indicators such as control cost and system stability. At the same time, under low signal-to-noise ratio conditions, there is a lack of mechanisms to ensure the reliable delivery of key semantic information, making it difficult to ensure that semantic information that plays an important role in control decisions is transmitted to the control end in a timely and reliable manner. Summary of the Invention

[0005] The main objective of this application is to propose a semantic-driven communication-control co-design method, electronic device, storage medium, and program product. This aims to address the problems in existing technologies, such as communication-control joint optimization methods mainly relying on ontological sensor state information, having a single sensing source, lacking global scene or collaborative task trend information, and semantic communication methods focusing on semantic transmission reliability without fully addressing closed-loop control tasks.

[0006] To achieve the above objectives, one aspect of this application proposes a semantic-driven communication-control co-design method, applied to an industrial IoT wireless control system, the method comprising the following steps: Acquire real-time status data collected by the sensors of each device body, and generate a first control command based on the real-time status data; The external sensing device is activated by an event triggering mechanism to collect external sensing data, and the semantics of the external sensing data are extracted to obtain spatiotemporal motion semantic information. A hybrid automatic repeat request mechanism oriented towards spatiotemporal motion semantics is adopted to transmit the spatiotemporal motion semantic information to the remote controller via the uplink wireless channel. The semantic block of each round of retransmission is determined by the semantic distortion score and semantic contribution score of the semantic block. The remote controller asynchronously fuses the received spatiotemporal motion semantic information with the real-time status data to obtain fused features; A second control command is generated based on the fusion features; Closed-loop control is performed on each device in the industrial Internet of Things system according to the first control command and / or the second control command; The semantic contribution score is used to characterize the degree of influence of each semantic block on the control cost, and the semantic distortion score is used to characterize the degree of distortion of each semantic block during wireless transmission.

[0007] In some embodiments, the step of activating an external sensing device through an event-triggered mechanism, collecting external sensing data, and performing semantic extraction on the external sensing data to obtain spatiotemporal motion semantic information includes: When a new control command is generated, the external sensing device is triggered to collect the external sensing data at the current moment, and the external sensing data is video sequence data; Spatiotemporal motion semantic features are extracted from the video sequence data by a joint source-channel coding module to obtain spatiotemporal motion semantic information, which includes historical and trend features of the system motion. The spatiotemporal motion semantic information is transmitted to the remote controller via an uplink wireless channel, and the remote controller performs noise reduction processing on the received signal through a multilayer perceptron.

[0008] In some embodiments, the hybrid automatic repeat request mechanism oriented towards spatiotemporal motion semantics specifically includes: A1, The receiving end obtains the first The system state error vector after round transmission is calculated, and the uncertainty of the received information is calculated based on the system state error vector; A2, Determine the first Check whether the received signal-to-noise ratio exceeds the threshold value determined by the uncertainty of the received information; if it exceeds, stop retransmission; if it does not exceed, proceed to step A3. A3, the first step is calculated from the distortion estimation network. The semantic distortion score of each semantic block in the round is calculated by the controller network. The semantic contribution score of each semantic block is calculated, and the integrated transmission feedback signal is calculated based on the semantic distortion score and the semantic contribution score. A4. Determine the first [missing information] based on the comprehensive transmission feedback signal. The set of semantic blocks that need to be retransmitted in a round; A5. Repeat steps A1 to A4 until the maximum number of transmission rounds is reached.

[0009] In some embodiments, the calculation method for the integrated transmission feedback signal is as follows:

[0010] in, For the first The semantic block in the first The indication signal indicating whether the wheel needs to be retransmitted. For indicator functions, For the first The semantic block in the first The semantic distortion score of the wheel, For the first The semantic block in the first The semantic contribution score of the wheel This represents a trade-off between distortion sensitivity and task relevance. This is the acceptable distortion threshold.

[0011] In some embodiments, the semantic contribution score is calculated as follows: During control time The remote controller performs forward reasoning based on the currently received real-time status data and spatiotemporal motion semantic information to generate the second control command; Calculate the control cost under the second control command; With fixed controller network parameters, the control cost is backpropagated with respect to the received spatiotemporal motion semantic information to obtain a semantic contribution score:

[0012] in, Contribute scores to semantics In order to control costs, For the received improved semantic information, This is the second control command. Designing networks for semantic-driven control This indicates taking the absolute value of each element.

[0013] In some embodiments, the asynchronous fusion specifically includes: Define a set of fusion moments, which is the union of a first control command moment determined solely by sensor data and a second control command moment jointly determined by sensor data and semantic information; At the current fusion moment, if only real-time status data exists, then the first control command is generated; At the current fusion moment, if real-time state data and received spatiotemporal motion semantic information exist simultaneously, the features of the real-time state data and the features of the spatiotemporal motion semantic information are fused through a cross-attention mechanism to obtain the fused features.

[0014] In some embodiments, the method further includes the following steps: A valid semantic age is defined to characterize the timeliness of the spatiotemporal motion semantic information after it reaches the remote controller. The valid semantic age is calculated as follows:

[0015] in, The sum of the joint source channel coding delay and the control command delay is calculated. This is the sum of the denoising delay, propagation delay, and feedback delay of the multilayer perceptron. For the current transmission round, For the first The number of semantic blocks retransmitted in round-robin mode. This represents the maximum number of transmission rounds. Furthermore, the effective semantic age is applied as a long-term constraint to the communication-control collaborative optimization process, namely:

[0016] in, This serves as a long-term constraint threshold for the effective semantic age.

[0017] In some embodiments, the generation of the first control command and / or the second control command is achieved using a hybrid strategy Lyapunov soft actor-critic algorithm, the algorithm comprising: Construct a constrained Markov decision process model, where the state space contains the system state and the desired state, and the action space is the set of control commands. Maintain two rewarded critic networks and employ double-Q learning to reduce overestimation bias; A Lyapunov critic network is constructed to evaluate the stability of the control system; the Lyapunov critic network is constructed as a positive definite function. Construct an actor network and update the policy parameters by maximizing an objective function that includes entropy regularization and Lyapunov stability constraints; A hybrid strategy guided by domain knowledge is adopted to linearly interpolate the learned control instructions output by the actor network with the control priors to generate the final control instructions.

[0018] In some embodiments, the method further includes decomposing the communication-control joint optimization problem into fast-time-scale subproblems and asynchronous-time-scale subproblems, wherein: The fast timescale subproblem is used to solve the first control command that depends only on real-time state data and is executed at the moment when the sensor data is updated. The asynchronous time-scale subproblem is used to jointly optimize the semantic extraction strategy, semantic transmission strategy, and compensation control instructions, and is executed at the moment the semantic information arrives. The fast timescale subproblem and the asynchronous timescale subproblem are coupled together through system state dynamics and long-term effective semantic age constraints.

[0019] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0020] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0021] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0022] The embodiments of this application include at least the following beneficial effects: This application provides a semantic-driven communication-control collaborative design method, device, electronic device, storage medium, and program product. This solution constructs a dual-stream sensing framework that integrates real-time sensor data streams and asynchronous external sensing semantic information, organically combining the immediacy of ontological perception with the globality of visual semantics, effectively overcoming the problem of incomplete information from a single sensing source in complex industrial scenarios. Simultaneously, by introducing the STKS-HARQ selective retransmission mechanism driven by both semantic distortion score and semantic contribution score, communication resources can prioritize serving key semantic blocks with high influence on the control task, rather than blindly retransmitting all damaged data, thereby maximizing transmission efficiency and control benefits under limited communication resources. Furthermore, by establishing a closed-loop collaborative optimization mechanism of "transmission-fusion-control-feedback," communication strategies and decision control are no longer separated, enabling dynamic adaptation to changes in channel conditions. This significantly improves control accuracy and system stability under adverse conditions such as low signal-to-noise ratio, reduces communication overhead while ensuring task reachability, and provides a systematic solution for industrial IoT wireless control that combines high efficiency, robustness, and stability. Attached Figure Description

[0023] Figure 1 This is a flowchart of a semantically driven communication-control co-design method provided in an embodiment of this application.

[0024] Figure 2 This is a schematic diagram of the overall architecture and information flow of the system in the embodiments of this application.

[0025] Figure 3 This is a schematic diagram of the timing of dual-stream transmission of real-time sensor data stream and asynchronous video semantic data stream in an embodiment of this application.

[0026] Figure 4 This is a network architecture diagram of the HLSAC and A-HLSAC algorithms in the embodiments of this application.

[0027] Figure 5 This is a flowchart illustrating the training and execution process of the A-HLSAC algorithm in this application embodiment.

[0028] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0031] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0032] 1) Communication-Control Co-Design (COCOCO): COCOCO is a system optimization method that jointly considers the wireless communication process and the closed-loop control process. Its core idea is to simultaneously optimize information transmission strategies and control decisions under conditions of limited communication resources and dynamically changing link states, thereby improving the system's real-time performance, reliability, and stability. In this application, COCOCO is used to coordinate the interaction between ontological perception data, external visual semantic information, and control strategies, providing stable and reliable decision support for closed-loop control tasks in the Industrial Internet of Things (IIoT).

[0033] 2) Semantic Communication (SemCom): Semantic communication is a novel communication paradigm oriented towards information meaning and task objectives. It no longer focuses solely on bit-level error-free transmission, but rather on the extraction, compression, transmission, and retrieval of key information relevant to task execution. By transmitting only semantic features useful to the task, semantic communication can reduce redundant data transmission overhead and improve information delivery efficiency under harsh channel conditions. In this application, semantic communication is used to extract spatiotemporal motion semantic information from external visual perception data.

[0034] 3) Soft Actor-Critic (SAC) Algorithm: The Soft Actor-Critic algorithm is a deep reinforcement learning algorithm based on the maximum entropy reinforcement learning framework. It improves the agent's exploration ability and decision stability in a continuous action space by simultaneously optimizing policy rewards and policy entropy. This algorithm is suitable for policy optimization problems in high-dimensional, dynamic, and uncertain environments. In this application, SAC is used to construct an asynchronous hybrid policy optimization algorithm and, combined with Lyapunov stability constraints, achieves joint optimization of semantic transmission policies and control decisions.

[0035] With the development of the Industrial Internet of Things (IIoT) towards wireless and intelligent directions, wireless networked control systems are gradually evolving from the traditional local programmable logic controller (PLC) architecture to a cloud-edge collaborative control paradigm. This evolution significantly improves the system's collaboration and scalability, but it also places higher demands on the control closed loop in terms of wireless link latency, transmission reliability, and task execution stability.

[0036] In typical industrial IoT closed-loop control systems, control task decisions increasingly rely on the fusion of multi-source information: On-body sensing measurements (such as joint angles and velocities) provide high-rate, lightweight instantaneous local state updates, forming the basis of real-time control; external sensing observations (such as image or video data) provide richer global scene information at lower rates, including environmental state changes, target motion trends, and multi-device collaborative situations. These two aspects are naturally complementary in terms of timeliness and information completeness, together constituting the information foundation for control system decisions.

[0037] However, under unfavorable channel conditions such as low signal-to-noise ratio, sensor state information and externally sensed information are easily affected by factors such as latency, packet loss, and channel fading during wireless transmission. This leads to reduced state update freshness, unreliable delivery of critical information, and further causes problems such as uncertain state estimation at the receiver, increased control errors, and even mission unreachability. In particular, high-dimensional externally sensed data (such as video streams) is often difficult to deliver to the remote controller in a timely and complete manner, resulting in missing observation information and severely affecting the quality of control decisions based on global perception.

[0038] In recent years, the rise of Semantic Communication (SemCom) has provided a new technological approach for the wireless transmission of high-dimensional data. Semantic Communication is a novel communication paradigm oriented towards information meaning and task objectives. It no longer focuses solely on bit-level error-free transmission, but rather on the extraction, compression, transmission, and recovery of key information relevant to task execution. By transmitting only semantic features useful to the task, Semantic Communication can reduce the overhead of redundant data transmission and improve information delivery efficiency under harsh channel conditions. However, existing Semantic Communication methods primarily optimize data reconstruction quality, semantic similarity, or communication efficiency, lacking joint consideration of closed-loop control performance indicators such as control cost and system stability. Furthermore, under low signal-to-noise ratio conditions, there is a lack of mechanisms to ensure the reliable delivery of key semantic information, making it difficult to ensure that semantic information crucial for control decisions is transmitted to the control end in a timely and reliable manner.

[0039] To address the above problems, some solutions have been proposed in this field.

[0040] Existing technical solution 1 Pang et al. (“Communication-control codesign for large-scale wirelessnetworked control systems,” IEEE J. Sel. Areas Commun., vol. 43, no. 10, pp.3295-3312, Oct. 2025) proposed a co-design method for communication and control in large-scale wireless networked control systems. This scheme uses deep reinforcement learning to jointly optimize communication resource scheduling and control decisions, thereby improving the overall system performance in large-scale scenarios. The core technical idea of ​​this scheme is to incorporate communication scheduling and control decisions into a unified learning framework for joint optimization, providing a relatively complete reference framework for the communication-control co-design of wireless networked control systems.

[0041] The main limitation of this scheme lies in its reliance on state information provided by the body sensor for control decisions, resulting in a relatively singular source of perception. While the body sensor can provide local state information such as joint angles and velocities, its ability to represent changes in the external environment, target motion trends, and the global scene state is limited, making it difficult to provide sufficient trend judgment and collaborative decision support for complex industrial control tasks. Under low signal-to-noise ratio conditions, strategies relying solely on body perception information are easily affected by the degradation of received state, leading to decreased control performance or even system instability. The main reason for these limitations is that the modeling of this scheme focuses on the joint optimization of communication resource scheduling and control decisions, without fully considering the impact of environmental state changes and multi-device collaboration in complex industrial tasks on control decisions, thus limiting the system's ability to perceive global scene changes and task trends.

[0042] Existing technical solution 2 Hu et al. (“SemHARQ: Semantic-aware hybrid automatic repeat request for multi-task semantic communications,” IEEE Trans. Wireless Commun., vol. 25, pp. 3170-3185, 2026) proposed a semantic-aware hybrid automatic repeat request method (SemHARQ) for multi-task semantic communication scenarios. This scheme identifies damaged semantic features through feature importance ranking and distortion estimation networks, and selectively retransmits the damaged parts, thereby improving the robustness of multi-task semantic communication under low signal-to-noise ratio conditions. This scheme embodies the technical idea of ​​extending the automatic repeat request mechanism from the bit domain to the semantic domain and selectively retransmitting key semantic elements, providing a useful reference for the reliable delivery of semantic information.

[0043] However, this scheme primarily studies semantic-level selective retransmission in multi-task semantic communication. Its optimization objective mainly focuses on semantic reconstruction quality or recognition accuracy, and retransmission decisions are primarily driven by the degree of semantic distortion on the communication side, failing to establish a correlation between semantic transmission and control performance for closed-loop control tasks. On the one hand, this scheme lacks joint consideration of closed-loop control performance indicators such as task error, control cost, and system stability; on the other hand, it fails to characterize the timeliness and task contribution of semantic information after it reaches the control end, i.e., it does not distinguish the importance of different semantic information to the current control update. The main reason for these limitations is that this scheme models from the perspective of semantic reconstruction or recognition reliability on the communication side, and its technical focus is on improving the transmission robustness of semantic communication itself, without considering the control closed loop or co-optimizing semantic transmission strategies and control decisions in a task contribution-oriented manner.

[0044] In view of this, this application provides a semantic-driven communication-control co-design method, electronic device, storage medium, and program product. This solution constructs a closed-loop mechanism of "transmission-fusion-control-feedback," incorporating real-time sensor data transmission, video spatiotemporal motion semantic extraction and asynchronous transmission, semantic-level selective retransmission for control tasks, and control decision optimization under stability constraints into a unified joint optimization framework. Through this closed-loop mechanism, communication transmission strategies and control commands are co-optimized, thereby improving the reliability of key semantic information delivery under unfavorable channel conditions such as low signal-to-noise ratio, improving closed-loop control performance, and maintaining task execution capability.

[0045] This application provides a semantic-driven communication-control collaborative design method, relating to the field of wireless control technology in the Industrial Internet of Things (IIoT). This semantic-driven communication-control collaborative design method can be applied to terminals, servers, or software running on either terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a semantic-driven communication-control collaborative design method, but is not limited to the above forms.

[0046] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0047] like Figure 1 As shown, this embodiment provides a semantic-driven communication-control co-design method applied to an industrial IoT wireless control system. The method includes the following steps: Step S1: Obtain real-time status data collected by the sensors of each device body, and generate a first control command based on the real-time status data.

[0048] Step S2: Activate the external sensing device through the event triggering mechanism, collect external sensing data, and perform semantic extraction on the external sensing data to obtain spatiotemporal motion semantic information.

[0049] In one embodiment, step S2 specifically includes the following steps: S21. When a new control command is generated, the external sensing device is triggered to collect the external sensing data at the current moment, wherein the external sensing data is video sequence data; S22. The video sequence data is subjected to spatiotemporal motion semantic features by a joint source-channel coding module to obtain spatiotemporal motion semantic information, which includes the historical and trend features of the system motion. S23. The spatiotemporal motion semantic information is transmitted to the remote controller via the uplink wireless channel, and the remote controller performs noise reduction processing on the received signal through a multilayer perceptron.

[0050] Step S3: Employing a hybrid automatic repeat request mechanism oriented towards spatiotemporal motion semantics, the spatiotemporal motion semantic information is transmitted to the remote controller via an uplink wireless channel. The semantic block for each round of retransmission is determined by the semantic distortion score and semantic contribution score of that semantic block. The semantic contribution score characterizes the impact of each semantic block on the control cost, while the semantic distortion score characterizes the degree of distortion of each semantic block during wireless transmission.

[0051] In one embodiment, step S3, specifically includes the hybrid automatic repeat request mechanism oriented towards spatiotemporal motion semantics, comprising: S31, The receiving end obtains the... The system state error vector after round transmission is calculated, and the uncertainty of the received information is calculated based on the system state error vector; S32, Determine the first Check whether the received signal-to-noise ratio exceeds the threshold value determined by the uncertainty of the received information: if it exceeds, stop retransmission; if it does not exceed, proceed to step S33. S33, the first step is calculated by the distortion estimation network. The semantic distortion score of each semantic block in the round is calculated by the controller network. The semantic contribution score of each semantic block is calculated, and the integrated transmission feedback signal is calculated based on the semantic distortion score and the semantic contribution score. S34. Determine the first step based on the comprehensive transmission feedback signal. The set of semantic blocks that need to be retransmitted in a round; S35. Repeat steps S31 to S34 until the maximum number of transmission rounds is reached.

[0052] In one embodiment, the integrated transmission feedback signal is calculated as follows:

[0053] in, For the first The semantic block in the first The indication signal indicating whether the wheel needs to be retransmitted. For indicator functions, For the first The semantic block in the first The semantic distortion score of the wheel, For the first The semantic block in the first The semantic contribution score of the wheel This represents a trade-off between distortion sensitivity and task relevance. This is the acceptable distortion threshold.

[0054] In one embodiment, the semantic contribution score is calculated as follows: During control time The remote controller performs forward reasoning based on the currently received real-time status data and spatiotemporal motion semantic information to generate the second control command; Calculate the control cost under the second control command; With fixed controller network parameters, the control cost is backpropagated with respect to the received spatiotemporal motion semantic information to obtain a semantic contribution score:

[0055] in, Contribute scores to semantics In order to control costs, For the received improved semantic information, This is the second control command. Designing networks for semantic-driven control This indicates taking the absolute value of each element.

[0056] In one embodiment, the distortion estimation network is pre-trained offline in the following manner: B1. Under the ideal transmission assumption, acquire the original spatiotemporal motion semantic information and the improved semantic information after decoding at the receiving end, and construct a training dataset, which includes joint samples and edge samples. B2. Using the mutual information between the original spatiotemporal motion semantic information and the improved semantic information as the optimization objective, the lower bound of the mutual information neural estimation is used for estimation and maximization. B3. The loss function of the distortion estimation network is:

[0057] in, and The first The original semantic information and improved semantic information of each semantic block. For mutual information, For the distortion estimation network; B4. During online deployment, freeze the parameters of the distortion estimation network, perform forward inference only, and output the semantic distortion score of each semantic block.

[0058] Step S4: The remote controller asynchronously fuses the received spatiotemporal motion semantic information with the real-time status data to obtain fused features.

[0059] In one embodiment, the asynchronous fusion in step S4 specifically includes: S41. Define a set of fusion moments, wherein the fusion moments are the union of the first control command moment determined solely by sensor data and the second control command moment determined jointly by sensor data and semantic information; S42. At the current fusion moment, if only real-time status data exists, then execute step S1 to generate the first control command; S43. At the current fusion moment, if real-time state data and received spatiotemporal motion semantic information exist simultaneously, the features of the real-time state data and the features of the spatiotemporal motion semantic information are fused through a cross-attention mechanism to obtain fused features.

[0060] Step S5: Generate a second control command based on the fusion features.

[0061] Step S6: Perform closed-loop control on each device in the industrial Internet of Things system according to the first control command and / or the second control command.

[0062] In one embodiment, step S6 is followed by steps S7-S8: Step S7: Define the effective semantic age, which characterizes the timeliness of the spatiotemporal motion semantic information after it reaches the remote controller. The effective semantic age is calculated as follows:

[0063] in, The sum of the joint source channel coding delay and the control command delay is calculated. This is the sum of the denoising delay, propagation delay, and feedback delay of the multilayer perceptron. For the current transmission round, For the first The number of semantic blocks in round-robin retransmission. This represents the maximum number of transmission rounds.

[0064] Step S8: Apply the effective semantic age as a long-term constraint to the communication-control collaborative optimization process, that is:

[0065] in, This serves as a long-term constraint threshold for the effective semantic age.

[0066] In one embodiment, both the generation of the first control command in step S1 and the generation of the second control command in step S5 are solved using the hybrid strategy Lyapunov soft actor-commentator algorithm, which includes: C1. Construct a constrained Markov decision process model, where the state space contains the system state and the desired state, and the action space is the set of control commands. C2. Maintain two rewarded critic networks and use double-Q learning to reduce overestimation bias; C3. Construct a Lyapunov commentator network to evaluate the stability of the control system. The Lyapunov commentator network is constructed as a positive definite function. C4. Construct an actor network and update the policy parameters by maximizing an objective function that includes entropy regularization and Lyapunov stability constraints. C5. A hybrid strategy guided by domain knowledge is adopted to linearly interpolate the learned control instructions with control priors to generate the final control instructions.

[0067] Furthermore, the objective function of the actor network in step C4 is:

[0068] in, For actor network parameters, The entropy regularity coefficient is... To be the minimum entropy bound, To reward the Q-value of the critics' network, Lagrange multipliers constrained by Lyapunov stability For Lyapunov critics network, These are the weighting coefficients. It is a function of state error.

[0069] Further, in step C5, the control prior is a conventional control baseline calculated through linear interpolation, and the final control command is calculated as follows:

[0070] in, For the final control command, The learning control instructions output to the actor network. This is the control prior. The weights are those of the learning control commands.

[0071] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application examples.

[0072] (I) System Model Construction and Mathematical Programming See Figure 2 This embodiment considers a complex wireless industrial IoT system consisting of M devices. Each device is equipped with a local sensor (expandable to multiple sensors), which uploads local data to a remote / cloud-based programmable logic controller (C-PLC) via orthogonal frequency division multiplexing. The C-PLC aggregates and analyzes the information to generate predictive control commands. Considering that the uplink in industrial scenarios needs to transmit massive amounts of data while the downlink control command payload is relatively small, this application mainly focuses on the uplink communication process.

[0073] To address the issue that sensor data is susceptible to noise during wireless transmission, leading to degraded reception and reduced control performance, this embodiment introduces event-triggered video information collected by a camera as compensation. A semantic extraction mechanism is employed to efficiently represent and transmit the video data, and the received semantic information is asynchronously fused with the current sensor data to predict control commands.

[0074] The system status is transmitted periodically at fixed intervals (one time slot). Let the system status of the m-th device be... ( For the state dimension, For time slots, , (Total number of time slots). The signal received at the C-PLC is... ,in, It is additive white Gaussian noise. The fading coefficient is the channel fading factor. Considering the coexistence of line-of-sight and non-line-of-sight paths, the channel is modeled as a Rician channel.

[0075] Control commands determined solely by sensor data are denoted as ,in, For predictive control command model, For the first A control command time determined solely by sensor data; once generated, the control command immediately takes effect on the system, and the sending end's state is determined by... Updated to .

[0076] To improve the accuracy of perception-based control while saving communication resources, the acquired video sequences ( Frame count, resolution per frame First, the spatiotemporal motion semantic features are extracted by the Joint Source-Channel Coding (JSCC) module. , 。 The signal is transmitted to the C-PLC via the uplink wireless channel and received. The improved signal is then denoised using a multilayer perceptron. Subsequently, the semantic-driven controller generates compensation control instructions. ,in Designing networks for semantic-driven control For the first A control command moment generated by sensor data and semantic information. Video acquisition uses an event-triggered mechanism. , As an indicator function, when a new control command is generated ( )hour The camera was activated.

[0077] make ,in, , , This represents an operator that returns the elements of a set in ascending order of time. The affine nonlinear dynamics of the control of the m-th device is: (1) in, The noise is a zero-mean Gaussian system. It is a nonlinear state transition function. This is the state-dependent control matrix.

[0078] Evaluate control performance using an instantaneous cost function: (2) in, semi-positive definite Zhengding, This represents the state error. To ensure stability, a Control Lyapunov Function (CLF) is introduced, defined as follows: function It is a CLF, when a control law exists. ,constant ,and For an extended Class function, such that for any and have: (3) (4) in, It is about The function, and .

[0079] (II) Spatiotemporal Motion Semantics—Hybrid Automatic Repeat Request Mechanism Spatio-Temporal Kinematic-Semantic (STKS) carries key historical and trend characteristics of system motion and is more sensitive to channel fading; even slight distortion can lead to semantic degradation. To enhance the reliability of the semantic link and the robustness of task execution, this embodiment introduces a hybrid automatic repeat request mechanism for STKS (STKS-HARQ). Let the number of transmission rounds be... , This is the maximum number of transmission rounds; the receiving end repeatedly requests retransmission until the [number]th round. Round-received signal-to-noise ratio Exceeding the uncertainty of information The threshold for determination, i.e. (5) in, It is a monotonically increasing function, defined as , To account for the uncertainty of the received information, For the first The system state error vector after round transmission, The conversion ratio from uncertainty to signal-to-noise ratio of the merged information. This represents the maximum permissible received signal-to-noise ratio.

[0080] Considering the semantic distortion introduced by wireless transmission, this embodiment introduces a distortion estimation network. Offline learning of original, lossless STKS vectors With improved receiving vector The distribution differences between them. The pre-training phase... and To optimize the semantic similarity of the target, mutual information is estimated and maximized using the lower bound of Mutual Information Neural Estimation (MINE). (6) in, and Represent and marginal distribution, represent and The joint distribution of . The loss function is defined as: (7) in, and They are and The nth STKS block (i.e., the smallest unit of semantic information transmission). Offline training is performed in an unsupervised, independent manner, with input consisting of joint samples and marginal samples from combinations of the same / different sources; during online deployment, parameters are frozen and forward inference is performed only, with the output vector being... As a semantic distortion score, it quantifies the degree of distortion of the semantic block received in the ξth round.

[0081] Due to distortion score To further reflect the task relevance, this embodiment introduces a semantic contribution score. This is used to quantify the contribution of semantic blocks to the control task, and based on this, a transmission feedback signal that simultaneously considers semantic distortion and task contribution is defined: (8) in, For indicator functions, For acceptable distortion threshold, This represents a trade-off between distortion sensitivity and task relevance. The number of STKS blocks per retransmission round is... Therefore, the STKS-HARQ transmission strategy is as follows: (9) In addition, communication overhead is defined to evaluate the performance of the communication system. Lower communication overhead means more resources are saved.

[0082] (iii) Effective semantic age Industrial Internet of Things (IIoT) systems heavily rely on real-time data uploaded by edge devices for decision-making, and the timeliness of this data directly impacts control accuracy and system stability. Traditional Age of Information (AoI) only characterizes information freshness based on timestamps, failing to adequately represent the timeliness of semantic information that truly impacts control updates. Therefore, this embodiment defines an Effective Semantic Age (AoES) at the C-PLC: (10) (11) in, , . , , , and The latency is composed of JSCC encoding, control command calculation, multilayer perceptron (MLP) denoising, propagation, and feedback. This latency varies with the number of retransmission rounds. Increase As it increases, it represents and Increased deviation and asynchronous transmission delay can lead to information obsolescence. Therefore, long-term constraints are imposed: (12) (iv) Optimize problem modeling The system objective is to rapidly obtain control commands that maintain the stability of the control system under stability constraints. AoES reflects the system's real-time performance and the accuracy of STKS transmission. This reflects the control performance and the magnitude of the control input. Accordingly, this embodiment optimizes the transmission strategy. STKS extraction model With control commands Construct a joint optimization problem: (13) (14) (15) (3), (4), (16) in, Constraint (14) indicates whether the nth STKS block needs to be transmitted, constraint (15) indicates the long-term timeliness constraint of AoES, and constraint (16) is the stability constraint of the nonlinear control system.

[0083] (v) Joint optimization method Due to the problem The problem is strongly nonconvex and involves a hybrid optimization of asynchronous semantic transmission and continuous control decision-making, making stable convergence difficult in end-to-end solutions. Utilizing the system's inherent dual-timescale structure, this invention... It can be broken down into the following two sub-problems: (17) (18) (19) (20) in, In Depend on The decision can be considered a constant during the period when the semantic block is not finally received. Control update moment on a fast timescale Solve and learn real-time control strategies; Control update time on asynchronous time scale Solve the semantic extraction, transmission strategy, and compensation control decision. and It affects the system state at different times, therefore and This invention couples state dynamics with long-term AoES constraints. Due to the complex coupling of nonlinear constraints and multiple factors such as semantic extraction networks, decoding networks, and controller network prediction, traditional optimization methods (such as gradient descent-based or convex optimization algorithms) are difficult to solve effectively. Considering the inherent dynamics and long-term dependencies of scheduling problems, reinforcement learning has significant advantages in policy adaptation and long-term optimization, making it suitable for intelligent scheduling scenarios. Therefore, this invention introduces a reinforcement learning method to learn the optimal policy through interaction with the environment to cope with high-dimensional, non-convex, and dynamically changing control tasks.

[0084] (vi) Hybrid Strategy Lyapunov Soft Actor-Critic Algorithm (Solving) ) The Soft Actor-Critic (SAC) algorithm enhances policy exploration through entropy regularization, which maximizes policy entropy, making it particularly suitable for multi-objective optimization problems. To obtain effective control commands that enable the control system to quickly and stably reach the target point, this embodiment proposes a hybrid policy Lyapunov Soft Actor-Critic (HLSAC) algorithm based on SAC to solve the problem. To mitigate overestimation bias, double-Q learning is employed, maintaining two reward commentators. and To reduce overestimation bias; at the same time, a parameter is introduced as follows: Lyapunov critic To assess stability; the actor network uses parameters as... of express.

[0085] The problem is modeled as a Constrained Markov Decision Process (CMDP), consisting of tuples. Indicates. For The state space and action space are respectively and Based on strategy Random sampling action This causes the system to transition to the next state. State transitions are determined by transition probabilities. Description. One-step reward is defined as... Used to evaluate the effectiveness of the strategy, its average result is constraint set Formula (16) is used.

[0086] To satisfy the constraints, we first construct Lyapunov critics to guarantee positive definiteness in the CLF, with the objective function being: (twenty one) (16) guarantees Positive definiteness of CLF in stability constraints. Through learning Continuous optimization is used to evaluate system stability. The objective function of the critic network is: (twenty two) in, This serves as an experience replay buffer, storing historical data on the agent's interactions with the environment; the target value , , As a discount factor, For the target network used for stable training; Used for booting The direction of study. The target network parameters are updated softly: (twenty three) in, Let be the soft update coefficients. The objective function of the actor network is: (twenty four) in, To be the minimum entropy bound, , These are Lagrange multipliers, used to enhance policy exploration (ensuring high entropy in action distribution) and to ensure Lyapunov stability, respectively. Take even Smaller values ​​are used to mitigate overestimation. Policy network parameters Update using stochastic gradient descent , Update and clip to positive values ​​using gradient ascent: (25) (26) in, and They are dual variables and Step size, , Furthermore, for high-dimensional action spaces (such as the two-arm planning problem), the exploration space is too large, making it difficult to effectively solve planning problems by directly applying reinforcement learning algorithms. To improve the efficiency of reinforcement learning, this embodiment adopts a hybrid strategy guided by domain knowledge, confining the exploration to a reasonable torque baseline: conventional control priors are calculated online through linear interpolation. and combine it with the learned instructions. Mixing forms a mixing action: (27) in, represent By assigning weights to these weights, the exploration is constrained to a reasonable torque baseline, thereby improving the efficiency of reinforcement learning.

[0087] (vii) Asynchronous HLSAC Algorithm Based on STKS-HARQ (Solving) ) The solution and Similarly, the key difference lies in Semantic information in asynchronous video needs to be considered separately. Therefore, for The state space is: The action space is Since the long-term AoES constraint in (15) must be satisfied, this embodiment uses the Lyapunov optimization framework to process it, thereby making corresponding modifications to the one-step reward function. First, an AoES deficit queue is constructed, the dynamics of which are: (28) in, This represents the deviation between the instantaneous AoES and its long-term requirements. Subsequently, the Lyapunov function is introduced to characterize the degree to which the long-term AoES constraints are satisfied, defined as... The smaller the value, the better the long-term AoES requirements are met. Next, a first-order Lyapunov drift is introduced to measure the change in the Lyapunov function between two adjacent AoES update times: , (29) in It is a constant. For the maximum AoES (when (Time). Therefore, the one-step reward can be updated to ,in These are the weight parameters.

[0088] Optimize variables The A-HLSAC algorithm is employed to solve the problem. This algorithm introduces a cross-attention mechanism into the traditional HLSAC framework to fuse sensor data features and semantic information. It achieves optimal results. After that, the network Corresponding parameters It was also determined to be the optimal value. .

[0089] Semantic contribution score The calculation method is as follows: (1) in the time slot After forward reasoning, calculate Control costs (2) Fixed network parameters, regarding Backpropagation yields: (30) in, Indicates A-HLSAC network, , This represents the element-wise absolute value operation—because It aims to characterize the magnitude of the impact of semantic block changes on control costs, regardless of direction.

[0090] See Figure 5 As shown in Table 1, the pseudocode for the A-HLSAC algorithm is in Algorithm 1, which consists of two stages. The first stage (lines 1–6) involves offline pre-training. Network: Under the ideal transmission assumption, the semantic information generated by JSCC and the decoder network is used as the training dataset and trained independently. Network. The second stage corresponds to A-HLSAC training: lines 19–21 mainly process the real-time transmitted sensor data and calculate control commands based on equation (27); it should be noted that, compared with the sensor data, the video semantic information arrives asynchronously. Accordingly, lines 11–18 mainly describe the process of combining the received semantic information with the current sensor data to calculate the predictive control commands under the STKS-HARQ-based transmission strategy. According to equation (9). From semantic distortion score With semantic contribution score This collective decision allows the scheduler to prioritize retransmitting semantic blocks that are severely distorted but contribute significantly to the task, even under poor channel conditions. Lines 24–30 provide a detailed description of the network update process. When both video data and the retransmission mechanism are disabled, Algorithm 1 degenerates into the HLSAC algorithm.

[0091] Table 1

[0092] (viii) Communication-control collaborative design closed loop See Figure 3 and Figure 4This embodiment employs a dual-stream sensing transmission and fusion framework to achieve a communication-control collaborative design: the sensor data stream serves as a low-latency link, with HLSAC generating real-time control commands based on the received sensor data; the video semantic stream, extracted by a 3D convolutional neural network using STKS, serves as an asynchronous supplementary link, fusing with sensor features through cross-attention to refine control decisions (A-HLSAC algorithm). For semantic stream transmission, the STKS-HARQ transmission strategy is based on semantic distortion scores... With semantic contribution score The two factors are jointly determined: the former measures communication-side distortion, and the latter characterizes the contribution of each semantic block to the control cost. The STKS-HARQ transmission strategy affects control performance through two paths: (1) changing the content of received semantic information, thereby affecting the fusion features and control commands; and (2) changing the AoES evolution and affecting the A-HLSAC strategy update through the reward function. Therefore, the communication transmission strategy and control commands are jointly optimized in the "transmission-fusion-control-feedback" closed loop to achieve communication-control collaborative design.

[0093] (ix) Beneficial effects In summary, compared with the prior art, this application has the following beneficial effects: Advantage 1: Improved robustness of closed-loop control under low signal-to-noise ratio conditions By constructing a real-time-asynchronous dual-stream sensing transmission and fusion framework, semantic information of video data is extracted at the source end and transmitted asynchronously. At the remote end, it is fused with real-time sensor state via a cross-attention mechanism. This application can provide trend-level compensation using global motion trends under low signal-to-noise ratio conditions, compensating for the insufficient environmental and cooperative situational awareness when relying solely on ontology perception, reducing decision bias, and improving the robustness of closed-loop control. Compared to existing solutions that rely solely on ontology sensor state information, this application has significant advantages in the diversity and completeness of sensing sources.

[0094] Advantage 2: Achieving a better trade-off between communication overhead and control precision By proposing a task-centric STKS-HARQ mechanism, selective retransmission is driven by both semantic distortion score and semantic contribution score. This application prioritizes the reliable delivery of key semantic information that contributes significantly to the control task, avoiding wasting communication resources on semantic blocks with minimal impact on control decisions. Compared to existing solutions that make retransmission decisions solely based on communication-side distortion, this application achieves a better trade-off between communication overhead and control accuracy, especially under low signal-to-noise ratio conditions, maintaining higher control accuracy with less retransmission overhead.

[0095] Advantage 3: Ensures the stability of the closed-loop control system By proposing an Effective Semantic Age (AoES) timeliness metric and employing the HLSAC / A-HLSAC algorithm with dual time-scale decomposition, this application directly embeds Lyapunov stability constraints into the reinforcement learning policy optimization process. This enables the joint optimization of communication transmission strategies and control commands while ensuring the stability of the closed-loop control system. This mechanism ensures stable control performance even under dynamic and uncertain environments, which is a crucial technical guarantee for industrial IoT applications.

[0096] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0097] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0098] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the methods described in the embodiments of this application. The input / output interface 603 is used to implement information input and output; The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604); The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0099] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0100] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0101] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0103] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0104] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0105] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0107] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0108] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0109] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0111] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A semantically driven communication-control co-design method, characterized in that, The method includes the following steps: Acquire real-time status data collected by the sensors of each device body, and generate a first control command based on the real-time status data; The external sensing device is activated by an event triggering mechanism to collect external sensing data, and the semantics of the external sensing data are extracted to obtain spatiotemporal motion semantic information. A hybrid automatic repeat request mechanism oriented towards spatiotemporal motion semantics is adopted to transmit the spatiotemporal motion semantic information to the remote controller via the uplink wireless channel. The semantic block of each round of retransmission is determined by the semantic distortion score and semantic contribution score of the semantic block. The remote controller asynchronously fuses the received spatiotemporal motion semantic information with the real-time status data to obtain fused features; A second control command is generated based on the fusion features; Closed-loop control is performed on each device in the industrial Internet of Things system according to the first control command and / or the second control command; The semantic contribution score is used to characterize the degree of influence of each semantic block on the control cost, and the semantic distortion score is used to characterize the degree of distortion of each semantic block during wireless transmission.

2. The method according to claim 1, characterized in that, The process of activating external sensing devices through an event-triggered mechanism, collecting external sensing data, and performing semantic extraction on the external sensing data to obtain spatiotemporal motion semantic information includes: When a new control command is generated, the external sensing device is triggered to collect the external sensing data at the current moment, and the external sensing data is video sequence data; Spatiotemporal motion semantic features are extracted from the video sequence data by a joint source-channel coding module to obtain spatiotemporal motion semantic information, which includes historical and trend features of the system motion. The spatiotemporal motion semantic information is transmitted to the remote controller via an uplink wireless channel, and the remote controller performs noise reduction processing on the received signal through a multilayer perceptron.

3. The method according to claim 1, characterized in that, The hybrid automatic repeat request mechanism oriented towards spatiotemporal motion semantics specifically includes: A1, The receiving end obtains the first The system state error vector after round transmission is calculated, and the uncertainty of the received information is calculated based on the system state error vector; A2, Determine the first Check whether the received signal-to-noise ratio exceeds the threshold value determined by the uncertainty of the received information; if it exceeds, stop retransmission; if it does not exceed, proceed to step A3. A3, the first step is calculated from the distortion estimation network. The semantic distortion score of each semantic block in the round is calculated by the controller network. The semantic contribution score of each semantic block is calculated, and the integrated transmission feedback signal is calculated based on the semantic distortion score and the semantic contribution score. A4. Determine the first [missing information] based on the comprehensive transmission feedback signal. The set of semantic blocks that need to be retransmitted in a round; A5. Repeat steps A1 to A4 until the maximum number of transmission rounds is reached.

4. The method according to claim 3, characterized in that, The calculation method for the integrated transmission feedback signal is as follows: in, For the first The semantic block in the first The indication signal indicating whether the wheel needs to be retransmitted. For indicator functions, For the first The semantic block in the first The semantic distortion score of the wheel, For the first The semantic block in the first The semantic contribution score of the wheel This represents a trade-off between distortion sensitivity and task relevance. This is the acceptable distortion threshold.

5. The method according to claim 1, characterized in that, The semantic contribution score is calculated as follows: During control time The remote controller performs forward reasoning based on the currently received real-time status data and spatiotemporal motion semantic information to generate the second control command; Calculate the control cost under the second control command; With fixed controller network parameters, the control cost is backpropagated with respect to the received spatiotemporal motion semantic information to obtain a semantic contribution score: in, Contribute scores to semantics In order to control costs, For the received improved semantic information ， This is the second control command. Designing networks for semantic-driven control This indicates taking the absolute value of each element.

6. The method according to claim 1, characterized in that, The asynchronous fusion specifically includes: Define a set of fusion moments, which is the union of a first control command moment determined solely by sensor data and a second control command moment jointly determined by sensor data and semantic information; At the current fusion moment, if only real-time status data exists, then the first control command is generated; At the current fusion moment, if real-time state data and received spatiotemporal motion semantic information exist simultaneously, the features of the real-time state data and the features of the spatiotemporal motion semantic information are fused through a cross-attention mechanism to obtain the fused features.

7. The method according to claim 1, characterized in that, The method further includes the following steps: A valid semantic age is defined to characterize the timeliness of the spatiotemporal motion semantic information after it reaches the remote controller. The valid semantic age is calculated as follows: in, The sum of the joint source channel coding delay and the control command delay is calculated. This is the sum of the denoising delay, propagation delay, and feedback delay of the multilayer perceptron. For the current transmission round, For the first The number of semantic blocks retransmitted in round-robin mode. This represents the maximum number of transmission rounds. Furthermore, the effective semantic age is applied as a long-term constraint to the communication-control collaborative optimization process, namely: in, This serves as a long-term constraint threshold for the effective semantic age.

8. The method according to claim 1, characterized in that, The generation of the first control command and / or the second control command is achieved using a hybrid strategy Lyapunov soft actor-critic algorithm, the algorithm comprising: Construct a constrained Markov decision process model, where the state space contains the system state and the desired state, and the action space is the set of control commands. Maintain two rewarded critic networks and employ double-Q learning to reduce overestimation bias; A Lyapunov critic network is constructed to evaluate the stability of the control system; the Lyapunov critic network is constructed as a positive definite function. Construct an actor network and update the policy parameters by maximizing an objective function that includes entropy regularization and Lyapunov stability constraints; A hybrid strategy guided by domain knowledge is adopted to linearly interpolate the learned control instructions output by the actor network with the control priors to generate the final control instructions.

9. The method according to claim 1, characterized in that, The method further includes decomposing the communication-control joint optimization problem into fast-time-scale subproblems and asynchronous-time-scale subproblems, wherein: The fast timescale subproblem is used to solve the first control command that depends only on real-time state data and is executed at the moment when the sensor data is updated. The asynchronous time-scale subproblem is used to jointly optimize the semantic extraction strategy, semantic transmission strategy, and compensation control instructions, and is executed at the moment the semantic information arrives. The fast timescale subproblem and the asynchronous timescale subproblem are coupled together through system state dynamics and long-term effective semantic age constraints.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 9.

Images