A method and apparatus for improving the reliability of small data packet service transmission

Abstract

This invention discloses a method and apparatus for improving the reliability of small data packet service transmission, relating to the field of data communication technology. The method includes: acquiring user node parameters and initializing the algorithm parameters of a KP-DQN model; calculating the arrival rate of service data based on the user node parameters and analyzing the statistical characteristics of the channel where the user node is located; calculating the service delay violation probability based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, and storing it in a knowledge base; pre-training the initial KP-DQN model based on the user node's idle resources and the knowledge base; selecting the user node's transmission mode, frame duration, and number of data packet replications based on the pre-trained KP-DQN model and the user node's state, thereby making a decision on the reliability improvement issue and obtaining a decision result; and completing the data packet transmission of the user node based on the decision result. Using this invention, the reliability of service transmission can be improved.

Description

Technical Field

[0001] This invention relates to the field of data communication technology, and in particular to a method and apparatus for improving the reliability of small data packet transmission. Background Technology

[0002] With the rapid development of mobile communication technology, especially in future 6G networks, the performance requirements of communication systems are constantly increasing. Particularly for small data packet services, the reliability requirement has reached an extremely high standard of 99.99999%. To meet this high reliability requirement, a common technique is to copy data and transmit it multiple times. While this significantly improves reliability, it also introduces an additional data transmission burden, leading to reduced system efficiency.

[0003] Meanwhile, the rise of semantic communication technology has provided a new direction for solving this problem. Semantic communication transmits information by extracting the core semantics rather than the raw data, thereby reducing the amount of data transmitted and improving transmission efficiency. However, the semantic extraction process takes time, which may increase transmission latency for latency-sensitive small data packet services. Therefore, how to combine data replication transmission with semantic communication and improve transmission reliability without increasing latency has become an urgent problem to be solved in the field of communications.

[0004] Current communication methods mostly rely on either replication frequency or semantic communication to improve reliability or reduce data transmission volume, but each method has its own advantages and disadvantages. For example, multiple replications can improve data reliability, but they increase the amount of data transmitted, affecting system bandwidth and energy efficiency; while semantic communication can reduce the amount of data transmitted, the time-consuming semantic extraction process may affect real-time performance.

[0005] Most existing solutions focus on the static selection of communication strategies, such as fixed replication counts or frame durations, and have poor adaptability to dynamic system states. Furthermore, artificial intelligence has a certain foundation in communication optimization, with intelligent algorithms such as reinforcement learning being used to select the optimal communication strategy in real time. However, these methods typically optimize within a fixed framework, failing to fully consider how to jointly optimize different transmission methods, frame durations, and replication counts to address varying service requirements and network state changes. Summary of the Invention

[0006] This invention provides a method and apparatus for improving the reliability of small data packet transmission. The technical solution is as follows:

[0007] On the one hand, a method for improving the transmission reliability of small data packet services is provided. This method is implemented by a device for improving the transmission reliability of small data packet services, and includes:

[0008] S1. Obtain the parameters of the user nodes and initialize the algorithm parameters of the KP-DQN model;

[0009] S2. Based on the parameters of the user node, calculate the arrival rate of service data and analyze the statistical characteristics of the channel where the user node is located.

[0010] S3. Calculate the service delay violation probability based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, and store it in the knowledge base.

[0011] S4. Based on the user node's idle resources and knowledge base, pre-train the initial KP-DQN model;

[0012] S5. Based on the pre-trained KP-DQN model and the state of the user nodes, select the transmission mode, frame duration, and number of data packet replications for the user nodes, and then make decisions on the reliability improvement issue to obtain the decision results.

[0013] S6. Based on the decision result, complete the data packet transmission of the user node.

[0014] On the other hand, an apparatus for improving the reliability of small data packet service transmission is provided. This apparatus is applied to a method for improving the reliability of small data packet service transmission. The apparatus includes:

[0015] The initialization module is used to obtain the parameters of user nodes and initialize the algorithm parameters of the KP-DQN model.

[0016] The analysis module is used to calculate the arrival rate of service data based on the parameters of the user node, and to analyze the statistical characteristics of the channel in which the user node is located.

[0017] The calculation module is used to calculate the probability of service delay violation based on the parameters of the user node, the arrival rate of service data, and the statistical characteristics of the channel, and store it in the knowledge base.

[0018] The training module is used to pre-train the initial KP-DQN model based on the user node's idle resources and knowledge base;

[0019] The decision module is used to select the transmission mode, frame duration, and number of data packet replications of user nodes based on the pre-trained KP-DQN model and the state of user nodes, thereby making decisions on the reliability improvement issue and obtaining the decision results.

[0020] The transmission module is used to complete the data packet transmission of the user node based on the decision result.

[0021] On the other hand, an apparatus for improving the reliability of small data packet service transmission is provided. The apparatus for improving the reliability of small data packet service transmission includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the methods described above for improving the reliability of small data packet service transmission is implemented.

[0022] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the methods described above for improving the reliability of small data packet service transmission.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] The system acquires user node parameters and initializes the algorithm parameters of the KP-DQN model. Based on the user node parameters, it calculates the arrival rate of service data and analyzes the statistical characteristics of the channel where the user node is located. Based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, it calculates the service delay violation probability and stores it in a knowledge base. Based on the user node's idle resources and the knowledge base, it pre-trains the initial KP-DQN model. Based on the pre-trained KP-DQN model and the user node's state, it selects the user node's transmission mode, frame duration, and data packet replication count, thereby making decisions on reliability improvement and obtaining the decision results. Based on the decision results, it completes the data packet transmission of the user node. This invention improves system communication reliability by transforming information into semantic transmission and combining flexible frame duration and data replication transmission. In this embodiment, semantic communication or direct communication can be selected, different frame durations can be used, and different data replication transmission counts can be selected. The selection rule is based on maximizing service quality, and the optimal selection of transmission mode, frame duration, and replication count under different system states is calculated using artificial intelligence, thereby improving data transmission reliability. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a method for improving the transmission reliability of small data packet services provided by an embodiment of the present invention;

[0027] Figure 2This is a block diagram of a device for improving the reliability of small data packet service transmission according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the structure of a device for improving the reliability of small data packet service transmission provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0031] In the embodiments of the present invention, the terms "image" and "picture" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same. The terms "of," "corresponding," and "corresponding" may be used interchangeably. It should be noted that, when the distinction between them is not emphasized, their intended meanings are the same.

[0032] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0033] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0034] This invention provides a method for improving the reliability of small data packet service transmission. This method can be implemented by a device for improving the reliability of small data packet service transmission, which can be a terminal or a server. Figure 1 The flowchart shown illustrates a method for improving the reliability of small data packet transmission. This method's processing flow may include the following steps:

[0035] S1. Obtain the parameters of the user nodes and initialize the algorithm parameters of the KP-DQN model.

[0036] Optionally, the parameters of the user node include latency requirements, reliability requirements, packet size, bandwidth, and power.

[0037] The algorithm parameters of the KP-DQN model include the Actor learning rate, Critic learning rate, discount factor, and decay factor.

[0038] In one feasible implementation, after initializing the algorithm parameters of the model, it is also necessary to initialize an experience replay memory to store state transition tuples (s,a,r,s'); and initialize a target network with the same structure as the Q network but different parameters to stabilize the learning process.

[0039] S2. Based on the parameters of the user node, calculate the arrival rate of service data and analyze the statistical characteristics of the channel where the user node is located.

[0040] Optionally, the specific operation steps of S2 may include S21-S22:

[0041] S21. Calculate the arrival rate of business data based on the average value over a certain period of time.

[0042] In one feasible implementation, in addition to statistical methods, convolution can also be used to calculate the arrival rate of business data. The method for calculating the arrival rate can adopt commonly used calculation methods in the prior art, which will not be elaborated here in this embodiment of the invention.

[0043] S22. Based on the channel information reported by the user over a certain period of time, perform statistical distribution analysis on the channel information, fit it with the existing channel model, and select the channel model with the smallest mean square error as the user's channel statistical characteristics. The channel statistical characteristics include channel fading, noise, interference, and signal quality fluctuations.

[0044] S3. Calculate the service delay violation probability based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, and store it in the knowledge base.

[0045] Alternatively, the specific steps for the root operation of S3 can be as follows:

[0046] Based on the arrival rate of the service data, the statistical characteristics of the channel, and the following formula (1), the service delay violation probability is calculated:

[0047]

[0048] in,

[0049] in,

[0050] in,

[0051] Where, ε kLet θ represent the probability of delay violation, be a positive parameter in the random network calculus, and N. k Indicates the block length. Let ε denote the inverse function of the Gaussian Q-function. c For decoding error rate, B represents the user's latency requirements. k W k Indicates the user's bandwidth; B k Indicates the bandwidth of the resource block; W k Let G represent the number of resource blocks; G represent the Meijer G function; α represent the first channel parameter; Ω represent the second channel parameter; Γ(G) represent the gamma function of G; Γ(Z) represent the gamma function of Z; where, to make the expression for the delay violation probability clearer, Z, U, Y, and C are used to replace the relevant variables; Z represents the first replaced relevant variable; U represents the second replaced relevant variable; Y represents the third replaced relevant variable; C represents the third replaced relevant variable; G G This represents the power form of the Meijer G function.

[0052] S4. Based on the user node's idle resources and knowledge base, pre-train the initial KP-DQN model.

[0053] Among them, the KP-DQN model is a knowledge-based parameterized deep Q-network.

[0054] Optionally, the specific operation steps of S4 may include S41-S45:

[0055] S41. Reset environment status;

[0056] S42. Check the knowledge base: If the current state is recorded in the knowledge base, extract the knowledge directly from the knowledge base and jump to S44; if the current state is not recorded in the knowledge base, execute S43.

[0057] S43. Calculate the probability of service latency violation: Calculate the probability of service latency violation corresponding to the current state;

[0058] Execute action: Select and execute an action according to the strategy, and obtain a new state;

[0059] Environment state update: Update the environment state based on the action result;

[0060] Store experience: Store the current experience quadruple [state, action, reward, state at the next moment] into the experience pool;

[0061] S44. Update network parameters: Optimize the model network through the following steps:

[0062] Calculate the target Q value:

[0063]

[0064] Among them, y t r represents the target Q value at the current moment. t This represents the immediate reward at the current moment. Let s represent the discount factor, a′ represent the action at the next moment, and s represent the action at the next moment. t+1 ω represents the state at the next moment. - These represent the parameters of the target Q-network;

[0065] Actor network update: Calculate TD error according to the formula and update;

[0066]

[0067] Among them, L A Let E represent the TD error of the Actor network, E represent the expectation, and ω represent the network parameters.

[0068] Critic network update: Calculate and update the TD error according to the formula;

[0069]

[0070] Among them, L C This represents the TD error of the Critic network.

[0071] S45. Determine if the current step count has reached the preset step count. If not, increment the current step count by 1 and jump to S41 for iterative training. If the preset step count has been reached, stop the iteration and end the pre-training.

[0072] In one feasible implementation, the knowledge base is a repository that stores the calculation results of the latency violation probability. Since the calculation process shown in formula (1) is very cumbersome, requiring calculation in every step of the KP-DQN algorithm's learning, it introduces significant computational latency and greatly reduces the algorithm's operating efficiency. Therefore, the latency violation probability is pre-calculated on idle resources of the base station or node, and the parameters used for calculation and the corresponding latency violation probability results are saved to form a knowledge base. When the algorithm is trained, it only needs to check if there are corresponding calculation results in the knowledge base, thereby avoiding redundant calculations and improving the algorithm's operating efficiency.

[0073] S5. Based on the pre-trained KP-DQN model and the state of the user nodes, select the transmission mode, frame duration, and number of data packet replications for the user nodes, and then make decisions on the reliability improvement issue to obtain the decision results.

[0074] Optionally, the specific operation steps of S5 may include S51-S54:

[0075] S51. Input the state of the user node into the pre-trained KP-DQN model. The state is represented as s = {s1, s2, s3, s4}, where s1 represents available resources, including idle bandwidth and available power at the current time, s2 represents channel quality, s3 represents user requirements, including latency requirements and reliability requirements, and s4 represents the number of constraint violations, used to quantify the negative impact of actions on rewards.

[0076] S52. Define the action space as a = {a1, a2, a3, a4, a5}, where a1 represents the transmission mode, which includes bit transmission and semantic transmission; a2 represents the frame duration; a3 represents the number of resource blocks; a4 represents the number of data packet copies; and a5 represents the semantic compression rate.

[0077] In one feasible implementation, the action space includes hybrid actions and continuous actions. The transmission mode of a1, the frame duration of a2, and the number of resource blocks of a3 are hybrid actions, while the number of data packet copies of a4 and the semantic compression rate of a5 are continuous actions. The KP-DQN algorithm divides hybrid actions into discrete actions and continuous parameters and models them separately, and jointly optimizes the Q-values ​​of discrete actions and continuous parameters. For discrete actions, the classic DQN algorithm is used to learn the Q-value. For continuous actions, a parameterized network is used for optimization. The parameterized network is a neural network used to generate continuous parameters corresponding to discrete actions in the hybrid action space to solve the problem of joint optimization of discrete and continuous actions. The input of the parameterized network is the state and discrete actions, and the output is the continuous parameters corresponding to the discrete actions. The optimization goal is to maximize the generated Q-value. The working method may include: (1) learning the relationship between discrete actions and continuous parameters: for each discrete action, the optimal continuous parameters are dynamically generated. (2) optimizing continuous parameters through gradient: using the Q-value to backpropagate through the parameterized network to learn to generate better continuous parameters.

[0078] This separation significantly reduces computational complexity, avoids direct searching of the mixed action space, and improves efficiency and stability. These will be described in detail below:

[0079] The transmission mode of a1 allows services to choose between bit transmission and semantic transmission. Semantic transmission compresses data packets, saving bandwidth and improving anti-interference capabilities in low signal-to-noise ratio conditions. However, semantic transmission introduces encoding and decoding latency, which is unfriendly to users with extremely strict latency requirements. Therefore, the method proposed in this invention allows for flexible selection of the service transmission mode based on the user's latency requirements and channel conditions.

[0080] For semantic transmission, the semantic compression ratio determines the size of the data packet, which in turn affects the probability of latency violation and the semantic similarity. Simultaneously, the semantic compression ratio determines the number of computation cycles required for semantic extraction, also affecting the encoding and decoding latency of the semantic portion. Therefore, this invention will flexibly select the semantic compression ratio based on user needs and channel conditions. Specifically, the semantic compression ratio ranges from 0 to 1; a higher value indicates a higher compression ratio and a smaller data packet.

[0081] a2 frame duration: The frame duration can be selected from three different frame durations: 1 millisecond, 0.5 milliseconds, and 0.25 milliseconds.

[0082] Number of resource blocks for a3: Allocate bandwidth resources for this user.

[0083] The number of packet copies for A4: Packet copying can be selected to copy the transmitted data 0 to 2 times. Zero copying means no copying. Packet copying transmission can significantly improve the reliability of wireless transmission. Packet copying can be flexibly selected in terms of whether to copy and the number of copies. When packet copying transmission is selected, the reliability is the product of the reliability of multiple links. However, packet copying actually sacrifices system resources for improved reliability. Specifically, because this invention uses flexible frame duration, when a short frame duration is used, the frequency domain occupancy is relatively large, so packet copying can be completed in the time domain. For a long frame duration, data copying can be completed in the frequency domain.

[0084] The number of packet copies for a5 affects the accuracy of semantic transmission and recovery, thus impacting reliability metrics. For traditional users, a3 = 0.

[0085] S53. Set the reward function: For bit transmission mode, set the reward function according to the following formula (2):

[0086] Ψ b =1-(1-ε k )(1-ε c )≈ε k +ε c (2)

[0087] Where, ε k Let ε represent the probability of time delay violation calculated by formula (1). c This indicates the decoding error rate.

[0088] For semantic transmission mode, the reward function is set according to the following formula (3):

[0089]

[0090] Where, Φ sThe semantic similarity, representing semantic transmission, is related to the semantic compression ratio, Φ max This represents the maximum semantic similarity.

[0091] In one feasible implementation, for bit users, reliability is jointly characterized by the delay violation probability and the decoding error rate. The delay violation probability is calculated using random network calculus, and the decoding error rate is considered a constant in this invention. Since the magnitudes of the two numbers are very small, the reliability is approximately the sum of the two numbers.

[0092] For semantic users, reliability is measured by two metrics: latency violation probability and semantic similarity. Semantic similarity is a function of semantic compression ratio and can be obtained through simulation fitting. Considering that latency violation probability and semantic similarity are not on the same order of magnitude, semantic similarity is first normalized before being expressed in a unified manner.

[0093] If the constraints cannot be met, the reward value is 0.

[0094] S54. Based on the state of the user node, calculate the next hybrid action, which includes discrete actions and continuous actions. The discrete actions include transmission mode, frame duration, number of resource blocks, and number of data packet copies. The continuous actions include semantic compression rate and power.

[0095] S55. Calculate the reward function based on the selected mixed action, put the state transition group into the experience replay pool, and obtain the decision result.

[0096] In one feasible implementation, after calculating the reward function based on the selected mixed action, the state transition group (s,a,r,s') is placed into the experience replay pool for fine-tuning and parameter updating of KP-DQN.

[0097] Optionally, the decision results include transmission mode, frame duration, number of packet replications, and resource allocation schemes for power and bandwidth.

[0098] S6. Based on the decision results, complete the data packet transmission for the user nodes.

[0099] In one feasible implementation, the decision result obtained includes a transmission mode of bit transmission or semantic transmission, a frame duration of 1 millisecond, 0.5 milliseconds, or 0.25 milliseconds, and a specific number of data packet replications. Based on these data, the user node transmits data according to a resource allocation scheme for power and bandwidth.

[0100] It should be noted that in practical use, KP-DQN can be continuously adjusted and optimized using predicted data. Other methods include:

[0101] Based on the learning records, the KP-DQN algorithm model was fine-tuned online, and the parameters of the adjusted neural network were updated.

[0102] One feasible implementation method is to continuously optimize the model to further enhance its superiority, making the selected transmission scheme more reliable and better.

[0103] In this embodiment of the invention, user node parameters are obtained, and the algorithm parameters of the KP-DQN model are initialized. Based on the user node parameters, the arrival rate of service data is calculated, and the statistical characteristics of the channel where the user node is located are analyzed. Based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, the service delay violation probability is calculated and stored in a knowledge base. Based on the user node's idle resources and the knowledge base, the initial KP-DQN model is pre-trained. Based on the pre-trained KP-DQN model and the user node's state, the transmission mode, frame duration, and number of data packet replications for the user node are selected, thereby making a decision on reliability improvement and obtaining a decision result. Based on the decision result, the data packet transmission of the user node is completed. Using this invention, by transforming information into semantic transmission and combining flexible frame duration and data replication transmission, the reliability of system communication is improved. In this embodiment of the invention, semantic communication or direct communication can be selected, different frame durations can be selected, and different numbers of data replication transmissions can be selected. The selection rules are based on maximizing service quality. Artificial intelligence is used to calculate the optimal choice for transmission method, frame duration, and number of replications under different system states, thereby improving the reliability of data transmission.

[0104] Figure 2 This is a block diagram of an apparatus for improving the reliability of small data packet service transmission according to an exemplary embodiment. The apparatus is used for a method of improving the reliability of small data packet service transmission. (Refer to...) Figure 2 The device includes:

[0105] Initialization module 210 is used to obtain the parameters of user nodes and initialize the algorithm parameters of the KP-DQN model;

[0106] Analysis module 220 is used to calculate the arrival rate of service data based on the parameters of the user node and to analyze the statistical characteristics of the channel where the user node is located.

[0107] The calculation module 230 is used to calculate the service delay violation probability based on the parameters of the user node, the arrival rate of service data and the statistical characteristics of the channel, and store it in the knowledge base.

[0108] Training module 240 is used to pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user nodes;

[0109] The decision module 250 is used to select the transmission mode, frame duration and number of data packet replications of the user node based on the pre-trained KP-DQN model and the state of the user node, and then make decisions on the reliability improvement issue to obtain the decision result.

[0110] The transmission module 260 is used to complete the data packet transmission of the user node based on the decision result.

[0111] The initialization module is used to obtain the parameters of user nodes and initialize the algorithm parameters of the KP-DQN model.

[0112] The analysis module is used to calculate the arrival rate of service data based on the parameters of the user node, and to analyze the statistical characteristics of the channel in which the user node is located.

[0113] The calculation module is used to calculate the probability of service delay violation based on the parameters of the user node, the arrival rate of service data, and the statistical characteristics of the channel, and store it in the knowledge base.

[0114] The training module is used to pre-train the initial KP-DQN model based on the user node's idle resources and knowledge base;

[0115] The decision module is used to select the transmission mode, frame duration, and number of data packet replications of user nodes based on the pre-trained KP-DQN model and the state of user nodes, thereby making decisions on the reliability improvement issue and obtaining the decision results.

[0116] The transmission module is used to complete the data packet transmission of the user node based on the decision result.

[0117] In this embodiment of the invention, user node parameters are obtained, and the algorithm parameters of the KP-DQN model are initialized. Based on the user node parameters, the arrival rate of service data is calculated, and the statistical characteristics of the channel where the user node is located are analyzed. Based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, the service delay violation probability is calculated and stored in a knowledge base. Based on the user node's idle resources and the knowledge base, the initial KP-DQN model is pre-trained. Based on the pre-trained KP-DQN model and the user node's state, the transmission mode, frame duration, and number of data packet replications for the user node are selected, thereby making a decision on reliability improvement and obtaining a decision result. Based on the decision result, the data packet transmission of the user node is completed. Using this invention, by transforming information into semantic transmission and combining flexible frame duration and data replication transmission, the reliability of system communication is improved. In this embodiment of the invention, semantic communication or direct communication can be selected, different frame durations can be selected, and different numbers of data replication transmissions can be selected. The selection rules are based on maximizing service quality. Artificial intelligence is used to calculate the optimal choice for transmission method, frame duration, and number of replications under different system states, thereby improving the reliability of data transmission.

[0118] Figure 3 This is a schematic diagram of the structure of a device for improving the reliability of small data packet service transmission according to an embodiment of the present invention, as shown below. Figure 3 As shown, devices for improving the reliability of small data packet service transmission can include the above-mentioned... Figure 2 The apparatus shown is for improving the reliability of small data packet service transmission. Optionally, the apparatus 310 for improving the reliability of small data packet service transmission may include a first processor 2001.

[0119] Optionally, the device 310 for improving the reliability of small data packet service transmission may also include a memory 2002 and a transceiver 2003.

[0120] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0121] The following combination Figure 3 The following is a detailed introduction to the various components of the device 310 for improving the reliability of small data packet service transmission:

[0122] The first processor 2001 is the control center of the device 310 for improving the reliability of small data packet service transmission. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0123] Optionally, the first processor 2001 can execute various functions of the device 310 to improve the reliability of small data packet service transmission by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0124] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0125] In a specific implementation, as one example, the device 310 for improving the reliability of small data packet service transmission may also include multiple processors, such as... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0126] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0127] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently, and may be connected via the interface circuit of the device 310 for improving the reliability of small data packet service transmission. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0128] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0129] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0130] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the device 310 that improves the reliability of small data packet service transmission. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0131] It should be noted that, Figure 3 The structure of the device 310 shown in the diagram for improving the reliability of small data packet service transmission does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0132] Furthermore, the technical effects of the device 310 that improves the reliability of small data packet service transmission can be referred to the technical effects of the method for improving the reliability of small data packet service transmission described in the above method embodiments, and will not be repeated here.

[0133] It should be understood that the first processor 2001 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0134] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0135] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0136] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0137] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0138] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0141] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for improving the reliability of small data packet service transmission, characterized in that, The method includes: S1. Obtain the parameters of the user nodes and initialize the algorithm parameters of the KP-DQN model; S2. Based on the parameters of the user node, calculate the arrival rate of service data and analyze the statistical characteristics of the channel where the user node is located. S3. Calculate the service delay violation probability based on the user node parameters, the arrival rate of service data, and the statistical characteristics of the channel, and store it in the knowledge base. S3 includes: Based on the arrival rate of the service data, the statistical characteristics of the channel, and the following formula (1), the service delay violation probability is calculated: （1） in, ; in, ; in, ; in, Indicates the probability of delay violation. For positive parameters in random network calculus, Indicates the block length. This represents the inverse function of the Gaussian Q-function. For decoding error rate, Indicates the user's latency requirements. Indicates the user's bandwidth; Indicates the bandwidth of the resource block; Indicates the number of resource blocks; G represents the Meijer G function; Indicates the first parameter of the channel; This is the second parameter of the channel; Represents the gamma function of G; Let Z be the gamma function; where, to make the expression for the time delay violation probability clearer, the relevant variables are replaced by Z, U, Y, and C; Z represents the first replaced relevant variable; U represents the second replaced relevant variable; Y represents the third replaced relevant variable; and C represents the fourth replaced relevant variable. Represents the power form of the Meijer G function; S4. Based on the user node's idle resources and knowledge base, pre-train the initial KP-DQN model; S5. Based on the pre-trained KP-DQN model and the state of the user nodes, select the transmission mode, frame duration, and number of data packet replications for the user nodes, and then make decisions on the reliability improvement issue to obtain the decision results. S6. Based on the decision result, complete the data packet transmission of the user node.

2. The method for improving the reliability of small data packet service transmission according to claim 1, characterized in that, The parameters of the user node include latency requirements, reliability requirements, data packet size, bandwidth, and power. The algorithm parameters of the KP-DQN model include Actor learning rate, Critic learning rate, discount factor, and decay factor.

3. The method for improving the reliability of small data packet service transmission according to claim 1, characterized in that, S2 calculates the arrival rate of service data based on the parameters of the user node and analyzes the statistical characteristics of the channel where the user node is located, including: S21. Calculate the arrival rate of business data based on the average value over a certain period of time. S22. Based on the channel information reported by the user over a certain period of time, perform statistical distribution analysis on the channel information, fit it with the existing channel model, and select the channel model with the smallest mean square error as the user's channel statistical characteristics. The channel statistical characteristics include channel fading, noise, interference, and signal quality fluctuations.

4. The method for improving the reliability of small data packet service transmission according to claim 1, characterized in that, S4 pre-trains the initial KP-DQN model based on the user node's idle resources and knowledge base, including: S41. Reset environment status; S42. Check the knowledge base: If the current state is recorded in the knowledge base, extract the knowledge directly from the knowledge base and jump to S44; if the current state is not recorded in the knowledge base, execute S43. S43. Calculate the probability of service latency violation: Calculate the probability of service latency violation corresponding to the current state; Execute action: Select and execute an action according to the strategy, and obtain a new state; Environment state update: Update the environment state based on the action result; Store experience: Store the current experience quadruple [state, action, reward, state at the next moment] into the experience pool; S44. Update network parameters: Optimize the model network through the following steps: Calculate the target Q value: ； in, This represents the target Q value at the current moment. This represents the immediate reward at the current moment. Indicates the discount factor. Indicates the action to be taken at the next moment. Indicates the state at the next moment. These represent the parameters of the target Q-network; Actor network update: Calculate TD error according to the formula and update; ； in, This represents the TD error of the Actor network. Expressing expectations, Indicates network parameters; Critic network update: Calculate and update the TD error according to the formula; ； in, This represents the TD error of the Critic network; S45. Determine if the current step count has reached the preset step count. If not, increment the current step count by 1 and jump to S41 for iterative training. If the preset step count has been reached, stop the iteration and end the pre-training.

5. The method for improving the reliability of small data packet service transmission according to claim 1, characterized in that, S5 selects the transmission mode, frame duration, and number of data packet replications for user nodes based on the pre-trained KP-DQN model and the state of the user nodes, thereby making decisions on reliability improvement and obtaining decision results, including: S51. Input the state of the user node into the pre-trained KP-DQN model. The state is represented as s={s1,s2,s3,s4}, where s1 represents available resources, including idle bandwidth and available power at the current time, s2 represents channel quality, s3 represents user requirements, including latency requirements and reliability requirements, and s4 represents the number of constraint violations, used to quantify the negative impact of actions on rewards. S52. Define the action space as a={a1,a2,a3,a4,a5}, where a1 represents the transmission mode, which includes bit transmission and semantic transmission; a2 represents the frame duration; a3 represents the number of resource blocks; a4 represents the number of data packet copies; and a5 represents the semantic compression rate. S53. Set the reward function: For bit transmission mode, set the reward function according to the following formula (2): （2） in, This represents the probability of time delay violation calculated by formula (1). Indicates the decoding error rate; For semantic transmission mode, the reward function is set according to the following formula (3): （3） in, The semantic similarity, representing semantic transmission, is related to the semantic compression ratio. This represents the maximum semantic similarity. S54. Based on the state of the user node, calculate the next hybrid action, which includes discrete actions and continuous actions. The discrete actions include transmission mode, frame duration, number of resource blocks, and number of data packet copies. The continuous actions include semantic compression rate and power. S55. Calculate the reward function based on the selected mixed action and put the state transition group into the experience replay pool.

6. The method for improving the reliability of small data packet service transmission according to claim 1, characterized in that, The decision results include transmission mode, frame duration, number of data packet replications, and resource allocation schemes for power and bandwidth.

7. An apparatus for improving the reliability of small data packet service transmission, wherein the apparatus is used to implement the method for improving the reliability of small data packet service transmission as described in any one of claims 1-6, characterized in that, The device includes: The initialization module is used to obtain the parameters of user nodes and initialize the algorithm parameters of the KP-DQN model. The analysis module is used to calculate the arrival rate of service data based on the parameters of the user node, and to analyze the statistical characteristics of the channel in which the user node is located. The calculation module is used to calculate the probability of service delay violation based on the parameters of the user node, the arrival rate of service data, and the statistical characteristics of the channel, and store it in the knowledge base. The training module is used to pre-train the initial KP-DQN model based on the user node's idle resources and knowledge base; The decision module is used to select the transmission mode, frame duration, and number of data packet replications of user nodes based on the pre-trained KP-DQN model and the state of user nodes, thereby making decisions on the reliability improvement issue and obtaining the decision results. The transmission module is used to complete the data packet transmission of the user node based on the decision result.

8. A device for improving the reliability of small data packet service transmission, characterized in that, The devices for improving the reliability of small data packet service transmission include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images