Method and device for improving transmission reliability of small data packet service

Abstract

The invention discloses a method and a device for improving the transmission reliability of a small data packet service, and relates to the technical field of data communication. The method comprises the following steps: acquiring parameters of a user node, and initializing algorithm parameters of a KP-DQN model; calculating the arrival rate of the service data according to the parameters of the user node, and analyzing the statistical characteristics of a channel where the user node is located; calculating a business time delay violation probability according to the parameters of the user nodes, the arrival rate of the business data and the statistical characteristics of the channel, and storing the business time delay violation probability into a knowledge base; pre-training the initial KP-DQN model according to the idle resources of the user nodes and a knowledge base; according to a pre-trained KP-DQN model and the state of the user node, selecting a transmission mode, a frame duration size and a data packet copying frequency of the user node, and then making a decision on a reliability improvement problem to obtain a decision result; and completing data packet transmission of the user node according to the decision result. According to the invention, the reliability of service transmission can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data communication technology, and in particular to a method and device for improving the transmission reliability of small data packet services. Background Art

[0002] With the rapid development of mobile communication technology, especially in the future 6G network, the performance requirements of communication systems are constantly increasing. In particular, for small data packet services, the reliability requirement has reached an extremely high standard of 99.99999%. In order to meet this high reliability requirement, a common technical means is to replicate data for multiple transmissions. Although it can significantly improve reliability, it will bring additional data transmission burden, resulting in reduced system efficiency.

[0003] At the same time, the rise of semantic communication technology provides a new direction for solving this problem. Semantic communication extracts the core semantics of information instead of the original data for transmission, thereby reducing the amount of data transmitted and improving transmission efficiency. However, the semantic extraction process takes a certain amount of time, which may increase transmission delays for small data packet services that are sensitive to latency. Therefore, how to combine data replication transmission with semantic communication and improve transmission reliability without increasing latency has become a difficult problem that needs to be solved in the communication field.

[0004] Most current communication methods rely solely on replication or semantic communication to improve reliability or reduce data transmission volume, but both methods have their own advantages and disadvantages. For example, replicating multiple transmissions can improve data reliability, but increases the amount of data transmitted, affecting the bandwidth and energy efficiency of the system; while semantic communication can reduce the amount of data transmitted, but due to the time-consuming semantic extraction process, it may affect real-time performance.

[0005] Most existing solutions focus on static selection of communication strategies, such as fixed number of replications or frame durations, and have poor dynamic adaptability to system status. In addition, the application of artificial intelligence in communication optimization has a certain foundation, and intelligent algorithms such as reinforcement learning are used to select the optimal communication strategy in real time. However, these methods are usually optimized within a fixed framework, and fail to fully consider how to jointly optimize different transmission modes, frame durations, and number of replications to cope with different business needs and changes in network status. Summary of the invention

[0006] The embodiment of the present invention provides a method and device for improving the transmission reliability of small data packet services. The technical solution is as follows:

[0007] On the one hand, a method for improving the transmission reliability of a small data packet service is provided, the method being implemented by a device for improving the transmission reliability of a small data packet service, the method comprising:

[0008] S1. Obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model;

[0009] S2. Calculate the arrival rate of service data based on the parameters of the user node and analyze the statistical characteristics of the channel where the user node is located;

[0010] S3. Calculate the service delay violation probability based on the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base;

[0011] S4. Pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user node;

[0012] S5. According to the pre-trained KP-DQN model and the status of the user node, the transmission mode, frame duration, and number of data packet replications of the user node are selected, and then a decision is made on the reliability improvement problem to obtain a decision result;

[0013] S6. According to the decision result, complete the data packet transmission of the user node.

[0014] On the other hand, a device for improving the transmission reliability of a small data packet service is provided, and the device is applied to a method for improving the transmission reliability of a small data packet service, and the device includes:

[0015] Initialization module, used to obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model;

[0016] An analysis module, used to calculate the arrival rate of service data according to the parameters of the user node, and analyze the statistical characteristics of the channel where the user node is located;

[0017] A calculation module, used to calculate the service delay violation probability according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base;

[0018] The training module is used to pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user node;

[0019] The decision module is used to select the transmission mode, frame duration, and number of data packet replications of the user node according to the pre-trained KP-DQN model and the status of the user node, and then make a decision on the reliability improvement problem and obtain the decision result;

[0020] The transmission module is used to complete the data packet transmission of the user node according to the decision result.

[0021] On the other hand, a device for improving the transmission reliability of small data packet services is provided, and the device for improving the transmission reliability of small data packet services includes: a processor; a memory, and the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the methods for improving the transmission reliability of small data packet services as described above is implemented.

[0022] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned methods for improving the transmission reliability of small data packet services.

[0023] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0024] Obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model; calculate the arrival rate of the service data according to the parameters of the user node, and analyze the statistical characteristics of the channel where the user node is located; calculate the service delay violation probability according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base; pre-train the initial KP-DQN model according to the idle resources of the user node and the knowledge base; select the transmission mode, frame duration and number of data packet replication of the user node according to the pre-trained KP-DQN model and the state of the user node, and then make a decision on the reliability improvement problem to obtain a decision result; according to the decision result, complete the data packet transmission of the user node. The present invention is adopted to improve the reliability of system communication by converting information into semantic transmission and combining flexible frame duration and data replication transmission. In an embodiment of the present invention, semantic communication or direct communication can be selected, different frame durations can be selected, and different data replication transmission times can be selected. The selection rule takes the maximum quality of service as the standard, and calculates the optimal selection of transmission mode, frame duration and number of replications under different system states by artificial intelligence, thereby improving the reliability of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 It is a flow chart of a method for improving the transmission reliability of small data packet services provided by an embodiment of the present invention;

[0027] Figure 2It is a block diagram of a device for improving the transmission reliability of small data packet services provided by an embodiment of the present invention;

[0028] Figure 3 It is a structural diagram of a device for improving the transmission reliability of small data packet services provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0030] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0031] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0032] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0033] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0034] The embodiment of the present invention provides a method for improving the transmission reliability of small data packet services. The method can be implemented by a device for improving the transmission reliability of small data packet services. The device for improving the transmission reliability of small data packet services can be a terminal or a server. Figure 1 The method flow chart of improving the transmission reliability of small data packet service shown in the figure, the processing flow of the method may include the following steps:

[0035] S1. Obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model.

[0036] Optionally, the parameters of the user node include delay requirement, reliability requirement, data packet size, bandwidth and power.

[0037] The algorithm parameters of the KP-DQN model include Actor learning rate, Critic learning rate, discount factor, and decay factor.

[0038] In a feasible implementation, after initializing the algorithm parameters of the model, it is also necessary to initialize an experience replay pool (Replay Memory) for storing state transition tuples (s, a, r, s'); initialize a target network with the same structure as the Q network but different parameters to stabilize the learning process.

[0039] S2. Calculate the arrival rate of service data based on the parameters of the user node and analyze the statistical characteristics of the channel where the user node is located.

[0040] Optionally, the specific operation steps of S2 may include S21-S22:

[0041] S21. Take the average value within a certain period of time in the past to count the traffic, and calculate the arrival rate of the business data based on the statistical traffic.

[0042] In a feasible implementation, in addition to the statistical method, the convolution method can also be used to calculate the arrival rate of the business data. The method for calculating the arrival rate can adopt the calculation method commonly used in the prior art, which will not be described in detail in the embodiment of the present invention.

[0043] S22. According to the channel information reported by the user within a certain period of time in the past, the channel information is statistically analyzed for distribution, and the channel information is fitted with the existing channel model, and the channel model with the smallest mean square error is selected as the user's channel statistical characteristics, wherein the channel statistical characteristics include channel fading, noise, interference, and fluctuations in signal quality.

[0044] S3. Calculate the service delay violation probability based on the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base.

[0045] Optionally, the specific operation steps of the S3 root can be as follows:

[0046] According to the arrival rate of service data, the statistical characteristics of the channel and the following formula (1), the service delay violation probability is calculated:

[0047]

[0048] in,

[0049] in,

[0050] in,

[0051] Among them, ε krepresents the delay violation probability, θ is a positive parameter of the random network calculus, N k Indicates the block length, represents the inverse function of the Gaussian Q function, ε c is the decoding error rate, represents the user's latency requirement, B k W k Indicates the user's bandwidth; B k represents the bandwidth of the resource block; W k represents the number of resource blocks; G represents the Mei jer G function; α represents the first channel parameter; Ω represents the second channel parameter; Γ(G) represents the gamma function of G; Γ(Z) represents the gamma function of Z; where, in order to make the expression of delay violation probability clearer, Z, U, Y and C are used to replace related variables; Z represents the first replacement related variable; U represents the second replacement related variable; Y represents the third replacement related variable; C represents the third replacement related variable; G G Represents the power form of the Mei jer G function.

[0052] S4. Pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user node.

[0053] Among them, the KP-DQN model is a knowledge-base parameterized deep Q-network (Knowledge-base ParameterizedDeep Q-Network).

[0054] Optionally, the specific operation steps of S4 may include S41-S45:

[0055] S41, reset the environment state;

[0056] S42, check the knowledge base: if the current state is recorded in the knowledge base, directly extract the knowledge from the knowledge base and jump to S44; if the current state is not recorded in the knowledge base, execute S43;

[0057] S43, calculating the service delay violation probability: calculating the service delay violation probability corresponding to the current state;

[0058] Execute actions: select and execute actions according to the strategy and obtain a new state;

[0059] Environmental status update: Update the environmental status according to the action results;

[0060] Storing experience: storing the current experience quadruple [state, action, reward, state at the next moment] into the experience pool;

[0061] S44, update network parameters: optimize the model network through the following steps:

[0062] Calculate the target Q value:

[0063]

[0064] Among them, y t represents the target Q value at the current moment, r t Indicates the instant reward at the current moment, represents the discount factor, a′ represents the action at the next moment, s t+1 Indicates the state at the next moment, ω - represents the parameters of the target Q network;

[0065] Actor network update: Calculate TD error according to the formula and update;

[0066]

[0067] Among them, L A represents the TD error of the Actor network, E represents the expectation, and ω represents the network parameter;

[0068] Critic network update: Calculate the TD error according to the formula and update it;

[0069]

[0070] Among them, L C Represents the TD error of the Critic network.

[0071] S45, determine whether the current number of steps has reached the preset number of steps. If not, add 1 to the current number of steps and jump to S41 for iterative training; if it has reached, stop the iteration and end the pre-training.

[0072] In a feasible implementation, the knowledge base is a library for storing the calculation results of the delay violation probability. Since the calculation process shown in formula (1) is very cumbersome, it needs to be calculated in each step of the KP-DQN algorithm learning, which will introduce a large calculation delay and greatly reduce the operation efficiency of the algorithm. Therefore, the delay violation probability is pre-calculated on the idle resources of the base station or node in advance, and the calculation parameters and the corresponding delay violation probability results are saved to form a knowledge base. When the algorithm is trained, it only needs to check whether there is a corresponding calculation result in the knowledge base, thereby avoiding repeated calculations and improving the operation efficiency of the algorithm.

[0073] S5. According to the pre-trained KP-DQN model and the status of the user node, the transmission mode, frame duration and number of data packet replications of the user node are selected, and then a decision is made on the reliability improvement problem to obtain a decision result.

[0074] Optionally, the specific operation steps of S5 may include S51-S54:

[0075] S51. Input the state of the user node into the pre-trained KP-DQN model. The state is represented by s = {s1, s2, s2, s4}, where s1 represents available resources, including idle bandwidth and available power at the current moment, s2 represents channel quality, s3 represents user demand, including latency requirements and reliability requirements, and s4 represents the number of constraint violations, which is used to quantify the negative impact of actions on rewards.

[0076] S52. Define the action space as a={a1,a2,a3,a4,a5}, where a1 represents the transmission mode, which includes bit transmission and semantic transmission, a2 represents the frame duration, a3 represents the number of resource blocks, a4 represents the number of data packet replications, and a5 represents the semantic compression rate.

[0077] In a feasible implementation, the action space includes mixed actions and continuous actions, the transmission mode of a1, the frame duration of a2, and the number of resource blocks of a3 are mixed actions, and the number of packet replications of a4 and the semantic compression rate of a5 are continuous actions. The KP-DQN algorithm divides the mixed actions into discrete actions and continuous parameters and models them separately, and jointly optimizes the Q values ​​of the discrete actions and continuous parameters. For discrete actions, the classic DQN algorithm is used to learn the Q value. For continuous actions, a parameterized network is used for optimization. The parameterized network is a neural network that is used to generate continuous parameters corresponding to discrete actions in the mixed action space to solve the problem of joint optimization of discrete and continuous actions. The parameterized network inputs are states and discrete actions, and the output is continuous parameters corresponding to discrete actions. The optimization goal is to maximize the generated Q value. The working method may include: (1) Learning the relationship between discrete actions and continuous parameters: For each discrete action, dynamically generate the optimal continuous parameters. (2) Optimizing continuous parameters through gradients: Backpropagating the parameterized network using the Q value to learn and generate better continuous parameters.

[0078] This separation greatly reduces the computational complexity, avoids direct search of the mixed action space, and improves efficiency and stability. The following are introduced separately:

[0079] Transmission mode of a1: The service can choose one of the two modes: bit transmission and semantic transmission. Semantic transmission compresses the data packet, which can save bandwidth and improve anti-interference ability under low signal-to-noise ratio conditions. However, semantic transmission introduces encoding and decoding delays, which is not friendly to users with extremely strict delay indicators. Therefore, the method proposed in the present invention can flexibly select the transmission mode of the service according to the user's delay requirements and channel conditions.

[0080] For semantic transmission, the semantic compression ratio determines the size of the data packet, which will affect the delay violation probability and the size of the semantic similarity. At the same time, the size of the semantic compression ratio determines the number of calculation cycles required for semantic extraction, and will also affect the encoding and decoding delay of the semantic part. Therefore, the semantic compression ratio will be flexibly selected in the present invention based on user needs and channel conditions. Specifically, the value range of the semantic compression ratio is between 0 and 1. The larger the value, the higher the compression ratio and the smaller the data packet.

[0081] a2 frame duration: The frame duration can be selected from three different frame durations: 1 millisecond, 0.5 milliseconds and 0.25 milliseconds.

[0082] The number of resource blocks of a3: allocates bandwidth resources to the user.

[0083] a4: Packet replication times: Packet replication can choose to replicate the transmitted data 0 to 2 times. Packet replication 0 times means no replication. Packet replication transmission can significantly improve the reliability of wireless transmission. The data packet can flexibly choose not to replicate and the number of replications. When the data packet chooses to replicate transmission, the reliability is the reliability of multiple links multiplied together. However, packet replication actually sacrifices system resources in exchange for improved reliability performance. Specifically, since the present invention adopts a flexible frame duration, when a short frame duration is selected, the frequency domain occupancy is relatively large, so packet replication can be completed in the time domain. For large frame durations, data version replication can be completed in the frequency domain.

[0084] a5: The number of times the data packet is replicated affects the accuracy of semantic transmission and recovery, thus affecting the reliability index. For traditional users, a3=0.

[0085] S53, setting reward function: For the bit transmission mode, set the reward function according to the following formula (2):

[0086] Ψ b =1-(1-ε k )(1-ε c )≈ε k +ε c (2)

[0087] Among them, ε k represents the delay violation probability calculated by formula (1), ε c Indicates the decoding error rate.

[0088] For the semantic transfer mode, the reward function is set according to the following formula (3):

[0089]

[0090] Among them, Φ sRepresents the semantic similarity of semantic transmission, which is related to the semantic compression ratio, Φ max Indicates the maximum value of semantic similarity.

[0091] In a feasible implementation, for a bit user, reliability is jointly characterized by the delay violation probability and the decoding error rate. The delay violation probability is calculated by random network calculation, and the decoding error rate is considered to be a constant in the present invention. Since the magnitude of the two numbers is very small, the reliability is approximately the sum of the two numbers.

[0092] For semantic users, reliability is measured by two indicators: delay violation probability and semantic similarity. Semantic similarity is a function of semantic compression ratio and can be obtained through simulation fitting. Considering that the delay violation probability and semantic similarity are not in the same order of magnitude, the semantic similarity is normalized first and then expressed uniformly.

[0093] If the constraints cannot be met, the reward value is 0.

[0094] S54, according to the state of the user node, calculate the next hybrid action, the hybrid action includes a discrete action and a continuous action, the discrete action includes a transmission mode, a frame duration, a number of resource blocks, and a number of data packet replications; the continuous action includes a semantic compression rate and power;

[0095] S55. Calculate the reward function according to the selected mixed action, put the state transition group into the experience replay pool, and obtain the decision result.

[0096] In one feasible implementation, after calculating the reward function based on the selected hybrid action, the state transition group (s, a, r, s') is put into the experience replay pool for fine-tuning and parameter updating of KP-DQN.

[0097] Optionally, the decision result includes a transmission mode, a frame duration, a number of data packet replications, and a resource allocation scheme of power and bandwidth.

[0098] S6. According to the decision result, complete the data packet transmission of the user node.

[0099] In a feasible implementation, in the decision result obtained, the transmission mode is bit transmission or semantic transmission, the frame duration is 1 millisecond, 0.5 milliseconds or 0.25 milliseconds, and the number of data packet replications is a specific number. Based on these data, the user node transmits data according to the resource allocation plan of power and bandwidth.

[0100] It should be noted that in actual use, the predicted data can be used to continuously adjust and optimize KP-DQN. The methods also include:

[0101] Fine-tune the KP-DQN algorithm model online based on the learning records, and update the parameters of the adjusted neural network.

[0102] A feasible implementation method, this continuous optimization can further enhance the superiority of the model, making the selected transmission scheme more reliable and better.

[0103] In an embodiment of the present invention, the parameters of the user node are obtained, and the algorithm parameters of the KP-DQN model are initialized; according to the parameters of the user node, the arrival rate of the service data is calculated, and the statistical characteristics of the channel where the user node is located are analyzed; according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, the probability of service delay violation is calculated and stored in the knowledge base; according to the idle resources of the user node and the knowledge base, the initial KP-DQN model is pre-trained; according to the pre-trained KP-DQN model and the state of the user node, the transmission mode, frame duration size and number of data packet replications of the user node are selected, and then the reliability improvement problem is decided to obtain a decision result; according to the decision result, the data packet transmission of the user node is completed. The present invention is adopted to improve the reliability of system communication by converting information into semantic transmission and combining flexible frame duration and data replication transmission. In an embodiment of the present invention, semantic communication or direct communication can be selected, different frame durations can be selected, and different data replication transmission times can be selected. The selection rule is based on the maximum quality of service. Through artificial intelligence, the optimal choice of transmission mode, frame duration and number of replications under different system conditions is calculated, thereby improving the reliability of data transmission.

[0104] Figure 2 1 is a block diagram of a device for improving the transmission reliability of a small data packet service according to an exemplary embodiment, wherein the device is used for improving the transmission reliability of a small data packet service. Figure 2 , the device comprises:

[0105] Initialization module 210, used to obtain parameters of user nodes and initialize algorithm parameters of the KP-DQN model;

[0106] The analysis module 220 is used to calculate the arrival rate of the service data according to the parameters of the user node and analyze the statistical characteristics of the channel where the user node is located;

[0107] A calculation module 230, used to calculate the service delay violation probability according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base;

[0108] A training module 240, configured to pre-train an initial KP-DQN model based on idle resources of the user node and the knowledge base;

[0109] The decision module 250 is used to select the transmission mode, frame duration and number of data packet replication of the user node according to the pre-trained KP-DQN model and the state of the user node, and then make a decision on the reliability improvement problem to obtain a decision result;

[0110] The transmission module 260 is used to complete the data packet transmission of the user node according to the decision result.

[0111] Initialization module, used to obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model;

[0112] An analysis module, used to calculate the arrival rate of service data according to the parameters of the user node, and analyze the statistical characteristics of the channel where the user node is located;

[0113] A calculation module, used to calculate the service delay violation probability according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base;

[0114] The training module is used to pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user node;

[0115] The decision module is used to select the transmission mode, frame duration, and number of data packet replications of the user node according to the pre-trained KP-DQN model and the status of the user node, and then make a decision on the reliability improvement problem and obtain the decision result;

[0116] The transmission module is used to complete the data packet transmission of the user node according to the decision result.

[0117] In an embodiment of the present invention, the parameters of the user node are obtained, and the algorithm parameters of the KP-DQN model are initialized; according to the parameters of the user node, the arrival rate of the service data is calculated, and the statistical characteristics of the channel where the user node is located are analyzed; according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, the probability of service delay violation is calculated and stored in the knowledge base; according to the idle resources of the user node and the knowledge base, the initial KP-DQN model is pre-trained; according to the pre-trained KP-DQN model and the state of the user node, the transmission mode, frame duration size and number of data packet replications of the user node are selected, and then the reliability improvement problem is decided to obtain a decision result; according to the decision result, the data packet transmission of the user node is completed. The present invention is adopted to improve the reliability of system communication by converting information into semantic transmission and combining flexible frame duration and data replication transmission. In an embodiment of the present invention, semantic communication or direct communication can be selected, different frame durations can be selected, and different data replication transmission times can be selected. The selection rule is based on the maximum quality of service. Through artificial intelligence, the optimal choice of transmission mode, frame duration and number of replications under different system conditions is calculated, thereby improving the reliability of data transmission.

[0118] Figure 3 is a schematic diagram of a structure of a device for improving the transmission reliability of small data packet services provided by an embodiment of the present invention, such as Figure 3 As shown, the device for improving the transmission reliability of small data packet services may include the above Figure 2 Optionally, the device 310 for improving the transmission reliability of a small data packet service may include a first processor 2001 .

[0119] Optionally, the device 310 for improving transmission reliability of small data packet services may further include a memory 2002 and a transceiver 2003 .

[0120] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0121] Combine the following Figure 3 The components of the device 310 for improving the transmission reliability of small data packet services are specifically introduced as follows:

[0122] The first processor 2001 is the control center of the device 310 for improving the transmission reliability of small data packet services, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0123] Optionally, the first processor 2001 may execute various functions of the device 310 for improving the transmission reliability of small data packet services by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0124] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0125] In a specific implementation, as an embodiment, the device 310 for improving the transmission reliability of small data packet services may also include multiple processors, such as Figure 3 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0126] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0127] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0128] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0129] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0130] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be implemented through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0131] It should be noted that Figure 3 The structure of the device 310 for improving the transmission reliability of small data packet services shown in the figure does not constitute a limitation on the router, and the actual knowledge structure identification device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0132] In addition, the technical effect of the device 310 for improving the transmission reliability of small data packet services can refer to the technical effect of the method for improving the transmission reliability of small data packet services described in the above method embodiment, which will not be repeated here.

[0133] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0134] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0135] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0136] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0137] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0138] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for improving the transmission reliability of small data packet services, characterized in that: The method comprises: S1. Obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model; S2. Calculate the arrival rate of service data based on the parameters of the user node and analyze the statistical characteristics of the channel where the user node is located; S3. Calculate the service delay violation probability based on the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base; S4. Pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user node; S5. According to the pre-trained KP-DQN model and the status of the user node, the transmission mode, frame duration, and number of data packet replications of the user node are selected, and then a decision is made on the reliability improvement problem to obtain a decision result; S6. According to the decision result, complete the data packet transmission of the user node.

2. The method for improving the transmission reliability of small data packet services according to claim 1, characterized in that: The parameters of the user node include delay requirement, reliability requirement, data packet size, bandwidth and power; The algorithm parameters of the KP-DQN model include Actor learning rate, Critic learning rate, discount factor and decay factor.

3. The method for improving the transmission reliability of small data packet services according to claim 1, characterized in that: The S2 calculates the arrival rate of the service data according to the parameters of the user node and analyzes the statistical characteristics of the channel where the user node is located, including: S21, taking the average value in the past certain period of time to count the traffic, and calculating the arrival rate of the business data based on the statistical traffic; S22. According to the channel information reported by the user within a certain period of time in the past, the channel information is statistically analyzed for distribution, and the channel information is fitted with the existing channel model, and the channel model with the smallest mean square error is selected as the user's channel statistical characteristics, wherein the channel statistical characteristics include channel fading, noise, interference, and fluctuations in signal quality.

4. The method for improving the transmission reliability of small data packet services according to claim 1, characterized in that: The step S3 calculates the service delay violation probability according to the arrival rate of the service data and the statistical characteristics of the channel, including: According to the arrival rate of service data, the statistical characteristics of the channel and the following formula (1), the service delay violation probability is calculated: in, in, in, Among them, ε k represents the delay violation probability, θ is a positive parameter of the random network calculus, N k Indicates the block length, represents the inverse function of the Gaussian Q function, ε c is the decoding error rate, represents the user's latency requirement, B k W k Indicates the user's bandwidth; B k represents the bandwidth of the resource block; W k represents the number of resource blocks; G represents the Meijer G function; α represents the first channel parameter; Ω represents the second channel parameter; Γ(G) represents the gamma function of G; Γ(Z) represents the gamma function of Z; where, in order to make the expression of delay violation probability clearer, Z, U, Y and C are used to replace related variables; Z represents the first replacement related variable; U represents the second replacement related variable; Y represents the third replacement related variable; C represents the third replacement related variable; G G Represents the power form of the Meijer G function.

5. The method for improving the transmission reliability of small data packet services according to claim 1, characterized in that: The S4 pre-trains the initial KP-DQN model according to the idle resources and knowledge base of the user node, including: S41, reset the environment state; S42, check the knowledge base: if the current state is recorded in the knowledge base, directly extract the knowledge from the knowledge base and jump to S44; if the current state is not recorded in the knowledge base, execute S43; S43, calculating the service delay violation probability: calculating the service delay violation probability corresponding to the current state; Execute actions: select and execute actions according to the strategy and obtain a new state; Environmental status update: Update the environmental status according to the action results; Storing experience: storing the current experience quadruple [state, action, reward, state at the next moment] into the experience pool; S44, update network parameters: optimize the model network through the following steps: Calculate the target Q value: Among them, y t represents the target Q value at the current moment, r t Indicates the instant reward at the current moment, represents the discount factor, a′ represents the action at the next moment, s t+1 Indicates the state at the next moment, ω - represents the parameters of the target Q network; Actor network update: Calculate TD error according to the formula and update; L A =E[Q(s,a;ω)-Q(s,a;ω - )] Among them, L A represents the TD error of the Actor network, E represents the expectation, and ω represents the network parameter; Critic network update: Calculate the TD error according to the formula and update it; Among them, L C Represents the TD error of the Critic network; S45, determine whether the current number of steps has reached the preset number of steps. If not, add 1 to the current number of steps and jump to S41 for iterative training; if it has reached, stop the iteration and end the pre-training.

6. The method for improving the transmission reliability of small data packet services according to claim 1, characterized in that: The S5 selects the transmission mode, frame duration, and number of data packet replications of the user node according to the pre-trained KP-DQN model and the state of the user node, and then makes a decision on the reliability improvement problem to obtain a decision result, including: S51, input the state of the user node into the pre-trained KP-DQN model, the state is represented by s={s1,s2,s2,s4}, where s1 represents available resources, including idle bandwidth and available power at the current moment, s2 represents channel quality, s3 represents user requirements, including latency requirements and reliability requirements, and s4 represents the number of constraint violations, which is used to quantify the negative impact of the action on the reward; S52, define the action space as a={a1,a2,a3,a4,a5}, where a1 represents the transmission mode, the transmission mode includes bit transmission and semantic transmission, a2 represents the frame duration, a3 represents the number of resource blocks, a4 represents the number of data packet replications, and a5 represents the semantic compression rate; S53, setting reward function: For the bit transmission mode, set the reward function according to the following formula (2): P b =1-(1-e k (1-e) c )≈e k +e c (2) Among them, ε k represents the delay violation probability calculated by formula (1), ε c Indicates the decoding error rate; For the semantic transfer mode, the reward function is set according to the following formula (3): Among them, Φ s Represents the semantic similarity of semantic transmission, which is related to the semantic compression ratio, Φ max Indicates the maximum value of semantic similarity; S54, according to the state of the user node, calculate the next hybrid action, the hybrid action includes a discrete action and a continuous action, the discrete action includes a transmission mode, a frame duration, a number of resource blocks, and a number of data packet replications; the continuous action includes a semantic compression rate and power; S55. Calculate the reward function according to the selected mixed action, and put the state transition group into the experience replay pool.

7. The method for improving the transmission reliability of small data packet services according to claim 1, characterized in that: The decision results include transmission mode, frame duration, number of data packet replications, and resource allocation schemes of power and bandwidth.

8. A device for improving the transmission reliability of a small data packet service, the device for improving the transmission reliability of a small data packet service is used to implement the method for improving the transmission reliability of a small data packet service as claimed in any one of claims 1 to 7, characterized in that: The device comprises: Initialization module, used to obtain the parameters of the user node and initialize the algorithm parameters of the KP-DQN model; An analysis module, used to calculate the arrival rate of service data according to the parameters of the user node, and analyze the statistical characteristics of the channel where the user node is located; A calculation module, used to calculate the service delay violation probability according to the parameters of the user node, the arrival rate of the service data and the statistical characteristics of the channel, and store it in the knowledge base; The training module is used to pre-train the initial KP-DQN model based on the idle resources and knowledge base of the user node; The decision module is used to select the transmission mode, frame duration, and number of data packet replications of the user node according to the pre-trained KP-DQN model and the status of the user node, and then make a decision on the reliability improvement problem and obtain the decision result; The transmission module is used to complete the data packet transmission of the user node according to the decision result.

9. A device for improving the transmission reliability of small data packet services, characterized in that: The device for improving the transmission reliability of small data packet services includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images