Data communication method, electronic device, storage medium and computer program product

Abstract

The invention provides a data communication method, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a spatio-temporal feature vector corresponding to communication environment data; a target deep learning reinforcement model is adopted to analyze and process the spatial-temporal feature vector to obtain a target frequency band allocation strategy, the target frequency band allocation strategy is used for allocating target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environment feedback data; and performing data transmission on the to-be-transmitted data according to the target frequency band allocation strategy in the multi-frequency-band cooperative communication process to obtain a target transmission result. According to the invention, the technical problems of low spectrum resource utilization efficiency, large inter-band interference and lack of adaptability of a communication strategy in a data communication method provided in the related technology are solved.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence, network and communication technology, and in particular to a data communication method, electronic equipment, storage medium and computer program product. Background Art

[0002] With the acceleration of the process of intelligence and digitalization, the demand for data communication has shown explosive growth, especially in cutting-edge fields such as the Internet of Things, 5G communications, and edge computing, where real-time transmission of massive data has become the norm. The data communication methods in related technologies usually use fixed spectrum allocation and preset communication protocols to achieve efficient data transmission. However, when facing high-density users and changing communication environments, the fixed spectrum allocation mechanism has low utilization efficiency of spectrum resources and cannot effectively cope with spectrum congestion and channel interference, resulting in increased communication delays and decreased data transmission rates; although the preset communication protocol performs well under ideal conditions, in actual application scenarios, due to the lack of sensitivity and responsiveness to environmental changes, it is unable to make rapid adjustments in the face of sudden network congestion, changes in device mobility, etc., affecting the continuity and service quality of data communication. Therefore, the data communication methods provided in related technologies are difficult to meet the needs of modern communication systems for high capacity, low latency, and flexible adaptability, and thus it is difficult to provide stable and efficient data communication services.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present disclosure provide a data communication method, an electronic device, a storage medium and a computer program product to at least solve the technical problems of low spectrum resource utilization efficiency, large interference between frequency bands and lack of adaptability of communication strategies existing in the data communication methods provided in the related arts.

[0005] According to one aspect of an embodiment of the present disclosure, there is provided a data communication method, comprising: acquiring a spatiotemporal feature vector corresponding to communication environment data;

[0006] The target deep learning reinforcement model is used to analyze and process the spatiotemporal feature vector to obtain the target frequency band allocation strategy, where the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated and processed through real-time environmental feedback data; in the process of multi-band collaborative communication, the data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result.

[0007] Optionally, obtaining the spatiotemporal feature vector corresponding to the communication environment data includes: obtaining the communication environment data, wherein the communication environment data includes at least: channel state data, received signal strength indication data, and spectrum usage data of neighboring base stations; performing a convolution operation on the communication environment data using a convolutional neural network to obtain a spatial feature vector corresponding to the communication environment data; performing feature extraction on the communication environment data using a long short-term memory network to obtain a time feature vector corresponding to the communication environment data; and determining the spatiotemporal feature vector based on the spatial feature vector and the time feature vector.

[0008] Optionally, the data communication method also includes: determining target state data based on the spatiotemporal feature vector, wherein the target state data is used to represent the current communication environment state; determining target action data using the target state data, wherein the target action data is used to represent the current frequency band allocation strategy corresponding to the current communication environment state; obtaining a real-time reward function and updated state data based on the target state data and the target action data; and storing the target state data, target action data, real-time reward function and updated state data in a historical interaction storage area.

[0009] Optionally, the data communication method also includes: obtaining historical experience samples from a historical interaction storage area; determining a target action value function and a target expected value based on the historical experience samples, wherein the target action value function is used to represent the expected reward for managing target state data and target action data; determining a first loss function based on the target action value function and the target expected value; and using the first loss function to iteratively train the initial deep learning reinforcement model to obtain a target deep learning reinforcement model.

[0010] Optionally, during the multi-band collaborative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result, including: determining multiple target transmission frequency bands using the target frequency band allocation strategy; dividing the data to be transmitted to obtain multiple data subsets; using multiple target transmission frequency bands to transmit multiple data subsets in parallel to obtain the target transmission result.

[0011] Optionally, the data communication method also includes: obtaining historical network load data; using a target load prediction network to analyze and predict the historical network load data and communication environment data to obtain a load prediction result, wherein the load prediction result is used to represent the network load in a future time period; determining bandwidth change data corresponding to the target transmission frequency band based on the load prediction result; and using the bandwidth change data to adjust the target transmission frequency band to obtain a bandwidth adjustment result corresponding to the target transmission frequency band.

[0012] Optionally, the data communication method also includes: using a target power prediction network to analyze and process communication environment data to obtain a target power allocation strategy, wherein the target power allocation strategy is used to determine the power allocation results in a future time period; determining power change data based on the target power allocation strategy; and using the power change data to adjust the target transmission frequency band to obtain a power adjustment result corresponding to the target transmission frequency band.

[0013] According to another aspect of an embodiment of the present disclosure, a data communication device is also provided, including: a first acquisition module, used to acquire the spatiotemporal feature vector corresponding to the communication environment data; a first processing module, used to analyze and process the spatiotemporal feature vector using a target deep learning reinforcement model to obtain a target frequency band allocation strategy, wherein the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated and processed through real-time environmental feedback data; a transmission module, used to transmit the data to be transmitted according to the target frequency band allocation strategy during multi-band collaborative communication to obtain a target transmission result.

[0014] Optionally, the first acquisition module is also used to: acquire communication environment data, wherein the communication environment data includes at least: channel state data, received signal strength indication data, and spectrum usage data of neighboring base stations; perform convolution operations on the communication environment data using a convolutional neural network to obtain a spatial feature vector corresponding to the communication environment data; perform feature extraction on the communication environment data using a long short-term memory network to obtain a time feature vector corresponding to the communication environment data; and determine the spatiotemporal feature vector based on the spatial feature vector and the time feature vector.

[0015] Optionally, the data communication device also includes: a first determination module, used to determine target state data based on the spatiotemporal feature vector, wherein the target state data is used to represent the current communication environment state; a second determination module, used to determine target action data using the target state data, wherein the target action data is used to represent the current frequency band allocation strategy corresponding to the current communication environment state; a second acquisition module, used to acquire a real-time reward function and updated state data based on the target state data and the target action data; and a storage module, used to store the target state data, target action data, real-time reward function and updated state data in a historical interaction storage area.

[0016] Optionally, the data communication device also includes: a third acquisition module, used to acquire historical experience samples from the historical interaction storage area; a third determination module, used to determine the target action value function and the target expected value based on the historical experience samples, wherein the target action value function is used to represent the expected reward for the management of the target state data and the target action data; a fourth determination module, used to determine the first loss function based on the target action value function and the target expected value; and a training module, used to iteratively train the initial deep learning reinforcement model using the first loss function to obtain a target deep learning reinforcement model.

[0017] Optionally, the transmission module is also used to: determine multiple target transmission frequency bands using a target frequency band allocation strategy; divide the data to be transmitted to obtain multiple data subsets; use multiple target transmission frequency bands to transmit multiple data subsets in parallel to obtain a target transmission result.

[0018] Optionally, the transmission module is also used to: obtain historical network load data; use the target load prediction network to analyze and predict the historical network load data and communication environment data to obtain a load prediction result, wherein the load prediction result is used to represent the network load in a future time period; determine the bandwidth change data corresponding to the target transmission frequency band based on the load prediction result; use the bandwidth change data to adjust the target transmission frequency band to obtain a bandwidth adjustment result corresponding to the target transmission frequency band.

[0019] Optionally, the transmission module is also used to: use the target power prediction network to analyze and process the communication environment data to obtain a target power allocation strategy, wherein the target power allocation strategy is used to determine the power allocation results in a future time period; determine power change data based on the target power allocation strategy; use the power change data to adjust the target transmission frequency band to obtain the power adjustment result corresponding to the target transmission frequency band.

[0020] According to another aspect of an embodiment of the present disclosure, an electronic device is further provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the data communication method in the embodiment of the present disclosure.

[0021] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program runs, the device where the storage medium is located is controlled to execute the data communication method in the embodiment of the present disclosure.

[0022] According to another aspect of the embodiments of the present disclosure, a computer program product is further provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the data communication method in the embodiments of the present disclosure is implemented.

[0023] In the disclosed embodiment, by acquiring the spatiotemporal feature vector corresponding to the communication environment data, and then using the target deep learning reinforcement model to analyze and process the spatiotemporal feature vector, a target frequency band allocation strategy is obtained, and finally in the multi-band collaborative communication process, the data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result, thereby achieving the purpose of dynamically optimizing frequency band selection and resource allocation, thereby achieving the technical effect of improving the efficiency of spectrum resource utilization, reducing interference between frequency bands and enhancing the adaptability of communication strategies, and thus solving the technical problems of low spectrum resource utilization efficiency, large interference between frequency bands and lack of adaptability of communication strategies existing in the data communication methods provided in the related arts. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0025] Figure 1 is a flow chart of a data communication method according to one embodiment of the present disclosure;

[0026] Figure 2 is a schematic diagram of a data communication system according to one embodiment of the present disclosure;

[0027] Figure 3 is a schematic diagram of a data communication method according to one embodiment of the present disclosure;

[0028] Figure 4 is a schematic diagram of another data communication method according to one embodiment of the present disclosure;

[0029] Figure 5 is a schematic diagram of another data communication method according to one embodiment of the present disclosure;

[0030] Figure 6 is a schematic diagram of another data communication method according to one embodiment of the present disclosure;

[0031] Figure 7 It is a structural block diagram of a data communication device according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0034] The data communication methods in the related art usually use fixed spectrum allocation and preset communication protocols to achieve efficient data transmission. However, when faced with high-density users and changing communication environments, the fixed spectrum allocation mechanism has low efficiency in spectrum resource utilization and cannot effectively deal with spectrum congestion and channel interference, resulting in increased communication delays and decreased data transmission rates. Although the preset communication protocol performs well under ideal conditions, in actual application scenarios, due to the lack of sensitivity and responsiveness to environmental changes, it is unable to make rapid adjustments in the face of sudden network congestion, changes in device mobility, and other conditions, affecting the continuity and service quality of data communications. Therefore, the data communication methods provided in the related art are difficult to meet the needs of modern communication systems for high capacity, low latency, and flexible adaptability, and thus it is difficult to provide stable and efficient data communication services.

[0035] Specifically, first of all, in high-density user scenarios, fixed spectrum allocation mechanisms are difficult to flexibly respond to dynamic demand for spectrum resources. When a large number of users access the network at the same time, some frequency bands may quickly reach saturation, while other frequency bands may have idle resources. In the above case, the lack of intelligent dynamic adjustment capabilities of the spectrum allocation mechanism will lead to uneven allocation of spectrum resources and inability to achieve efficient collaboration between frequency bands, thereby reducing the overall data transmission rate, increasing communication delays, and reducing user experience.

[0036] Secondly, the design of preset communication protocols is often based on stable and preset rules and conditions. Once environmental conditions fluctuate rapidly, such as sudden network congestion or sudden changes in device location, it is difficult to quickly adjust key communication parameters, such as coding rate and modulation method. This inflexible response mechanism will lead to an increase in errors during data transmission and frequent data packet retransmissions, which in turn destroys the continuity of communication and reduces the quality of service. Especially in application scenarios that have extremely high requirements for communication stability and efficiency, such as telemedicine, autonomous driving, and real-time video conferencing, the limitations of preset communication protocols limit the widespread application and deep optimization of communication technology in the above fields.

[0037] Furthermore, the combination of fixed spectrum allocation and preset communication protocols cannot provide sufficient bandwidth and fast response when facing application scenarios that require real-time transmission of large amounts of data, such as high-definition video streaming, online games, and real-time industrial control. The queuing waiting time for data packets is too long, affecting the real-time nature of communication and user experience.

[0038] Finally, as communication technology expands to higher frequency bands, such as millimeter wave and terahertz bands, the above frequency bands include higher propagation loss and stronger directivity. The data communication methods in related technologies often do not fully consider the above characteristics when designing, resulting in that in practical applications, the communication system cannot fully utilize high-frequency resources, the spectrum utilization rate and communication efficiency are low, and it cannot meet the future communication technology requirements for high capacity and flexible adaptability.

[0039] According to an embodiment of the present disclosure, a method embodiment of a data communication method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] The method embodiment can be executed in an electronic device or a similar computing device including a memory and a processor. Taking running on a computer terminal as an example, the computer terminal may include one or more processors (processors may include but are not limited to central processing units (CPU), graphics processing units (GPU), digital signal processing (DSP) chips, microprocessors (MCU), programmable logic devices (Field Programmable Gate Array, FPGA), neural network processors (Neural-network Processor Unit, NPU), tensor processors (TensorProcessing Unit, TPU), artificial intelligence (Artificial Intelligence, AI) type processors and other processing devices) and a memory for storing data. Optionally, the above-mentioned computer terminal may also include a transmission device, an input and output device, and a display device for communication functions. It can be understood by those of ordinary skill in the art that the above-mentioned structural description is only for illustration, and it does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include more or fewer components than the above-mentioned structural description, or have a configuration different from the above-mentioned structural description.

[0041] The memory can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data communication method in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the computer program stored in the memory, that is, realizing the above-mentioned data communication method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0042] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0043] The display device may be, for example, a touch screen type liquid crystal display (LCD) and a touch display (also referred to as a "touch screen" or "touch display screen"). The liquid crystal display may enable a user to interact with a user interface of a mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), and a user may interact with the GUI by finger contacts and / or gestures on a touch-sensitive surface, wherein the human-computer interaction functions here may optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for executing the above human-computer interaction functions are configured / stored in a computer program product or readable storage medium executable by one or more processors.

[0044] Figure 1 is a flow chart of a data communication method according to one embodiment of the present disclosure, such as Figure 1 As shown, the method comprises the following steps:

[0045] Step S11, obtaining a spatiotemporal feature vector corresponding to the communication environment data;

[0046] Step S12, using the target deep learning reinforcement model to analyze and process the spatiotemporal feature vector to obtain a target frequency band allocation strategy, wherein the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environmental feedback data;

[0047] Step S13, in the multi-band cooperative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain a target transmission result.

[0048] The above communication environment data refers to a collection of information collected within a specific time and space range in a communication scenario that can reflect the current communication environment status and characteristics. Specifically, communication environment data includes but is not limited to:

[0049] 1) Channel State Information (CSI): The transmission characteristics of the wireless channel, such as signal attenuation, multipath effect, and channel quality, are used to evaluate signal transmission quality and predict future channel status.

[0050] 2) Received Signal Strength Indication (RSSI): The signal strength detected by the receiving end helps to identify communication quality, locate devices and evaluate potential interference sources.

[0051] 3) Spectrum usage data of neighboring base stations: including the frequency bands being used by neighboring base stations, the occupancy status of each frequency band, channel bandwidth allocation, power level, and possible interference level, etc.

[0052] 4) User device location information: This information is obtained using GPS, base station positioning, or Wi-Fi signals to achieve location-aware communication optimization, such as directional communication or avoiding local interference.

[0053] 5) Network traffic status: including the number, size, type and direction of data packets, which helps predict network congestion and optimize frequency band bandwidth allocation.

[0054] 6) Channel interference level: describes the interference intensity of the same frequency and adjacent frequency, as well as the interference of non-communication signal sources (such as electromagnetic noise), which is used to design interference suppression strategies.

[0055] 7) User behavior and needs: including the activity patterns of user devices, data transmission requirements (such as upload and download rates), and Quality of Service (QoS) requirements, which are used to dynamically adjust communication strategies to meet user expectations.

[0056] 8) Time-varying environmental parameters: such as temperature, humidity, electromagnetic environment, etc. These parameters may affect the performance of wireless communication. By monitoring these parameters, the communication system can be appropriately adjusted.

[0057] 9) System performance indicators: including packet loss rate, latency, throughput, etc., reflecting the actual performance of the communication system in the current environment.

[0058] 10) Historical communication data: used for pattern recognition and prediction, helping the system understand long-term communication behavior and make more accurate decisions.

[0059] The above-mentioned spatiotemporal feature vector refers to a data representation that is generated after deep learning processing of the communication environment data of the communication environment and can comprehensively reflect the spatial distribution characteristics and time series characteristics of the communication environment.

[0060] The above-mentioned target deep learning reinforcement model includes but is not limited to the Deep Q-Network (DQN) and policy-based deep reinforcement learning models, such as the Actor-Critic model. Among them, DQN is a combination of deep learning and Q learning, using neural networks to approximate the Q-value function, that is, the expected future reward given a state and action. By continuously interacting with the environment, DQN can learn which frequency band allocation strategy can produce the largest long-term cumulative reward under different target state data, thereby achieving adaptive frequency band selection. Actor-Critic combines two components: Actor (policy network, determining action) and Critic (value network, evaluating action). In the communication system, the Actor network can select the optimal frequency band allocation strategy based on the current communication environment state, while the Critic network evaluates the effect of this strategy and gives reward feedback. Through the collaborative work of Actor and Critic, the system can more efficiently optimize the frequency band selection and parameter adjustment strategy to adapt to the dynamic changes of the communication environment and improve communication performance.

[0061] Exemplarily, by deploying sensors, monitoring equipment or user equipment in the communication network, the communication environment data is continuously collected, and the deep learning model is used to automatically extract the spatiotemporal features in the communication environment data to generate a feature vector that can comprehensively describe the state of the communication environment. Furthermore, the spatiotemporal feature vector obtained above is input into the target deep learning reinforcement model for analysis and processing to obtain the target frequency band allocation strategy. Finally, based on the above target frequency band allocation strategy, in multi-band collaborative communication, the data to be transmitted is intelligently allocated to different frequency bands for transmission, and the transmission parameters, such as power and bandwidth, are dynamically adjusted to adapt to the current communication environment.

[0062] Based on the above steps S11 to S13, by acquiring the spatiotemporal feature vector corresponding to the communication environment data, and then using the target deep learning reinforcement model to analyze and process the spatiotemporal feature vector, the target frequency band allocation strategy is obtained, and finally in the multi-band collaborative communication process, the data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result, thereby achieving the purpose of dynamically optimizing frequency band selection and resource allocation, thereby achieving the technical effect of improving the efficiency of spectrum resource utilization, reducing interference between frequency bands and enhancing the adaptability of communication strategies, and thus solving the technical problems of low spectrum resource utilization efficiency, large interference between frequency bands and lack of adaptability of communication strategies existing in the data communication methods provided in the relevant technologies.

[0063] Figure 2 is a schematic diagram of a data communication system according to one embodiment of the present disclosure, such as Figure 2As shown, the system includes the following components: an input unit, a communication environment feature extraction unit, an adaptive frequency band selection unit, a multi-frequency band cooperative communication unit, a dynamic parameter optimization unit and an output unit.

[0064] Optionally, in step S11, obtaining the spatiotemporal feature vector corresponding to the communication environment data includes:

[0065] Step S111, acquiring communication environment data, wherein the communication environment data at least includes: channel state data, received signal strength indication data, and spectrum usage data of neighboring base stations;

[0066] Step S112, performing a convolution operation on the communication environment data using a convolutional neural network to obtain a spatial feature vector corresponding to the communication environment data;

[0067] Step S113, using a long short-term memory network to extract features from the communication environment data to obtain a time feature vector corresponding to the communication environment data;

[0068] Step S114, determining a spatiotemporal feature vector based on the spatial feature vector and the temporal feature vector.

[0069] The above-mentioned convolutional neural network (CNN) refers to a deep learning model used to process data with a two-dimensional structure, such as images and signal matrices, and can perform feature detection in local areas while reducing the number of model parameters and improving computational efficiency. In the disclosed embodiment, CNN is used to extract and analyze spatial features in communication environment data.

[0070] The above-mentioned Long Short-Term Memory (LSTM) refers to a special recurrent neural network (RNN) that is used to overcome the gradient vanishing or gradient exploding problems of traditional RNN when processing long sequence data, and can learn long-term dependencies in the data. Since the communication environment has strong time series characteristics, LSTM can capture the time evolution trend of factors such as channel state and interference, and when learning time series characteristics, it can retain key information over a period of time in the past, helping the system predict the changing trend of the channel. In the disclosed embodiment, LSTM is used to extract time series features from communication environment data.

[0071] Specifically, LSTM is divided into several parts: forget gate, input gate, memory unit state, and output gate:

[0072] 1) Forget Gate: It is used to determine how much of the memory cell state at the previous moment should be retained at the current moment. The output of the forget gate is generated by the Sigmoid function and ranges between 0 and 1, where 0 means complete forgetfulness and 1 means complete retention.

[0073] 2) Input Gate: It is used to control the writing of new information into the memory cell state. The output of the input gate is also generated by the Sigmoid function, while the new information is generated by the tanh function. The output of the input gate is multiplied by the value of the new information and then added to the memory cell state.

[0074] 3) Cell State: The cell state of LSTM is the core of its memory mechanism. It stores important information about the sequence history through the control of the forget gate and input gate.

[0075] 4) Output Gate: It is used to determine how much information in the memory cell state should be output as part of the current hidden state. The hidden state encapsulates the information accumulated during sequence processing. The output of the output gate is generated by the Sigmoid function, and the memory cell state is transformed by the tanh function. The output of the output gate is multiplied by the transformed memory cell state to obtain the hidden state output at the current moment.

[0076] Exemplarily, CNN is used to perform convolution operation on the two-dimensional features in the communication environment data to extract the spatial feature vector in the communication environment data. The specific calculation process is shown in formula (1):

[0077] F l =ReLU(conv(W l ,X t )+b l ) Formula (1)

[0078] Among them, X t is the input feature matrix, representing the communication environment data collected at time t, X t ={x1,x2,…,x t}; F l is the feature map output after the l-th layer of convolution; W l is the weight of the convolution kernel of the lth layer; b l is the bias term; conv(·) represents the convolution operation.

[0079] Furthermore, maximum pooling is used to reduce the number of parameters and extract important features. Then, according to the output P after the convolution and pooling operations, P is flattened to obtain the vector p, and further feature integration and classification are performed through the fully connected layer, and finally the vector representation related to the spatial features is output.

[0080] Exemplarily, LSTM is used to extract features from the communication environment data to obtain a time feature vector corresponding to the communication environment data. The specific calculation process is shown in formulas (2) to (7):

[0081] f t =σ(W f ·[h t-1 ,X t ]+b f ) Formula (2)

[0082] i t =σ(W i ·[h t-1 ,X t ]+b i ) Formula (3)

[0083]

[0084] o t =σ(W o ·[h t-1 ,X t ]+b o ) Formula (6)

[0085] h t =o t *tanh(C t ) Formula (7)

[0086] Among them, f t is the output of the forget gate, controlling the memory C of the previous moment t-1 Whether to keep; σ is the Sigmoid activation function; h t-1 is the hidden state of the previous moment; i t It is the output of the input gate, which controls the writing of new information; is the new memory content at the current moment; C t is the current memory state; o t is the output of the output gate, controlling the hidden state h of the output t ; * indicates element-by-element multiplication; W f , W i , W C , W o Represent the weight matrices of the forget gate, input gate, output gate and memory content update respectively.

[0087] Furthermore, after CNN and LSTM processing, the spatial and temporal features of the communication environment data are integrated into a feature vector. The final generated spatiotemporal feature vector is shown in formula (8):

[0088] F=w1·fspace +w2·f time Formula (8)

[0089] Among them, f space is the spatial feature extracted by CNN; f time is the time series feature extracted by LSTM; w1 and w2 are weights, which can be adjusted according to the performance during model training; F is the final generated spatiotemporal feature vector.

[0090] Based on the above optional embodiments, by combining CNN and LSTM, key spatial and temporal features can be automatically learned and extracted from communication environment data without the need for manually pre-designing feature extraction algorithms, thereby improving the adaptability and intelligence level of the communication system.

[0091] Figure 3 is a schematic diagram of a data communication method according to one embodiment of the present disclosure, such as Figure 3 As shown in the figure, in the process of data communication, it is first necessary to obtain multi-source data in the communication environment, such as channel state information, received signal strength indication and other communication data. Secondly, a convolutional neural network is used to extract the spatial features of the above communication data. Furthermore, a long short-term memory network is used to extract the temporal features of the above communication data. Finally, the spatial features and temporal features of the above communication data are combined to obtain a spatiotemporal feature vector.

[0092] Optionally, the data communication method further includes:

[0093] Step S21, determining target state data based on the spatiotemporal feature vector, wherein the target state data is used to represent the current communication environment state;

[0094] Step S22, determining target action data using the target state data, wherein the target action data is used to represent a current frequency band allocation strategy corresponding to a current communication environment state;

[0095] Step S23, obtaining a real-time reward function and updating state data based on the target state data and the target action data;

[0096] Step S24, storing the target state data, target action data, real-time reward function and updated state data in the historical interaction storage area.

[0097] The above-mentioned target state data refers to a quantitative description of the current communication environment state, which is determined based on the spatiotemporal feature vector and includes all environmental information that the system needs to consider, such as channel status, signal strength, spectrum usage of neighboring base stations, etc.

[0098] For example, the state s at time t t It can be expressed as s t=[channel status, interference intensity, user density, …].

[0099] The above-mentioned target action data refers to the current frequency band allocation strategy determined according to the target state data, that is, the "action" to be performed. In the reinforcement learning framework, an action generally refers to measures that an agent can take to influence the state of the environment, and in the disclosed embodiment, the target action data specifically refers to which frequency band or bands to select for data transmission, and how to allocate the resources of the frequency band (such as bandwidth, power, etc.) to optimize communication performance and spectrum efficiency.

[0100] For example, for a communication system, each action a t It is to select one or more frequency bands from multiple candidate frequency bands for transmission, represented by a t ∈A={band 1, band 2,…, band N}.

[0101] The above real-time reward function refers to an important indicator for evaluating the effect of frequency band selection actions, reflecting multiple goals such as spectrum efficiency, interference suppression, and system capacity. Assume that the system takes action a at time t t The result is an increase in spectrum utilization and a decrease in interference, and its reward value is expressed as r t =f(spectral efficiency, interference reduction, capacity improvement), where f is a custom function that can be used to determine how to comprehensively evaluate the above factors based on specific communication system requirements and performance indicators.

[0102] The above-mentioned updated state data refers to the new state data after the state of the environment changes after the intelligent agent (such as a communication system) interacts with the environment according to its selected actions (such as a frequency band allocation strategy) in the deep reinforcement learning framework.

[0103] For example, it is assumed that the deep learning reinforcement model targeted by this application is the DQN algorithm, the core of which is to update the Q value through continuous learning. The Q value function Q(s t ,a t ) means in state s t Next select action a t The expected reward after . The update process is shown in formula (9):

[0104]

[0105] Among them, α is the learning rate, which determines the degree of absorption of new information each time the Q value is updated; γ is the discount factor, which reflects the importance of future rewards to the current decision; r t is the immediate reward, which is the system performing action a t Reward feedback immediately obtained after the event; Indicates the maximum Q value of the next state. Through iterative updates of the Q value, the system can gradually optimize the frequency band selection strategy and adapt to the dynamically changing communication environment.

[0106] The above-mentioned historical interaction storage area refers to the "experience replay pool" used in the deep reinforcement learning framework, which is used to store historical data of the interaction between the intelligent agent and the environment, which can avoid data correlation problems and improve learning efficiency.

[0107] For example, during the learning process, the communication system continuously interacts with the communication environment. First, at each time t, the current communication environment state s is observed. t , select actions according to the greedy strategy ∈, so that the system can not only explore new frequency band selection possibilities during the learning process, but also use the learned knowledge to select the optimal frequency band. Further, perform action a t , get instant rewards t and the new state s t+1 Finally, (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool.

[0108] Based on the above optional embodiment, the target state data, target action data, real-time reward function and updated state data are stored in the historical interaction storage area, and samples can be randomly selected during the training process, thereby breaking the time correlation between the data, avoiding overfitting problems in model training, and improving the generalization ability of the learning algorithm.

[0109] Figure 4 is a schematic diagram of another data communication method according to one embodiment of the present disclosure, such as Figure 4 As shown in the figure, the core workflow of the adaptive frequency band selection unit is presented. First, based on the state space, action space and reward function, the current state is observed, and an action is selected according to the greedy strategy, that is, a suitable frequency band allocation strategy is selected. Furthermore, the above actions are executed to obtain rewards and new states, and the experience is stored in the experience replay pool. Finally, samples are extracted from the replay pool and parameters are updated. By repeating the above actions, the parameter updates are optimized, thereby adaptively selecting frequency bands.

[0110] Optionally, the data communication method further includes:

[0111] Step S31, obtaining historical experience samples from the historical interaction storage area;

[0112] Step S32, determining a target action value function and a target expected value based on historical experience samples, wherein the target action value function is used to represent the expected reward of target state data and target action data management;

[0113] Step S33, determining a first loss function based on the target action value function and the target expected value;

[0114] Step S34, using the first loss function to iteratively train the initial deep learning reinforcement model to obtain the target deep learning reinforcement model.

[0115] For example, a small batch of experience samples, i.e., historical experience samples, are randomly selected from the experience replay pool, and their target action value function y can be calculated by formula (10): t :

[0116]

[0117] Furthermore, the first loss function of the historical experience sample is shown in formula (11):

[0118]

[0119] in, is the target expected value; θ is the parameter of the DQN network, and the gradient descent method or its variants (such as the Adam algorithm) can be used to update the parameter θ of the DQN network.

[0120] For example, at a certain number of steps, the parameters θ of the DQN network are copied to the parameters θ′ of the target network. The target network is a network with the same structure as the DQN network but with slower parameter updates. Its function is to stabilize the learning process and avoid excessive fluctuations in the target value. By continuously repeating the above process, the system can gradually optimize the frequency band selection strategy and adapt to the dynamically changing communication environment.

[0121] Based on the above optional embodiments, by continuously extracting historical experience samples from the experience replay pool, calculating the target action value function, determining the loss function and iteratively training the deep learning reinforcement model, the parameters of the model can be gradually optimized so that the model can learn to make efficient and robust frequency band selection and resource allocation decisions in a complex and changeable communication environment.

[0122] Optionally, in step S13, in the multi-band cooperative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy, and obtaining the target transmission result includes:

[0123] Step S131, determining multiple target transmission frequency bands using a target frequency band allocation strategy;

[0124] Step S132, dividing the data to be transmitted to obtain multiple data subsets;

[0125] Step S133: Use multiple target transmission frequency bands to transmit multiple data subsets in parallel to obtain a target transmission result.

[0126] For example, a multi-band cooperative communication method based on multi-task learning is adopted to jointly train multiple related tasks to achieve efficient cooperative transmission between different frequency bands, thereby improving the performance of the entire communication system. This method makes full use of the characteristics of different frequency bands, and at the same time reduces interference between frequency bands by sharing implicit representations, optimizes resource allocation, and achieves the purpose of maximizing the utilization efficiency of each frequency band.

[0127] Exemplarily, according to the target frequency band allocation strategy, multiple target transmission frequency bands are selected from the available frequency bands. After the target transmission frequency bands are determined, the transmission data D is divided into different subsets D i , which can be expressed as D = {D1, D2, ..., D N}. When the target transmission frequency band and data subsets are determined, the communication system allocates each subset to different frequency bands for parallel transmission. For example, data with high real-time requirements (such as video streams) are allocated to frequency bands with higher bandwidth and lower interference, while data with lower bandwidth requirements (such as text information) are allocated to other frequency bands.

[0128] Furthermore, the implicit representation h is defined as a shared parameter, and its calculation method is shown in formula (12):

[0129] h=f θ (X) Formula (12)

[0130] Among them, X is the input feature containing various communication environment characteristics and frequency band related information; θ is the shared model parameter. Implicit representation plays a key role in multi-band collaborative communication. It is a carrier for sharing knowledge between different tasks and can help various frequency bands work better together.

[0131] In multi-band cooperative communication, the transmission of each frequency band is regarded as an independent task. These tasks include but are not limited to frequency band selection, power allocation, interference management, etc. Each task has its specific goals and requirements. For example, the frequency band selection task needs to select the most suitable frequency band according to the current communication environment, the power allocation task needs to determine the transmission power of each user on different frequency bands, and the interference management task needs to minimize the interference between different frequency bands.

[0132] For each task (band transmission), define its loss function L i The loss function consists of two parts. One part is the loss related to the task itself. For example, the frequency band selection task can use the cross entropy loss function to measure the difference between the predicted frequency band and the actual frequency band. The other part is the loss related to interference suppression, which is expressed as λ·interference suppression, where λ is the weight coefficient used to balance the relationship between the task loss and the interference suppression loss. Therefore, the loss function of each task can be expressed by formula (13):

[0133] Li =g i (h)+λ·Interference suppression formula (13)

[0134] The total loss function L is the weighted sum of all tasks, and its calculation process is shown in formula (14):

[0135]

[0136] Among them, α i is the weight coefficient for each task, which is used to further balance the contribution of different tasks in the total loss function. By minimizing the total loss function shown in formula (14), the model can find an optimal balance point between different tasks, so that the transmission of each frequency band can be optimized, and the interference between frequency bands can also be effectively controlled.

[0137] For example, during the training process, the input feature X is first input into the shared network, and the shared network calculates the implicit representation based on its parameters θ. Then, the implicit representation h is input into each task-specific network respectively, and each task-specific network calculates the predicted output of each task based on its own parameters and implicit representation. Next, the loss function L of each task is calculated based on the true label i , and the total loss function L is obtained by weighted summation. Finally, the gradient descent method or its variants (such as Adam) are used to update the shared network parameters θ and the task-specific network parameters according to the total loss function L. Through multiple iterations of the above process, the model gradually learns the synergistic relationship between different frequency bands and the optimal strategy for each task.

[0138] Based on the above optional embodiments, by intelligently selecting multiple target transmission frequency bands and specifically dividing the data to be transmitted into multiple subsets, efficient parallel transmission of data can be achieved, and spectrum resource utilization and communication performance can be optimized.

[0139] Figure 5 is a schematic diagram of another data communication method according to one embodiment of the present disclosure, such as Figure 5 As shown in the figure, during the data communication process, the transmitted data is first divided into different subsets, and the implicit representation is defined as a shared parameter. Furthermore, the loss function of each frequency band task is defined, and multiple tasks are jointly trained to optimize the total loss function. Finally, the gradient descent algorithm is used to update the parameters and output the optimal strategy.

[0140] Optionally, the data communication method further includes:

[0141] Step S41, obtaining historical network load data;

[0142] Step S42, using the target load prediction network to analyze and predict the historical network load data and the communication environment data to obtain a load prediction result, wherein the load prediction result is used to represent the network load in a future time period;

[0143] Step S43, determining bandwidth change data corresponding to the target transmission frequency band based on the load prediction result;

[0144] Step S44: adjusting the target transmission frequency band using the bandwidth change data to obtain a bandwidth adjustment result corresponding to the target transmission frequency band.

[0145] The above-mentioned historical network load data refers to the record of the load conditions experienced by the communication network in a certain period of time in the past, including but not limited to data transmission volume, user behavior patterns, channel usage, network congestion status and historical bandwidth allocation.

[0146] For example, in order to adaptively adjust the frequency band bandwidth, a deep learning model that can predict network load changes is first needed. The historical network load data (such as L(t), L(t-1), ...) and other related communication environment features are used as inputs, and the future changes in network load are predicted by learning the patterns and rules in the above data. Assume that the deep learning model is f DL , then the prediction process of future network load L(t+1) is as shown in formula (15):

[0147] L(t+1)=f DL (L(t),L(t-1),…) Formula (15)

[0148] Furthermore, the bandwidth change data corresponding to the target transmission frequency band is determined based on the load prediction result. The bandwidth allocation of the frequency band is defined as ΔB i , the bandwidth B at time t+1 can be calculated using formula (16) i (t+1):

[0149] B i (t+1)=B i (t)+ΔB i (t) Formula (16)

[0150] Among them, B i (t) is the bandwidth at the current time t; ΔB i (t) is the adjustment amount predicted according to the network load change. If it is predicted that the network load will increase in the future, then ΔB i (t) may be negative, indicating that the bandwidth of the current frequency band needs to be reduced to avoid network congestion; if it is predicted that the network load will decrease in the future, then ΔB i (t) may be a positive number, indicating that the bandwidth of the current frequency band can be increased to improve spectrum utilization.

[0151] In practical applications, ΔB is determined based on the predicted network load changes. i (t) There are many factors to consider. First, the characteristics of different frequency bands and their current usage should be considered. Some frequency bands may be more sensitive to changes in network load and require more precise adjustments; while some frequency bands may have larger bandwidths and are more adaptable to changes in network load, so relatively rough adjustments can be made. Second, the load balance of the entire communication system should be considered. If bandwidth adjustments to a certain frequency band will cause an imbalance in the load of the entire system, then the adjustment amount needs to be appropriately corrected.

[0152] Based on the above optional embodiments, the frequency band bandwidth adaptive adjustment strategy combined with deep learning and intelligent decision-making can ensure that under any circumstances, the system can adjust the frequency band bandwidth in time to cope with load changes, while maintaining coordination and consistency between frequency bands to achieve optimization of communication performance and spectrum efficiency.

[0153] Optionally, the data communication method further includes:

[0154] Step S51, using the target power prediction network to analyze and process the communication environment data to obtain a target power allocation strategy, wherein the target power allocation strategy is used to determine the power allocation result in a future time period;

[0155] Step S52, determining power change data based on the target power allocation strategy;

[0156] Step S53: adjusting the target transmission frequency band using the power change data to obtain a power adjustment result corresponding to the target transmission frequency band.

[0157] Exemplarily, a reinforcement learning model is used to adaptively adjust power control, with current communication environment data as input, and the target power allocation strategy is determined by learning the relationship between communication environment characteristics and power allocation. Assuming that the reinforcement learning model is G, the target power allocation strategy can be obtained by formula (17):

[0158] P i (t+1)=P i (t)+ΔP i (t) Formula (17)

[0159] Among them, P i (t) represents the power allocation value on the i-th frequency band at time t; ΔP i (t) represents the power change data on the i-th frequency band at time t; P i (t+1) represents the new power allocation value on the i-th frequency band at time t+1.

[0160] Furthermore, the target transmission frequency band is adjusted using the power change data to obtain the power adjustment result corresponding to the target transmission frequency band. Specifically, when the interference increase is detected, ΔP i (t) may be a negative number, indicating that the power of the current frequency band needs to be reduced to reduce interference; when the communication link is detected to be unstable, ΔP i (t) may be a positive number, indicating that the power of the current frequency band needs to be increased to improve the stability of the communication link.

[0161] Based on the above optional embodiments, through the deep analysis and intelligent processing of communication environment data by the reinforcement learning model, adaptive power control in dynamic communication scenarios can be achieved, so that power allocation can be intelligently adjusted to cope with communication challenges such as interference and link instability, thereby achieving fine control of power.

[0162] Figure 6 is a schematic diagram of another data communication method according to one embodiment of the present disclosure, such as Figure 6 As shown, in the process of data communication, the network load change is first predicted based on the output data of the deep learning model. Further, based on the above predicted network load change, the frequency band bandwidth is adaptively adjusted and the optimal power allocation strategy is determined. Finally, based on the reinforcement learning model, the power control is adaptively adjusted.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0164] In the embodiments of the present disclosure, a data communication device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0165] Figure 7 is a structural block diagram of a data communication device according to one embodiment of the present disclosure, such as Figure 7 As shown, the device comprises:

[0166] The first acquisition module 701 is used to acquire the spatiotemporal feature vector corresponding to the communication environment data;

[0167] The first processing module 702 is used to analyze and process the spatiotemporal feature vector using a target deep learning reinforcement model to obtain a target frequency band allocation strategy, wherein the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environmental feedback data;

[0168] The transmission module 703 is used to transmit the data to be transmitted according to the target frequency band allocation strategy during the multi-frequency band cooperative communication process to obtain a target transmission result.

[0169] Optionally, the first acquisition module 701 is also used to: acquire communication environment data, wherein the communication environment data includes at least: channel state data, received signal strength indication data, and spectrum usage data of neighboring base stations; perform convolution operations on the communication environment data using a convolutional neural network to obtain a spatial feature vector corresponding to the communication environment data; perform feature extraction on the communication environment data using a long short-term memory network to obtain a time feature vector corresponding to the communication environment data; and determine the spatiotemporal feature vector based on the spatial feature vector and the time feature vector.

[0170] Optionally, the data communication device further includes:

[0171] A first determination module 704 is used to determine target state data based on the spatiotemporal feature vector, wherein the target state data is used to represent the current communication environment state;

[0172] A second determination module 705 is used to determine target action data using the target state data, wherein the target action data is used to represent a current frequency band allocation strategy corresponding to a current communication environment state;

[0173] A second acquisition module 706, for acquiring a real-time reward function and updating state data based on the target state data and the target action data;

[0174] The storage module 707 is used to store the target state data, the target action data, the real-time reward function and the updated state data in the historical interaction storage area.

[0175] Optionally, the data communication device further includes:

[0176] The third acquisition module 708 is used to acquire historical experience samples from the historical interaction storage area;

[0177] A third determination module 709 is used to determine a target action value function and a target expected value based on historical experience samples, wherein the target action value function is used to represent the expected reward of target state data and target action data management;

[0178] A fourth determination module 710, configured to determine a first loss function based on the target action value function and the target expected value;

[0179] The training module 711 is used to iteratively train the initial deep learning reinforcement model using the first loss function to obtain the target deep learning reinforcement model.

[0180] Optionally, the transmission module 703 is further used to: determine multiple target transmission frequency bands using a target frequency band allocation strategy; divide the data to be transmitted to obtain multiple data subsets; and use multiple target transmission frequency bands to transmit multiple data subsets in parallel to obtain a target transmission result.

[0181] Optionally, the transmission module 703 is also used to: obtain historical network load data; use the target load prediction network to analyze and predict the historical network load data and communication environment data to obtain a load prediction result, wherein the load prediction result is used to represent the network load in a future time period; determine the bandwidth change data corresponding to the target transmission frequency band based on the load prediction result; use the bandwidth change data to adjust the target transmission frequency band to obtain a bandwidth adjustment result corresponding to the target transmission frequency band.

[0182] Optionally, the transmission module 703 is also used to: use the target power prediction network to analyze and process the communication environment data to obtain a target power allocation strategy, wherein the target power allocation strategy is used to determine the power allocation results in a future time period; determine power change data based on the target power allocation strategy; use the power change data to adjust the target transmission frequency band to obtain a power adjustment result corresponding to the target transmission frequency band.

[0183] According to another aspect of an embodiment of the present disclosure, an electronic device is further provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the data communication method in the embodiment of the present disclosure.

[0184] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0185] S1, obtain the spatiotemporal feature vector corresponding to the communication environment data;

[0186] S2, using the target deep learning reinforcement model to analyze and process the spatiotemporal feature vector to obtain the target frequency band allocation strategy, where the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environmental feedback data;

[0187] S3, in the multi-band cooperative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result.

[0188] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program runs, the device where the storage medium is located is controlled to execute the data communication method in the embodiment of the present disclosure.

[0189] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0190] S1, obtain the spatiotemporal feature vector corresponding to the communication environment data;

[0191] S2, using the target deep learning reinforcement model to analyze and process the spatiotemporal feature vector to obtain the target frequency band allocation strategy, where the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environmental feedback data;

[0192] S3, in the multi-band cooperative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result.

[0193] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0194] According to another aspect of the embodiments of the present disclosure, a computer program product is further provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the data communication method in the embodiments of the present disclosure is implemented.

[0195] Optionally, in this embodiment, the computer program product may be configured as a computer program for executing the following steps:

[0196] S1, obtain the spatiotemporal feature vector corresponding to the communication environment data;

[0197] S2, using the target deep learning reinforcement model to analyze and process the spatiotemporal feature vector to obtain the target frequency band allocation strategy, where the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environmental feedback data;

[0198] S3, in the multi-band cooperative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain the target transmission result.

[0199] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0200] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0201] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0202] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0203] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0205] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the present disclosure.

Claims

1. A data communication method, characterized in that: include: Obtaining a spatiotemporal feature vector corresponding to the communication environment data; The target deep learning reinforcement model is used to analyze and process the spatiotemporal feature vector to obtain a target frequency band allocation strategy, wherein the target frequency band allocation strategy is used to allocate target spectrum resources during data transmission, and the deep learning reinforcement model is updated through real-time environmental feedback data; In the multi-band cooperative communication process, data to be transmitted is transmitted according to the target frequency band allocation strategy to obtain a target transmission result.

2. The data communication method according to claim 1, characterized in that: Acquiring the spatiotemporal feature vector corresponding to the communication environment data includes: Acquire the communication environment data, wherein the communication environment data at least includes: channel state data, received signal strength indication data, and spectrum usage data of neighboring base stations; Using a convolutional neural network to perform a convolution operation on the communication environment data to obtain a spatial feature vector corresponding to the communication environment data; Extracting features from the communication environment data using a long short-term memory network to obtain a time feature vector corresponding to the communication environment data; The spatiotemporal feature vector is determined based on the spatial feature vector and the temporal feature vector.

3. The data communication method according to claim 1, characterized in that: The method further comprises: Determining target state data based on the spatiotemporal feature vector, wherein the target state data is used to represent a current communication environment state; Determining target action data using the target state data, wherein the target action data is used to represent a current frequency band allocation strategy corresponding to the current communication environment state; Acquire a real-time reward function and update state data based on the target state data and the target action data; The target state data, the target action data, the real-time reward function and the updated state data are stored in a historical interaction storage area.

4. The data communication method according to claim 3, characterized in that: The method further comprises: Acquire historical experience samples from the historical interaction storage area; Determining a target action value function and a target expected value based on the historical experience sample, wherein the target action value function is used to represent the expected reward of the target state data and the target action data management; Determining a first loss function based on the target action value function and the target expected value; The initial deep learning reinforcement model is iteratively trained using the first loss function to obtain the target deep learning reinforcement model.

5. The data communication method according to claim 1, characterized in that: In the multi-band cooperative communication process, the data to be transmitted is transmitted according to the target frequency band allocation strategy, and obtaining the target transmission result includes: Determining a plurality of target transmission frequency bands using the target frequency band allocation strategy; Dividing the data to be transmitted to obtain multiple data subsets; The multiple data subsets are transmitted in parallel using the multiple target transmission frequency bands to obtain the target transmission result.

6. The data communication method according to claim 5, characterized in that: The method further comprises: Get historical network load data; Analyzing and predicting the historical network load data and the communication environment data using a target load prediction network to obtain a load prediction result, wherein the load prediction result is used to represent the network load in a future time period; Determine bandwidth change data corresponding to the target transmission frequency band based on the load prediction result; The target transmission frequency band is adjusted using the bandwidth change data to obtain a bandwidth adjustment result corresponding to the target transmission frequency band.

7. The data communication method according to claim 5, characterized in that: The method further comprises: Analyzing and processing the communication environment data using a target power prediction network to obtain a target power allocation strategy, wherein the target power allocation strategy is used to determine a power allocation result in a future time period; Determining power change data based on the target power allocation strategy; The target transmission frequency band is adjusted using the power change data to obtain a power adjustment result corresponding to the target transmission frequency band.

8. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the data communication method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the data communication method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises computer instructions, which, when executed by a processor, implement the data communication method as claimed in any one of claims 1 to 7.

Images