Intelligent underwater acoustic communication method and device based on mixed training sequence strategy

Abstract

The invention belongs to the field of underwater acoustic signal communication, and discloses an intelligent underwater acoustic communication method and device based on a mixed training sequence strategy, and the method comprises the following steps: dividing a subcarrier into a plurality of sub-channels through employing an orthogonal frequency division multiplexing method, modulating the parameters of the subcarrier through employing the mixed training sequence strategy, therefore, the plurality of sub-channels transmit the underwater acoustic signals in parallel; modeling a mixed training sequence subcarrier parameter adjustment problem in the orthogonal frequency division multiplexing method into a Markov decision model; designing a reward function based on bit error rate and transmission performance planning, and building a deep reinforcement learning DDPG model; and training the deep reinforcement learning DDPG model by using the collected underwater acoustic signal positive and negative samples, and detecting the bit error rate in the actual underwater acoustic communication process by using the trained deep reinforcement learning DDPG model. The underwater acoustic communication method has good generalization ability and robustness, reduces calculation complexity and communication overhead, and improves underwater acoustic communication efficiency in a complex marine environment.

Description

Technical Field

[0001] The present application belongs to the field of underwater acoustic signal communication, and more specifically, to an intelligent underwater acoustic communication method and device based on a hybrid training sequence strategy. Background Art

[0002] Underwater acoustic communication is a communication technology used in underwater environments. Its basic principle is to modulate the frequency, amplitude and other parameters of the acoustic wave signal in the water, use the sound speed difference and sound pressure difference formed by the water medium to send the sound wave to the target location, and then demodulate the received sound wave to restore the real information. Orthogonal frequency division multiplexing (OFDM) technology is a multi-carrier modulation technology that divides a single high-speed transmission information data stream into multiple parallel low-speed data streams, modulates them on different orthogonal subcarrier channels and superimposes them to form an OFDM signal. OFDM has the advantages of anti-inter-symbol interference, high spectrum efficiency, and easy implementation, and is widely used in the field of underwater acoustic communication. However, the underwater acoustic channel is a time-varying, space-varying, and frequency-varying channel, which is affected by many factors, such as water depth, water temperature, ocean currents, seabed topography, marine life, etc. These factors cause the Doppler spread, delay spread, coherence bandwidth, coherence time and other characteristics of the underwater acoustic channel to vary with time and space, thereby affecting the transmission quality of the OFDM signal.

[0003] In order to adapt to the changes in the underwater acoustic channel, it is usually necessary to insert a training sequence subcarrier (Training Sequence symbol) known to both the sender and the receiver into the signal, and dynamically adjust the parameters of the OFDM training sequence carrier to correctly estimate the current underwater acoustic channel state. However, most traditional methods use fixed training sequence subcarrier modulation, or use parameter adjustment based on channel feedback information. These methods have problems such as high bit error rate, slow delayed feedback, and large additional overhead. For example, this preset strategy is prone to excessive system time-frequency resource consumption due to excessive modulated subcarriers under stable underwater acoustic conditions. On the contrary, under large-scale time-varying underwater acoustic conditions, it may be difficult to effectively reflect the true characteristics of the channel. Therefore, this type of traditional method is difficult to adapt to the communication needs of different underwater environments and scenarios. Summary of the invention

[0004] In view of the defects of the prior art, the purpose of the present application is to provide an intelligent underwater acoustic communication method and device based on a hybrid training sequence strategy, aiming to solve the problems that traditional underwater acoustic channel anomaly detection methods cannot effectively reflect the true characteristics of the channel and are difficult to adapt to the communication needs of different underwater environments and scenarios.

[0005] To achieve the above object, the present application provides an intelligent underwater acoustic communication method based on a hybrid training sequence strategy, comprising the following steps:

[0006] S1 uses an orthogonal frequency division multiplexing method to divide the subcarrier into multiple subchannels, and uses a mixed training sequence strategy to modulate the subcarrier parameters so that the multiple subchannels transmit underwater acoustic signals in parallel;

[0007] S2 is based on the channel characteristics of the underwater acoustic modulation communication environment and models the subcarrier parameter adjustment problem of the mixed training sequence in the orthogonal frequency division multiplexing method as a Markov decision model;

[0008] S3 designs a reward function based on bit error rate and transmission performance planning according to the Markov decision model, and builds a deep reinforcement learning DDPG model, wherein the deep reinforcement learning DDPG model includes a policy network structure, an internal network structure, and a value network structure;

[0009] S4 collects the output water acoustic signal and the received water acoustic signal as positive and negative samples of the water acoustic signal, uses the positive and negative samples of the water acoustic signal to train the deep reinforcement learning DDPG model, and uses the trained deep reinforcement learning DDPG model to restore the actual water acoustic signal.

[0010] The present application proposes an intelligent underwater acoustic communication method based on hybrid training sequence carrier orthogonal frequency division multiplexing (TC-OFDM). The method can autonomously learn and select the optimal OFDM training sequence carrier parameters according to the real-time state of the underwater acoustic channel using deep deterministic policy gradient (DDPG), that is, use the reinforcement learning DDPG model to set dynamically variable subcarriers for the underwater acoustic signal, enhance the OFDM signal quality, and achieve the goal of maximizing communication efficiency.

[0011] Furthermore, the step of enabling the plurality of sub-channels to transmit the underwater acoustic signal in parallel comprises:

[0012] S101 divides the underwater acoustic channel according to the allocatable idle bandwidth, and caches the underwater acoustic signal data to be transmitted in segments;

[0013] S102 divides the cached multiple segments of underwater acoustic signal data into signal data queues, and adds the training sequence subcarriers to the signal data queues;

[0014] S103 uses multiple threads to perform inverse fast Fourier transform on the underwater acoustic signal to which the training sequence subcarrier is added, and transmits the underwater acoustic signal in serial form;

[0015] S104 repeats steps S101 to S103 until all the underwater acoustic signals to be transmitted are transmitted.

[0016] Furthermore, the specific steps of constructing the Markov decision model include:

[0017] S201 defines the state space of the Markov decision model: selects all water acoustic signal data segments with the same <hydrophone ID, hydrophone address, duration, modulation protocol> from the water acoustic signal output in step S1, and groups the water acoustic signal data segments;

[0018] S202 defines the action space ai of the Markov decision model as:

[0019] ai={A_a,A_t,A_f}

[0020] Wherein, A_a is the training sequence type, A_t represents the set of interpolation positions of the training sequence in the time domain, and A_f represents the set of interpolation positions of the training sequence in the frequency domain;

[0021] S203 decomposes the vector of the action space into vectors including two dimensions of time domain and frequency domain, and converts the discrete action space parameters into probability distribution vectors.

[0022] Furthermore, in step S202, if the training sequence subcarrier interpolation is performed in the time domain, A_a is a normal training sequence, and its time domain and frequency domain positions are recorded; if the training sequence subcarrier interpolation is performed in both the time domain and the frequency domain, A_a is an orthogonal training sequence, and its time domain and frequency domain positions are recorded.

[0023] Furthermore, in step S3, the step of designing a reward function based on bit error rate and transmission performance planning includes:

[0024] S301 builds an average bit error rate calculation model:

[0025]

[0026] Among them, α and β are modulation coefficients, and F(x) is the cumulative distribution function of the signal-to-noise ratio of the underwater acoustic channel;

[0027] S302 constructs a signal power factor model of the underwater acoustic signal transmission process:

[0028]

[0029] Among them, R p is the signal power factor, p(x) is the OFDM signal power value transmitted through the underwater acoustic channel, x represents the xth underwater acoustic signal transmission sequence, and N is the total number of underwater acoustic signal transmissions;

[0030] S303 constructs the reward function based on the average bit error rate calculation model and the signal power factor model.

[0031] Furthermore, the reward function is expressed as:

[0032]

[0033] Among them, w k is the underwater acoustic signal weight, r i,j is the linear relationship between the average bit error rate and signal power during the system transmission process, b is a constant parameter, s t is the state space, a t is the action space, R(s t， a t ) represents the sum of the weighted benefits of each transmitted underwater acoustic signal, R D is the average bit error rate, R p is the signal power factor, k represents the kth underwater acoustic transmission sequence, and n is the total number of transmission sequence numbers.

[0034] Furthermore, attention modules are integrated into both the strategy network structure and the value network structure.

[0035] In a second aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is used to execute the intelligent underwater acoustic communication method described in the first aspect or any possible implementation method of the first aspect.

[0036] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the intelligent underwater acoustic communication method described in the first aspect or any possible implementation of the first aspect.

[0037] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the intelligent underwater acoustic communication method described in the first aspect or any possible implementation of the first aspect.

[0038] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0039] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0040] (1) The detection method of this application combines the deep reinforcement learning DDPG model technology, proposes a deep reinforcement learning-based OFDM parameter optimization framework, and designs a hybrid deep neural network structure based on a shared actor-critic (Co-Actor-Critic). The hybrid deep neural network structure combines the self-attention mechanism and can effectively extract the time-frequency characteristics and time-varying characteristics of the underwater acoustic channel. The hybrid deep neural network structure can effectively extract the time-frequency characteristics and time-varying characteristics of the underwater acoustic channel and output the optimal OFDM parameters; a reward function based on bit error rate and transmission power is designed. The reward function can comprehensively consider the communication quality and energy efficiency, guide the hybrid deep neural network structure to learn and optimize, has good generalization ability and robustness, and can cope with unknown or dynamically changing underwater acoustic channel environments.

[0041] (2) The present application models the training sequence carrier parameter adjustment problem in OFDM as a Markov decision process (MDP), where the state space is a vector composed of underwater acoustic channel characteristics, the action space is a set composed of optional OFDM parameters, and the reward function is a function defined by communication performance indicators (bit error rate, throughput, etc.); in addition, a deep deterministic policy gradient reinforcement learning algorithm (DDPG) is used to train an agent (Agent), and the training process consists of three deep networks, among which the internal network (Internal) is used to transfer gradient feature parameters between the actor network and the critic network, one is the policy network (Actor), which is used to generate OFDM training sequence subcarrier parameter actions; the other is the value network (Critic), which is used to evaluate the value of the action so that it can select the optimal action according to the current state and update its strategy according to the feedback reward; in this way, the agent can adaptively learn the characteristics and dynamic change laws of the underwater acoustic channel without any prior knowledge or external guidance, thereby realizing OFDM efficient parameter optimization underwater acoustic communication.

[0042] (3) The present application can implement an adaptive adjustment strategy for the subcarrier parameters of the OFDM training sequence of underwater acoustic communication, and adapt to changes in the underwater acoustic channel by balancing the time-frequency resource consumption and channel communication performance of the OFDM underwater acoustic communication system. During the underwater acoustic communication process, there is no need to perform channel estimation or feedback information based on traditional preset parameters to avoid falling into a local optimal solution in a fixed environment. At the same time, the pre-trained model of deep reinforcement learning can also be deployed on the transducer client, reducing the computational complexity and communication overhead, and can effectively improve the efficiency of underwater acoustic communications in complex marine environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flow chart of an intelligent underwater acoustic communication method based on a hybrid training sequence strategy provided in this application;

[0044] Figure 2 OFDM underwater acoustic communication architecture diagram based on hybrid training sequence strategy provided by this application;

[0045] Figure 3 OFDM channel modulation diagram of the hybrid training sequence strategy provided by this application;

[0046] Figure 4 A diagram of the deep learning model of OFDM underwater acoustic signals for the deterministic strategy provided in this application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0049] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0050] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0051] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0052] The technical solutions provided in the embodiments of the present application are described in detail below.

[0053] This embodiment provides an intelligent underwater acoustic communication method based on a hybrid training sequence strategy. Figure 1 and 2 As shown, the intelligent underwater acoustic communication method includes the following steps:

[0054] S1 initializes the underwater acoustic modulation communication environment: the subcarrier of the signal is divided into multiple subchannels using the orthogonal frequency division multiplexing method, and the subcarrier parameters are modulated using a mixed training sequence strategy so that multiple subchannels can transmit underwater acoustic signals in parallel.

[0055] The specific steps to initialize the underwater acoustic modulation communication environment are as follows:

[0056] S101 divides the underwater acoustic channel according to the allocatable idle bandwidth, and caches the underwater acoustic signal data to be transmitted in segments;

[0057] S102 divides the cached multiple segments of underwater acoustic signal data into signal data queues, and adds the training sequence subcarriers to the signal data queues;

[0058] Specifically, at the underwater acoustic signal data transmitting end of the underwater acoustic signal communication system (hereinafter referred to as the system), the OFDM underwater acoustic signal is divided into underwater acoustic signal data queues, and the transmitted data is recorded as:

[0059]

[0060] The system sending module sets the size of the parallel signal data queue and divides the sending data into M parallel data blocks according to the number of parallel threads supported by the system, expressed as:

[0061] [D s ] M ={[D s ] 1 ,[D s ] 2 , [D s ] 3 ,...,[D s ] M-1},

[0062] [D s ] k ∈D s and k={0,1,2,...,N}∈M; (2)

[0063] Among them, D s is the transmitted water acoustic signal data, M is the number of parallel data blocks, k represents the total number of transmitted water acoustic signals, and n represents the nth transmitted water acoustic signal data.

[0064] S103 uses multithreading to perform inverse fast Fourier transform on the underwater acoustic signal after the training sequence subcarrier is added, and serializes the transmitted underwater acoustic signal;

[0065] Specifically, initialize the OFDM underwater acoustic communication bandwidth, record the system dynamically allocable bandwidth as B, then divide the carrier into N bandwidths, the bandwidth of each subcarrier is f = B / N. Record the phase factor of the wave modulation process as In which, is the sth subcarrier, j is the modulation amplitude, is the phase rotation angle. The inverse fast Fourier transform algorithm (IFFT) is used to process the underwater acoustic signal after adding the training sequence subcarrier. It can be parallelized in many ways. For example, OpenMP can be used to create multiple threads and assign different data blocks to each thread for IFFT calculation. Specifically, the system multiplies the scrambled phase sequence with the transmitted underwater acoustic data, adds redundant information to the data signal, and generates an underwater acoustic signal data transmission set at the system transmitter.

[0066] S104 repeats steps S101 to S103 until all OFDM underwater acoustic signals to be processed are successfully sent.

[0067] S2 builds a Markov decision model: Based on the channel characteristics of the underwater acoustic modulation communication environment, the subcarrier parameter adjustment problem of the mixed training sequence in the orthogonal frequency division multiplexing method is modeled as a Markov decision model. Specifically, the following steps are included:

[0068] S201 defines the state space of the Markov decision model: select all water acoustic signal data segments with the same <hydrophone ID, hydrophone address, duration, modulation protocol> from the water acoustic signal output in step S1, and group the water acoustic signal data segments into {S_1, S_2, S_3, …, S_n}, where S_n represents the nth data segment;

[0069] S202 defines the action space of the Markov decision model as:

[0070] ai={A_a,A_t,A_f} (3)

[0071] Wherein, A_a={a1, a2} is the training sequence type, A_t={t1, t2} represents the set of interpolation positions of the training sequence in the time domain, A_f={f1, f2} represents the set of interpolation positions of the training sequence in the frequency domain, and ai represents the action space;

[0072] Specifically, Figure 3 As shown, if the training sequence subcarrier interpolation is performed in the time domain, then in ai={A_a, A_t, A_f}, A_a is a normal training sequence, and its time domain and frequency domain positions are recorded; if the training sequence subcarrier interpolation is performed in both the time domain and the frequency domain, then in ai={A_a, A_t, A_f}, A_a is an orthogonal training sequence, and its time domain and frequency domain positions are recorded.

[0073] S203 decomposes the action space ai vector into Figure 3 The vector shown in the figure contains two dimensions: time domain and frequency domain. t is the component of the underwater acoustic signal in the time domain, n f The components of the underwater acoustic signal in the frequency domain are represented, and the Gumbel-softmax method (a reparameterization technique for processing discrete variables) is used to transform the discrete action space parameters into probability distribution vectors, which serve as the input of the deterministic gradient strategy DDPG neural network constructed subsequently.

[0074] Specifically, we record an n-dimensional vector v output in the action space and generate n independent samples {∈ 1 ,…,∈ n}, through G i =-log(-log(∈ i )) Calculate G i ,∈ n is the nth independent sample that follows the U(0,1) distribution, G i Gumbel distribution is a function for random sampling of discrete values.

[0075] The corresponding addition results in a new value vector:

[0076] v′=[v 1 +G 1 ,v 2 +G 2 , ..., v n +G n ] (4)

[0077] Among them, v' is the new value vector, v n is the nth original discrete vector, G n is the nth continuous distribution vector.

[0078] The selection probability of each category is calculated by the softmax function, and the value is gradually reduced in subsequent model training to approach the real discrete distribution:

[0079]

[0080] Where τ is the temperature parameter, v i ' and v j ' represents the unnormalized score, e is a natural constant; i and j represent the category numbers.

[0081] S3 designs a reward function based on bit error rate and transmission performance planning according to the Markov decision model. The specific steps include:

[0082] S301 builds an average bit error rate calculation model:

[0083]

[0084] Where α and β are modulation coefficients, F(x) is the cumulative distribution function of the signal-to-noise ratio of the underwater acoustic channel, x represents the xth underwater acoustic signal transmission sequence, and N is the total number of underwater acoustic signal transmissions;

[0085] S302 builds a signal power factor model for the underwater acoustic signal transmission process:

[0086]

[0087] Among them, R p is the signal power factor, p(x) is the OFDM signal power value transmitted in the underwater acoustic channel;

[0088] S303 constructs a reward function based on the average bit error rate calculation model and the signal power factor model. The reward function is expressed as:

[0089]

[0090] Among them, w k is the underwater acoustic signal weight, r i,j is the linear relationship between the average bit error rate and signal power during the system transmission process, b is a constant parameter, s t is the state space, a t is the action space, R(s t， a t ) represents the sum of the weighted benefits of each transmitted underwater acoustic signal, R D is the average bit error rate, R p is the signal power factor, k represents the kth underwater acoustic transmission sequence, and n is the total number of transmission sequence numbers.

[0091] The above reward function provides a feedback mechanism for the agent, enabling it to adjust its behavior according to the feedback from the environment, which helps to achieve effective reinforcement learning. That is, the agent can learn how to take actions in the underwater acoustic communication environment to achieve its goals by maximizing the cumulative reward, and has good generalization ability and robustness.

[0092] like Figure 4 As shown in the figure, the strategy network structure and value network structure of the DDPG model are constructed. Compared with the ordinary DDPG network model, this embodiment designs a hybrid network structure based on a shared actor-critic (Co-Actor-Critic) and combines the self-attention module transformer to obtain a deep reinforcement learning DDPG model. The specific steps include:

[0093] (1) Design a hybrid deterministic gradient strategy DDPG network structure based on a shared actor-critic (Co-Actor-Critic). The policy network Actor integrates an attention transformer structure. The output of Actor is a deterministic action, which generates the network definition of this deterministic action action. Actor also has a target network with the same structure but different parameters, which is used to update the value network Critic. The internal network Internal is a shared two-layer fully connected layer, which serves as the shared underlying network of the actor network and the critic network to ensure that the gradient features can be passed to the corresponding network during the training process. The input of the value network Critic is the state-action pair. A transformer module is added to the deep neural network. The transformer module obtains the weights trained by the internal network as input, and then encodes the input features by stacking multiple Encoder structures to obtain the encoded information matrix. Finally, the information is input into the stacked Decode structure to decode the encoded information matrix in space or channel, and the weighted global information is used as the output Q value, so as to use the transformer's self-attention mechanism to enhance the state representation in DDPG and capture dependencies over a longer time range, thereby obtaining a deep reinforcement learning DDPG model.

[0094] (2) In the training of the deep reinforcement learning DDPG model, first, take a sample (S, a, r, S′) from the experience pool for training and update the current critic network. Input S and a in the experience pool (S, a, r, S′) into the current critic to obtain the current Q value. Then, update the current Actor network, calculate the action corresponding to the state s by using the Actor network, give the Q value in the current critic, and update the Actor to maximize the Q value output. Finally, in the internal network, the output of the underwater acoustic signal sample is divided into several two-dimensional matrices of the same size through the Encoder and Decoder structures, and the shared fully connected layer is used for feature extraction according to mean pooling or maximum pooling. The obtained hidden features are concat-operated with the action action, and the parameters of the current network are used to update the target Actor network and the target Critic network.

[0095] S4 trains the deep reinforcement learning DDPG model (i.e., the deterministic gradient strategy DDPG neural network): collects the output hydroacoustic signal and the received hydroacoustic signal as positive and negative samples of the hydroacoustic signal, and uses the positive and negative samples of the hydroacoustic signal to train the deep reinforcement learning DDPG model. The specific training steps need to be combined with the aforementioned reward function. Since the model training method is a conventional technical means in this field and is not the innovation of this application, it will not be repeated here; then use the trained deep reinforcement learning DDPG model to restore the hydroacoustic signal in the actual hydroacoustic communication process.

[0096] Specifically, at the receiving end of the system, the system tests and monitors the underwater acoustic communication link signal, and uses the underwater acoustic signal samples sent and received by the system as the input of the DDPG model. The policy network and evaluation network in the DDPG model are trained through the captured positive and negative samples of the underwater acoustic signals. Combined with the reward function, the system automatically adjusts the time-frequency parameters of the training sequence carrier according to the model decision results, and continuously monitors the bit error rate during the underwater acoustic communication process.

[0097] After getting the trained deep learning model, perform the following steps for practical application:

[0098] (1) Deploy the trained deep learning model on the target server.

[0099] (2) The system turns on the underwater acoustic receiving module, collects OFDM underwater acoustic signals with underwater acoustic noise, records the OFDM underwater acoustic signals sent and received as samples, and calculates transmission characteristic information such as channel bit error rate, communication power, and training sequence occupied bandwidth.

[0100] (3) The system starts the underwater acoustic signal transmission analysis module, loads the trained deep neural network model, uses the channel estimation algorithm to calculate the channel characteristics, and then performs a fast Fourier transform to restore the underwater acoustic signal.

[0101] This application proposes an intelligent underwater acoustic modulation detection method based on hybrid training sequence carrier orthogonal frequency division multiplexing (TC-OFDM). This method can autonomously learn and select the optimal OFDM training sequence carrier parameters according to the real-time state of the underwater acoustic channel using deep deterministic policy gradient (DDPG) to achieve the goal of maximizing communication efficiency.

[0102] This application combines the deep reinforcement learning DDPG model technology to propose a new OFDM parameter optimization framework based on deep reinforcement learning, and designs a hybrid deep neural network structure based on a shared actor critic (Co-Actor-Critic). The hybrid deep neural network structure combines the self-attention mechanism, which can effectively extract the time-frequency characteristics and time-varying characteristics of the underwater acoustic channel and output the optimal OFDM parameters; it also designs a reward function based on bit error rate and transmission power. The reward function can comprehensively consider the communication quality and energy efficiency, guide the deep neural network to learn and optimize, has good generalization ability and robustness, and can cope with unknown or dynamically changing underwater acoustic channel environments.

[0103] It can be understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0104] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method, which will not be repeated here.

[0105] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device, which may include: a processor, a communication interface, a memory, and an underwater acoustic transducer, wherein the processor, the communication interface, and the memory communicate with each other through the underwater acoustic transducer, and the underwater acoustic transducer can convert the underwater acoustic signal into an electrical signal. The processor can call the logic instructions in the memory to execute the method in the above embodiment.

[0106] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0107] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0108] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0109] It is understandable that the processor in the embodiment of the present application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0110] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0111] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0112] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0113] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An intelligent underwater acoustic communication method based on a hybrid training sequence strategy, characterized in that: The following steps are involved: S1 uses an orthogonal frequency division multiplexing method to divide the subcarrier into multiple subchannels, and uses a mixed training sequence strategy to modulate the subcarrier parameters so that the multiple subchannels transmit underwater acoustic signals in parallel; S2 is based on the channel characteristics of the underwater acoustic modulation communication environment and models the subcarrier parameter adjustment problem of the mixed training sequence in the orthogonal frequency division multiplexing method as a Markov decision model; S3 designs a reward function based on bit error rate and transmission performance planning according to the Markov decision model, and builds a deep reinforcement learning DDPG model, wherein the deep reinforcement learning DDPG model includes a policy network structure, an internal network structure, and a value network structure; S4 collects the output water acoustic signal and the received water acoustic signal as positive and negative samples of the water acoustic signal, uses the positive and negative samples of the water acoustic signal to train the deep reinforcement learning DDPG model; and uses the trained deep reinforcement learning DDPG model to restore the actual water acoustic signal.

2. The intelligent underwater acoustic communication method based on a hybrid training sequence strategy as claimed in claim 1, characterized in that: In step S1, multiple sub-channels are made to transmit underwater acoustic signals in parallel, which specifically includes the following steps: S101 divides the underwater acoustic channel according to the allocatable idle bandwidth, and caches the underwater acoustic signal data to be transmitted in segments; S102 divides the cached multiple sections of underwater acoustic signal data into signal data queues, and adds the training sequence subcarriers to the signal data queues; S103 uses multiple threads to perform inverse fast Fourier transform on the underwater acoustic signal to which the training sequence subcarrier is added, and transmits the underwater acoustic signal in serial form; S104 repeats steps S101 to S103 until all the underwater acoustic signals to be transmitted are transmitted.

3. The intelligent underwater acoustic communication method based on a hybrid training sequence strategy as claimed in claim 1, characterized in that: Step S2 specifically includes the following sub-steps: S201 defines the state space of the Markov decision model: selects all water acoustic signal data segments with the same <hydrophone ID, hydrophone address, duration, modulation protocol> from the water acoustic signal output in step S1, and groups the water acoustic signal data segments; S202 defines the action space ai of the Markov decision model as: ai={A_a,A_t,A_f} Wherein, A_a is the training sequence type, A_t represents the set of interpolation positions of the training sequence in the time domain, and A_f represents the set of interpolation positions of the training sequence in the frequency domain; S203 decomposes the vector of the action space into vectors including two dimensions of time domain and frequency domain, and converts the discrete action space parameters into probability distribution vectors.

4. The intelligent underwater acoustic communication method based on a hybrid training sequence strategy as claimed in claim 3, characterized in that: In step S202, if the training sequence subcarrier interpolation is performed in the time domain, A_a is a common training sequence, and its time domain and frequency domain positions are recorded; if the training sequence subcarrier interpolation is performed in both the time domain and the frequency domain, A_a is an orthogonal training sequence, and its time domain and frequency domain positions are recorded.

5. The intelligent underwater acoustic communication method based on a hybrid training sequence strategy as claimed in claim 1, characterized in that: In step S3, the steps of designing a reward function based on bit error rate and transmission performance planning include: S301 builds an average bit error rate calculation model: Where α and β are modulation coefficients, F(x) is the cumulative distribution function of the signal-to-noise ratio of the underwater acoustic channel, x represents the xth underwater acoustic signal transmission sequence, and N is the total number of underwater acoustic signal transmissions; S302 constructs a signal power factor model of the underwater acoustic signal transmission process: Among them, R p is the signal power factor, p(x) is the OFDM signal power value transmitted in the underwater acoustic channel; S303 constructs the reward function based on the average bit error rate calculation model and the signal power factor model.

6. The intelligent underwater acoustic communication method based on a hybrid training sequence strategy as claimed in claim 5, characterized in that: The reward function is expressed as: Among them, w k is the underwater acoustic signal weight, r i,j is the linear relationship between the average bit error rate and signal power during the system transmission process, b is a constant parameter, s t is the state space, a t is the action space, R(s t， a t ) represents the sum of the weighted benefits of each transmitted underwater acoustic signal, R D is the average bit error rate, R p is the signal power factor, k represents the kth underwater acoustic transmission sequence, and n is the total number of transmission sequence numbers.

7. The intelligent underwater acoustic communication method based on a hybrid training sequence strategy according to claim 1, characterized in that: Attention modules are integrated into both the strategy network structure and the value network structure.

8. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the intelligent underwater acoustic communication method as described in any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program runs on a processor, the processor executes the intelligent underwater acoustic communication method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product runs on a processor, the processor executes the intelligent underwater acoustic communication method according to any one of claims 1 to 7.

Images