Signal processing method, device, electronic device and storage medium

Abstract

The present invention provides a signal processing method, device, electronic device and storage medium, the method comprising: based on a received first received signal, using reinforcement learning to perform threshold judgment to obtain a threshold signal; inputting the threshold signal into a signal correction model to obtain a second received signal output by the signal correction model. The improved signal processing method, device, electronic device and storage medium of the present invention, based on the first received signal, continuously reinforces learning to accurately judge it, obtains a threshold signal, uses the digital time series corresponding to the threshold signal as the input of the signal correction model, and the output result is a second received signal after error correction. The time series obtained through supervised training is used to check errors, and even if the number of error bits is large, multiple errors can be corrected, and even in the case of a high error rate under a complex channel, it has a high error correction rate.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a signal processing method, device, electronic equipment and storage medium. Background Art

[0002] In a communication system, after the transmitted signal reaches the receiving end, it is usually processed by several filters. These filters may include low-pass filters, matched filters, equalizers, etc. After sampling by various filters, the decision device completes the signal detection and decision function.

[0003] The decision maker at the receiving end generally makes a decision through the BCH code, which is an abbreviation of Bose, Ray-Chaudhuri and Hocquenghem, and is used to correct multiple random error patterns of multi-level, cyclic, error correction, variable-length digital coding. However, as the communication rate increases, the probability of bit errors (i.e., the error rate) will increase, and the error correction performance of traditional BCH error correction coding will decrease, and at the same time, it will have a certain impact on the communication performance of wireless communication equipment. Summary of the invention

[0004] The present invention provides a signal processing method, device, electronic equipment and storage medium, which are used to solve the defect of low error correction rate under high error rate conditions in the prior art.

[0005] The present invention provides a signal processing method, comprising:

[0006] Based on the received first received signal, using reinforcement learning to perform threshold judgment to obtain a threshold signal;

[0007] Inputting the threshold signal into a signal correction model to obtain a second receiving signal output by the signal correction model;

[0008] Among them, the first received signal is determined according to the original baseband signal sent by the transmitting end; the signal correction model is trained based on the sample received signal received by the receiving end and the sample original baseband signal sent by the transmitting end corresponding to the sample received signal.

[0009] According to a signal processing method provided by the present invention, the method uses reinforcement learning to perform threshold judgment based on the received first received signal to obtain a threshold signal, including:

[0010] Subdividing the first received signal into amplitude values, and acquiring state information corresponding to each candidate amplitude value;

[0011] Based on the reward value corresponding to the next state information obtained after performing amplitude selection on each state information, a target amplitude is selected from the candidate amplitudes;

[0012] If it is determined that the target amplitude is greater than or equal to the reference value, the judgment result corresponding to the target amplitude is set to 1;

[0013] If it is determined that the target amplitude is less than the reference value, the judgment result corresponding to the target amplitude is set to 0;

[0014] The threshold signal is composed of the decision results corresponding to all the target amplitudes; and the reference value is determined according to the first received signal.

[0015] According to a signal processing method provided by the present invention, the reward value corresponding to the next state information obtained after performing amplitude selection on each state information, screening out a target amplitude from the candidate amplitudes, comprises:

[0016] Establishing a Q table; wherein the Q table is composed of various states, behaviors in each state, and the estimated reward value obtained by any behavior in any state;

[0017] After determining the next state information by using the epsilon-greedy strategy, updating the Q table based on the reward value and the estimated reward value corresponding to the state information being transferred to the next state information;

[0018] Based on the updated Q table, the target amplitude is selected from the candidate amplitudes.

[0019] According to a signal processing method provided by the present invention, the signal correction model includes an embedding layer, a recurrent neural network layer and a signal conversion layer;

[0020] Inputting the threshold signal into the embedding layer, and obtaining a feature vector output by the embedding layer corresponding to the threshold signal;

[0021] Inputting the feature vector into the recurrent neural network layer for time sequence arrangement, and obtaining a signal sequence output by the recurrent neural network layer;

[0022] The signal sequence is input into the signal conversion layer, and the second receiving signal output by the signal conversion layer is obtained.

[0023] According to a signal processing method provided by the present invention, the recurrent neural network layer includes a recurrent neural network model;

[0024] The step of inputting the feature vector into the recurrent neural network layer to obtain the second received signal output by the recurrent neural network layer includes:

[0025] Inputting the feature vector into the recurrent neural network model to obtain an intermediate hidden state;

[0026] Determine the weight of each of the intermediate hidden states according to the attention mechanism;

[0027] Based on the weight of each of the intermediate hidden states, the signal sequence output by the recurrent neural network model is obtained.

[0028] According to a signal processing method provided by the present invention, the recurrent neural network model includes one of a long short-term memory network LSTM, a gated recurrent unit GRU, and a bidirectional long short-term memory network BiLSTM.

[0029] The present invention also provides a signal processing device, comprising:

[0030] A threshold decision module, used to perform threshold decision based on the received first received signal by using reinforcement learning to obtain a threshold signal;

[0031] A correction processing module, used for inputting the threshold signal into a signal correction model to obtain a second received signal output by the signal correction model;

[0032] Among them, the first received signal is determined according to the original baseband signal sent by the transmitting end; the signal correction model is trained based on the sample received signal received by the receiving end and the sample original baseband signal sent by the transmitting end corresponding to the sample received signal.

[0033] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned signal processing methods when executing the program.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements any of the signal processing methods described above.

[0035] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the signal processing method described above is implemented.

[0036] The signal processing method, device, electronic device and storage medium provided by the present invention, the embodiment of the present invention is based on the first received signal, and continuously reinforces learning to accurately judge it, and obtains a threshold signal, and uses the digital time series corresponding to the threshold signal as the input of the signal correction model, and the output result is the second received signal after error correction. The time series obtained through supervised training is used to check errors, and even if the number of error bits is large, multiple errors can be corrected, and even in the case of a high error rate under a complex channel, it has a high error correction rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flow chart of the signal processing method provided by the present invention;

[0039] Figure 2 is a structural schematic diagram of a signal processing device provided by the present invention;

[0040] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0043] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present invention, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0044] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0045] Figure 1 Schematic diagram of the signal processing method provided by the present invention. Figure 1As shown, the signal processing method provided by the embodiment of the present invention includes: step 101, based on the received first received signal, using reinforcement learning to perform threshold judgment to obtain a threshold signal.

[0046] The first received signal is determined according to the original baseband signal sent by the transmitting end.

[0047] It should be noted that the execution subject of the signal processing method provided in the embodiment of the present invention is a signal processing device arranged at the receiving end.

[0048] The application scenario of the signal processing method provided by the embodiment of the present invention is that if a digital signal is to be transmitted in a channel, pulse shaping must be performed in the baseband part of the transmitting end to convert the digital signal into an original baseband signal. Since attenuation changes the voltage of the original baseband signal during signal transmission, the signal is destroyed during transmission. After the generated error signal is transmitted to the receiving end, sampling and judgment are performed in the baseband part of the receiving end to restore the digital signal.

[0049] The signal processing method provided in the embodiment of the present application is suitable for judging and recovering any received signal through the electronic device on which the receiving end relies.

[0050] The above-mentioned electronic devices can be implemented in various forms. For example, the electronic devices described in the embodiments of the present application may include mobile terminals such as mobile phones, smart phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), navigation devices, smart bracelets, smart watches, etc., and fixed terminals such as digital TVs, desktop computers, etc. Below, it is assumed that the electronic device is a mobile terminal. However, it will be understood by those skilled in the art that, in addition to components specifically for mobile purposes, the construction according to the embodiments of the present application can also be applied to fixed type terminals.

[0051] Specifically, in step 101, the signal processing device maps the received first received signal into a one-dimensional array. In the decision-making process of reinforcement learning, the reward and punishment mechanism is used to continuously collect training experience so that the correct threshold decision can be obtained in a certain state until all states are traversed and the threshold signal is output.

[0052] The threshold signal refers to the result of judging the signal at each moment in the first received signal. If its value is near the maximum end of the amplitude, it is judged as 1. Otherwise, if it is near the minimum end of the amplitude, it is judged as 0.

[0053] Step 102: Input the threshold signal to a signal correction model to obtain a second received signal output by the signal correction model.

[0054] The signal correction model is trained based on a sample reception signal received by a receiving end and a sample original baseband signal corresponding to the sample reception signal sent by a transmitting end.

[0055] It should be noted that the signal correction model can be a neural network model, and the structure and parameters of the neural network include but are not limited to the number of layers of the input layer, hidden layer and output layer of the neural network, and the weight parameters of each layer, etc. The embodiment of the present invention does not specifically limit the type and structure of the neural network.

[0056] For example, the signal correction model can be a neural network model consisting of an input layer, a hidden layer, and an output layer, where:

[0057] The input layer is at the front end of the entire network and directly receives threshold signals composed of 0 and 1.

[0058] The hidden layer can have one or more layers, and the characteristic vector corresponding to the threshold signal is calculated by weighted summation through its own neurons.

[0059] The output layer is the last layer, which is used to decode the vector obtained after weighted summation and output the second received signal.

[0060] It should be noted that the sample data includes a sample received signal corresponding to the sample data and a sample original baseband signal sent by the transmitter corresponding to the sample received signal. The sample data is divided into a training set and a test set according to a certain ratio.

[0061] Exemplarily, the ratio of the training set to the test set in the sample data includes, but is not limited to, 9:1, 8:2, etc., which is not specifically limited in the embodiment of the present invention.

[0062] Specifically, in step 102, the signal processing device initializes the weight coefficients between the layers of the constructed signal correction model, and then inputs a set of sample received signals in the training set and the sample original baseband signals corresponding to the sample received signals into the neural network under the initial weight coefficients, and calculates the outputs of each node of the input layer, hidden layer and output layer in turn. The cumulative error between the final output result of the output layer and its actual connection position state type is corrected according to the gradient descent method to correct the weight coefficients between the input layer and the nodes of the hidden layer. According to the above process, until all samples in the training set are traversed, the weight coefficients of the input layer and the hidden layer can be obtained.

[0063] The signal processing device restores the signal correction model in step 102 according to the weight coefficients of the neural network input layer and the hidden layer, and inputs the threshold signal corresponding to the first received signal in the test set into the trained signal correction model, so as to obtain a second received signal after the first received signal is corrected.

[0064] The embodiment of the present invention performs continuous reinforcement learning based on the first received signal to accurately judge it, obtains a threshold signal, uses the digital time series corresponding to the threshold signal as the input of the signal correction model, and outputs the second received signal after error correction. The time series obtained through supervised training is used to check errors, and even if the number of error bits is large, multiple errors can be corrected, and even when the error rate is high under complex channels, it has a high error correction rate.

[0065] On the basis of any of the above embodiments, based on the received first received signal, threshold judgment is performed using reinforcement learning to obtain a threshold signal, including: subdividing the first received signal into amplitudes and obtaining state information corresponding to each candidate amplitude.

[0066] Specifically, in step 101, the signal processing device subdivides the first received signal, uses the inflection points contained therein as candidate amplitudes, and obtains state information corresponding to each candidate amplitude.

[0067] Among them, the smaller the interval when the amplitude is subdivided, the more candidate amplitudes and state information of the candidate amplitudes are obtained, and the better the effect of reinforcement learning is.

[0068] The target amplitude is selected from the candidate amplitudes based on the reward value corresponding to the next state information obtained after the amplitude selection is performed on each state information.

[0069] It should be noted that reinforcement learning refers to the intelligent agent (Agent) learning in a "trial and error" manner, and the rewards obtained by interacting with the environment guide behavior. Its goal is to enable the intelligent agent to obtain the maximum reward.

[0070] Reinforcement learning can be a Markov decision process (MDPs), where:

[0071] (1) A finite state set, s represents a specific state, that is, the state information of the candidate amplitude at a certain sampling time.

[0072] (2) A finite set of actions, where a represents a specific action, i.e., an action of selecting upper and lower amplitudes.

[0073] (3) State transfer, that is, according to the state information s of a candidate amplitude and action a, traverse to the state information s' of the next candidate amplitude.

[0074] (4) Reward value, which represents the immediate reward after taking a certain action, and may also correspond to a punishment.

[0075] (5) Policy: A strategy for generating action selection based on the state information of a candidate amplitude.

[0076] Specifically, the signal processing device uses the state information corresponding to a candidate amplitude to test the environment by taking the upper and lower amplitude selection behavior, and obtains the reward value and next state information of the environment feedback for this behavior action. If the reward value is greater than 0, the tendency to execute the candidate amplitude selection strategy will be strengthened, and the candidate amplitude will be used as the target amplitude. Otherwise, the tendency will be weakened.

[0077] If it is determined that the target amplitude is greater than or equal to the reference value, the judgment result corresponding to the target amplitude is set to 1.

[0078] If it is determined that the target amplitude is less than the reference value, the judgment result corresponding to the target amplitude is set to 0.

[0079] The threshold signal is composed of the decision results corresponding to all target amplitudes. The reference value is determined according to the first received signal.

[0080] It should be noted that the reference value refers to a reference value used by the receiving end to make a decision on the received signal. Therefore, the reference value may be determined by the received first received signal, and the embodiment of the present invention does not specifically limit this.

[0081] Exemplarily, the reference value may be 0.

[0082] Specifically, the signal processing device sequentially determines the target amplitudes updated and traversed by the reinforcement learning:

[0083] If the target amplitude is greater than or equal to the reference value, that is, it is above the decision baseline, the decision result of the digital signal corresponding to the target amplitude is set to 1.

[0084] If the target amplitude is smaller than the reference value, that is, it is below the decision baseline, the decision result of the digital signal corresponding to the target amplitude is set to 0.

[0085] The threshold signal refers to a sequence signal of the decision result corresponding to each target amplitude, which is composed of "0" and "1".

[0086] The embodiment of the present invention performs amplitude subdivision on the first received signal to obtain state information corresponding to all candidate amplitudes of reinforcement learning. In the process of traversing the states of all candidate amplitudes, a threshold signal is obtained by making a judgment through the target amplitude determined by reinforcement learning. Then, the digital time series corresponding to the threshold signal is used as the input of the signal correction model, and the output result is the second received signal after error correction. Reinforcement learning can be used to cope with complex channel environments, and through continuous learning, the signal can be accurately judged, so as to have a high error correction rate even when the communication rate increases and the transmission error rate is high.

[0087] On the basis of any of the above embodiments, based on the reward value corresponding to the next state information obtained after performing amplitude selection on each state information, the target amplitude is screened out from the candidate amplitudes, including: establishing a Q table.

[0088] The Q table consists of various states, behaviors in each state, and the estimated reward value obtained for any behavior in any state.

[0089] Specifically, the signal processing device uses Q-Learning for reinforcement learning, constructs the mapping relationship between state and action into a Q table to store Q values, and then selects the action that can obtain the maximum benefit according to the Q value.

[0090] Among them, the rows of the Q table are the states of each target amplitude, and the columns of the Q table are the upper and lower amplitude selection actions taken by each target amplitude. Any row and any column can uniquely determine a corresponding Q value.

[0091] The Q value, that is, Q(s,a), is the expectation of gaining profit by taking action a under the state information s corresponding to the target amplitude at a certain moment. The environment will feedback the corresponding reward value based on the selected action.

[0092] After the next state information is determined using the epsilon-greedy strategy, the Q table is updated based on the reward value and estimated reward value corresponding to the state information transition to the next state information.

[0093] Specifically, each time a state transition is performed, the epsilon-greedy strategy is used to explore and utilize the actions taken. After determining and executing the action, the state information of the current target amplitude is calculated to transfer to the estimated reward value corresponding to the next state information.

[0094] The calculation formula for the estimated reward value new Q(s,a) is:

[0095] new Q(s,a)=Q(s,a)+α(R(s,a)+γ*max Q'(s',a')-Q(s,a))

[0096] Among them, Q(s,a) is the Q value corresponding to action a under the state information s corresponding to the target amplitude stored in the Q table. α is the learning rate, which defines how much of the new Q value learned from the estimated reward value in the original Q value in a Q table accounts for itself. Its value range is between 0 and 1. Its value of 0 means that the old information is important, and its value of 1 means that the newly discovered information is the only important information. γ is the discount factor, which defines the importance of future rewards. Its value range is between 0 and 1. Its value of 0 means that only short-term rewards are considered, and its value of 1 means that long-term rewards are more important.

[0097] α(R(s,a)+γ*max Q'(s',a')-Q(s,a)) is the reward learned from this action (the reward brought by the action itself and the potential reward in the future). R(s,a) is the reward value obtained by taking action a under the state information s corresponding to the target amplitude. max Q'(s',a') is the maximum Q value under the given state and action enumerated in the Q table.

[0098] Based on the updated Q table, the target amplitude is screened out from the candidate amplitudes.

[0099] Specifically, before traversing all states, the signal processing device updates the Q table each time, and updates the original Q value in a progressive manner. When it converges to the optimal Q value, the current candidate amplitude is used as the target amplitude for judgment, and the action value function and state value function are immediately updated to make the next traversal judgment.

[0100] The embodiment of the present invention uses Q-learning to reinforce the state information corresponding to all candidate amplitudes, and after traversing all candidate amplitudes, determines the target amplitude for judgment. The time difference method is used to integrate Monte Carlo and dynamic programming for off-policy learning, and the Bellman equation can be used to solve the optimal strategy for the Markov process.

[0101] Based on any of the above embodiments, the signal correction model includes an embedding layer, a recurrent neural network layer and a signal conversion layer.

[0102] The threshold signal is input into the embedding layer, and the feature vector output by the embedding layer corresponding to the threshold signal is obtained.

[0103] The feature vector is input into the recurrent neural network layer for time sequence arrangement to obtain the signal sequence output by the recurrent neural network layer.

[0104] The signal sequence is input into the signal conversion layer, and a second receiving signal output by the signal conversion layer is obtained.

[0105] Specifically, the signal correction model built into the signal processing device is composed of an input layer, a hidden layer and an output layer. The hidden layer is used to perform timing correction on the decision result (ie, threshold signal) of the first received signal to obtain the second received signal.

[0106] The embodiment of the present invention does not specifically limit the structure of the hidden layer.

[0107] Preferably, the hidden layer includes at least three layers, namely, an embedding layer, a recurrent neural network layer and a signal conversion layer. Among them:

[0108] The function of the embedding layer is to receive the threshold signal sent by the input layer, convert each decision result in the threshold signal into a feature vector with a fixed size, and use the feature vector as the output of the embedding layer.

[0109] The feature vector may include an operation feature and a sequence feature, wherein the operation feature is used to characterize the relevant information corresponding to each decision result, and the sequence feature is used to characterize the position information of the decision result in the time sequence.

[0110] The function of the recurrent neural network layer is to take the time-series related data as input, recursively in the direction of the sequence evolution, and all nodes are connected in a chain, sharing parameters at different time positions. In addition to the input at this moment, there is also a part of the output memory from the previous moment. At the same time, part of the output at this moment will also be passed to the next moment as "memory", and finally the recursive output is the signal sequence after the corrected timing.

[0111] The function of the signal conversion layer is to convert the judgment results of each target amplitude arranged in the correct time sequence in the signal sequence in sequence and restore them to the baseband signal.

[0112] The embodiment of the present invention converts the threshold signal into a feature vector containing operation features and sequence features based on the embedding layer, performs time sequence information association on the feature vector with time sequence features through the recurrent neural network layer, recursively obtains the signal sequence, and converts the signal sequence after the time sequence is corrected into a second received signal. It can fully explore the time sequence correlation between signals, avoid the errors caused by the fact that reinforcement learning can only make judgments based on the existing time sequence, and greatly improve the error correction rate.

[0113] Based on any of the above embodiments, the recurrent neural network layer includes a recurrent neural network model.

[0114] Inputting the feature vector into a recurrent neural network layer to obtain a second received signal output by the recurrent neural network layer includes: inputting the feature vector into a recurrent neural network model to obtain an intermediate hidden state output by the recurrent neural network model.

[0115] According to the attention mechanism, the weight of each intermediate hidden state is determined.

[0116] Based on the weight of each intermediate hidden state, the signal sequence output by the recurrent neural network model is obtained.

[0117] Specifically, the signal processing device inputs the feature vector corresponding to the threshold signal into the recurrent neural network model for processing, and uses each feature vector as a node to obtain the corresponding hidden state of each decision result in the threshold signal. The hidden state of the next node output of the node is predicted based on the hidden state of the node, and the hidden state of the last node is output as the intermediate hidden state corresponding to the threshold signal. The intermediate hidden state corresponding to each moment is continuously recursively cycled, and after obtaining the weight and bias of the recurrent neural network model, the recurrent neural network model is configured using the trained model parameters to correct the timing in the threshold signal and obtain the signal sequence.

[0118] Among them, the intermediate hidden state contains the relevant information of the previous node in the threshold signal.

[0119] The embodiment of the present invention is based on inputting the feature vector corresponding to the threshold signal into the recurrent neural network model, recursively looping through the intermediate hidden state, and obtaining the signal sequence after determining the model parameters. It can integrate the characteristics of the time-series evolution of the differential equation of the physical system, reduce the computational complexity, and improve the efficiency of the model while improving the error correction rate.

[0120] Based on any of the above embodiments, the recurrent neural network model includes one of a long short-term memory network LSTM, a gated recurrent unit GRU, and a bidirectional long short-term memory network BiLSTM.

[0121] Specifically, the recurrent neural network model set in the signal processing device will bring the previous output result to the next hidden layer for training each time.

[0122] The embodiment of the present invention does not specifically limit the recurrent neural network model.

[0123] Optionally, the recurrent neural network model may be a long short-term memory network (LSTM), which introduces a memory cell for recording additional information. In order to control the memory cell, three types of gates need to be set.

[0124] Among them, the output gate is used to output entries from the unit.

[0125] The input gate is used to decide when to read data into the cell.

[0126] The forget gate is used to reset the contents of the unit through a mechanism that is motivated by the same principles as the gated recurrent unit, which uses a dedicated mechanism to decide when to remember or ignore inputs in the hidden state.

[0127] Optionally, the recurrent neural network model can be a gated recurrent unit (GRU), which combines the forget gate and the input gate into an "update gate", which combines the neuron state and the hidden layer state, and also makes some adaptive changes.

[0128] Optionally, the recurrent neural network model can be a bidirectional long short-term memory network (Bidirectional LongShort-Term Memory, BiLSTM), which combines the information of the input sequence in both the forward and backward directions on the basis of LSTM. For the output at time t, the forward LSTM layer has the information of time t and before in the input sequence, and the backward LSTM layer has the information of time t and after in the input sequence. The output vector of the forward LSTM layer at time t and the output vector of the backward LSTM layer at time t are processed by addition, average value or connection.

[0129] The embodiment of the present invention is based on a recurrent neural network model including a long short-term memory network LSTM, a gated recurrent unit GRU or a bidirectional long short-term memory network BiLSTM. The corresponding model can be set according to different task requirements. While improving the error correction rate, the efficiency of the model is improved.

[0130] Figure 2 is a schematic diagram of the structure of the signal processing device provided by the present invention. Based on any of the above embodiments, Figure 2 As shown, the device includes: a threshold determination module 210 and a correction processing module 220, wherein:

[0131] The threshold decision module 210 is used to perform threshold decision based on the received first received signal by using reinforcement learning to obtain a threshold signal.

[0132] The correction processing module 220 is used to input the threshold signal into the signal correction model to obtain the second receiving signal output by the signal correction model.

[0133] The first received signal is determined according to the original baseband signal sent by the transmitting end. The signal correction model is trained based on the sample received signal received by the receiving end and the sample original baseband signal sent by the transmitting end corresponding to the sample received signal.

[0134] Specifically, the threshold determination module 210 and the correction processing module 220 are electrically connected in sequence.

[0135] The threshold decision module 210 maps the received first received signal into a one-dimensional array. In the decision-making process of reinforcement learning, it continuously collects training experience through a reward and punishment mechanism so that it can obtain the correct threshold decision in a certain state until all states are traversed and a threshold signal is output.

[0136] The correction processing module 220 restores the signal correction model according to the weight coefficients of the neural network input layer and the hidden layer, and inputs the threshold signal corresponding to the first received signal in the test set into the trained signal correction model, so as to obtain a second received signal after the first received signal is corrected.

[0137] Optionally, the threshold decision module 210 includes an amplitude subdivision unit, a reinforcement learning unit and a decision unit, wherein:

[0138] The amplitude subdivision unit is used to perform amplitude subdivision on the first received signal and obtain state information corresponding to each candidate amplitude.

[0139] The reinforcement learning unit is used to select a target amplitude from candidate amplitudes based on a reward value corresponding to the next state information obtained after performing amplitude selection on each state information.

[0140] The decision unit is used to set the decision result corresponding to the target amplitude to 1 if it is determined that the target amplitude is greater than or equal to the reference value.

[0141] The decision unit is further configured to set the decision result corresponding to the target amplitude to 0 if it is determined that the target amplitude is less than the reference value.

[0142] The threshold signal is composed of the decision results corresponding to all target amplitudes. The reference value is determined according to the first received signal.

[0143] Optionally, the reinforcement learning unit includes a Q table initialization subunit, a Q table update subunit and a learning subunit, wherein:

[0144] The Q table initialization subunit is used to establish a Q table. The Q table is composed of various states, behaviors in each state, and the estimated reward value obtained by any behavior in any state;

[0145] The Q table updating subunit is used to update the Q table based on the reward value and estimated reward value corresponding to the state information transferred to the next state information after the next state information is determined using the epsilon-greedy strategy.

[0146] The learning subunit is used to select a target amplitude from candidate amplitudes based on the updated Q table.

[0147] Optionally, the signal correction model includes an embedding layer, a recurrent neural network layer, and a signal conversion layer.

[0148] Accordingly, the correction processing module 220 includes a feature extraction unit, a time sequence arrangement unit and a conversion unit, wherein:

[0149] The feature extraction unit is used to input the threshold signal into the embedding layer and obtain the feature vector output by the embedding layer corresponding to the threshold signal.

[0150] The timing arrangement unit is used to input the feature vector into the recurrent neural network layer for timing arrangement, and obtain the signal sequence output by the recurrent neural network layer.

[0151] The conversion unit is used to input the signal sequence into the signal conversion layer and obtain the second receiving signal output by the signal conversion layer.

[0152] Optionally, the recurrent neural network layer includes a recurrent neural network model.

[0153] Accordingly, the time sequence arrangement unit includes an intermediate hidden state acquisition subunit, a weight determination subunit and a sorting error correction subunit, wherein:

[0154] The intermediate hidden state acquisition subunit is used to input the feature vector into the recurrent neural network model to obtain the intermediate hidden state.

[0155] The weight determination subunit is used to determine the weight of each intermediate hidden state according to the attention mechanism.

[0156] The sorting and error correction subunit is used to obtain the signal sequence output by the recurrent neural network model based on the weight of each intermediate hidden state.

[0157] Optionally, the recurrent neural network model includes one of a long short-term memory network LSTM, a gated recurrent unit GRU, and a bidirectional long short-term memory network BiLSTM.

[0158] The signal processing device provided in the embodiment of the present invention is used to execute the above-mentioned signal processing method of the present invention. Its implementation method is consistent with the implementation method of the signal processing method provided by the present invention and can achieve the same beneficial effects, which will not be repeated here.

[0159] The embodiment of the present invention performs continuous reinforcement learning based on the first received signal to accurately judge it, obtains a threshold signal, uses the digital time series corresponding to the threshold signal as the input of the signal correction model, and outputs the second received signal after error correction. The time series obtained through supervised training is used to check errors, and even if the number of error bits is large, multiple errors can be corrected, and even when the error rate is high under complex channels, it has a high error correction rate.

[0160] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the signal processing method, which includes: based on the received first received signal, using reinforcement learning to perform threshold judgment to obtain a threshold signal; inputting the threshold signal into a signal correction model to obtain a second received signal output by the signal correction model; wherein the first received signal is determined according to the original baseband signal sent by the transmitting end; the signal correction model is obtained by training based on the sample received signal received by the receiving end and the sample original baseband signal sent by the transmitting end corresponding to the sample received signal.

[0161] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0162] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the signal processing method provided by the above-mentioned methods, and the method includes: based on the received first received signal, using reinforcement learning to perform a threshold judgment to obtain a threshold signal; inputting the threshold signal into a signal correction model to obtain a second received signal output by the signal correction model; wherein the first received signal is determined based on the original baseband signal sent by the transmitting end; the signal correction model is based on the sample received signal received by the receiving end, and the sample original baseband signal sent by the transmitting end corresponding to the sample received signal is trained.

[0163] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the signal processing method provided by the above-mentioned methods, the method comprising: based on a received first received signal, using reinforcement learning to perform a threshold decision to obtain a threshold signal; inputting the threshold signal into a signal correction model to obtain a second received signal output by the signal correction model; wherein the first received signal is determined based on an original baseband signal sent by a transmitting end; and the signal correction model is trained based on a sample received signal received by a receiving end, and a sample original baseband signal sent by a transmitting end corresponding to the sample received signal.

[0164] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A signal processing method, characterized in that: include: Based on the received first received signal, using reinforcement learning to perform threshold judgment to obtain a threshold signal; Inputting the threshold signal into a signal correction model to obtain a second receiving signal output by the signal correction model; The first received signal is determined according to the original baseband signal sent by the transmitting end; the signal correction model is obtained by training based on the sample received signal received by the receiving end and the sample original baseband signal sent by the transmitting end and corresponding to the sample received signal; The method of performing threshold judgment based on the received first received signal by using reinforcement learning to obtain a threshold signal includes: Subdividing the first received signal into amplitude values, and acquiring state information corresponding to each candidate amplitude value; Based on the reward value corresponding to the next state information obtained after performing amplitude selection on each state information, a target amplitude is selected from the candidate amplitudes; If it is determined that the target amplitude is greater than or equal to the reference value, the judgment result corresponding to the target amplitude is set to 1; If it is determined that the target amplitude is less than the reference value, the judgment result corresponding to the target amplitude is set to 0; The threshold signal is composed of the decision results corresponding to all the target amplitudes; and the reference value is determined according to the first received signal.

2. The signal processing method according to claim 1, characterized in that: The step of selecting a target amplitude from the candidate amplitudes based on the reward value corresponding to the next state information obtained after performing amplitude selection on each state information comprises: Establishing a Q table; wherein the Q table is composed of various states, behaviors in each state, and the estimated reward value obtained by any behavior in any state; After determining the next state information by using the epsilon-greedy strategy, updating the Q table based on the reward value and the estimated reward value corresponding to the state information being transferred to the next state information; Based on the updated Q table, the target amplitude is selected from the candidate amplitudes.

3. The signal processing method according to claim 1, characterized in that: The signal correction model includes an embedding layer, a recurrent neural network layer and a signal conversion layer; Inputting the threshold signal into the embedding layer, and obtaining a feature vector output by the embedding layer corresponding to the threshold signal; Inputting the feature vector into the recurrent neural network layer for time sequence arrangement, and obtaining a signal sequence output by the recurrent neural network layer; The signal sequence is input into the signal conversion layer, and the second receiving signal output by the signal conversion layer is obtained.

4. The signal processing method according to claim 3, characterized in that: The recurrent neural network layer includes a recurrent neural network model; The step of inputting the feature vector into the recurrent neural network layer to obtain a signal sequence output by the recurrent neural network layer includes: Inputting the feature vector into the recurrent neural network model to obtain an intermediate hidden state; Determine the weight of each of the intermediate hidden states according to the attention mechanism; Based on the weight of each of the intermediate hidden states, the signal sequence output by the recurrent neural network model is obtained.

5. The signal processing method according to claim 4, characterized in that: The recurrent neural network model includes one of a long short-term memory network LSTM, a gated recurrent unit GRU, and a bidirectional long short-term memory network BiLSTM.

6. A signal processing device, characterized in that: include: A threshold decision module, used to perform threshold decision based on the received first received signal by using reinforcement learning to obtain a threshold signal; A correction processing module, used for inputting the threshold signal into a signal correction model to obtain a second received signal output by the signal correction model; The first received signal is determined according to the original baseband signal sent by the transmitting end; the signal correction model is obtained by training based on the sample received signal received by the receiving end and the sample original baseband signal sent by the transmitting end and corresponding to the sample received signal; The threshold decision module is specifically used for: Subdividing the first received signal into amplitude values, and acquiring state information corresponding to each candidate amplitude value; Based on the reward value corresponding to the next state information obtained after performing amplitude selection on each state information, a target amplitude is selected from the candidate amplitudes; If it is determined that the target amplitude is greater than or equal to the reference value, the judgment result corresponding to the target amplitude is set to 1; If it is determined that the target amplitude is less than the reference value, the judgment result corresponding to the target amplitude is set to 0; The threshold signal is composed of the decision results corresponding to all the target amplitudes; and the reference value is determined according to the first received signal.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the signal processing method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the signal processing method according to any one of claims 1 to 5 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the signal processing method according to any one of claims 1 to 5 is implemented.

Images