Compressed sensing method and device for sparse signal recovery
Through deep reinforcement learning and neural network training, combined with Markov decision-making process and ε-greedy strategy, training data is automatically generated, which solves the problem of insufficient generalization ability of deep learning-based compression perception methods in dynamic environments, and achieves accurate recovery and efficient adaptation of sparse signals.
Patent Information
- Application Number
- CN202510623901.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2025-08-26
AI Technical Summary
The compression perception method based on deep learning has limited generalization capabilities in dynamic or non-static environments, resulting in poor model adaptability and high cost of relying on high-quality labeled data.
The deep reinforcement learning method is adopted to learn the predicted return value of the state sequence of sparse signal recovery by training neural networks, and combine the Markov decision-making process and ε-greedy strategy to gradually recover the sparse signal, use a multi-layer perceptron to make action decisions, and automatically generate training data through environmental interaction.
It realizes accurate recovery of sparse signals in dynamic environments, improves the adaptability and generalization capabilities of the model, avoids the increase in the computational complexity of high-dimensional data, and does not need to rely on explicit return value tables.
Smart Images
Figure CN120546701A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of signal processing, and in particular to a compressed sensing method for sparse signal recovery. Background Art
[0002] Compressed sensing (CS) is an emerging signal processing method that uses a specific method to restore undersampled signals. It has been widely used in many fields, including the Internet of Things, information security, wireless communications, image processing, pattern recognition, medical imaging, etc.
[0003] With the development of machine learning (ML) technology, the application areas of deep learning (DL) and reinforcement learning (RL) are gradually expanding, providing new development opportunities for CS methods.
[0004] Currently, ML-based compressed sensing methods mainly use DL as the basic framework, such as the learned iterative shrinkage-thresholding algorithm (LISTA) and the learned approximate message passing (LAMP). There is no mature RL-based CS method yet.
[0005] DL-based CS methods usually rely on a large amount of labeled data for supervised training. For example, most DL models rely on a large amount of high-quality labeled data. However, in practical applications, the cost of obtaining high-quality labeled data is high, and the signal distribution in different scenarios may vary greatly, resulting in limited generalization ability of the model. For example, the data-driven training model makes the DL model less adaptable in dynamic or non-static environments and has limited generalization ability. Summary of the Invention
[0006] The present invention provides a compressed sensing method for sparse signal recovery, so as to at least improve the technical defect of limited generalization capability of the DL-based CS method.
[0007] A first aspect of the present invention provides a compressed sensing method for sparse signal recovery, the method comprising:
[0008] The current state sequence for sparse signal recovery is input into the trained first neural network, and the predicted reward value of each predicted action is obtained through the reasoning of the first neural network; wherein, the position information of the state sequence of any state except the last state in the state sequence is used to represent the index value corresponding to the recovered signal of the sparse signal to be recovered, and the state value of the remaining state itself is used to represent whether the recovered signal is a non-zero element or a zero element; the state value of the last state itself is used to represent the predicted action output by the first neural network based on the state values of all the remaining states, and the action is used to represent: selecting the position information with the same state value as the last state in the current state sequence as the index value corresponding to the non-zero element of the recovered signal,
[0009] From each predicted valid action in each predicted action, a predicted action is selected according to a set first selection strategy, wherein the valid action includes actions excluding the selected action,
[0010] According to the selected prediction action, the state value of the state corresponding to the prediction action and the current prediction residual are updated, and the state value of the last state in the current state sequence is updated according to the current prediction residual and the preset first perception matrix to obtain an updated state sequence, wherein the prediction residual is used to represent the error between the observation value of the recovered signal and the observation value of the sparse signal to be recovered.
[0011] The updated state sequence is used as the current state sequence, and the step of inputting the current state sequence for sparse signal recovery into the trained first neural network is returned to, until the number of selected prediction actions reaches a set prediction action number threshold,
[0012] A restoration signal is determined according to each selected prediction action.
[0013] As a possible implementation, the first neural network is trained in the following manner:
[0014] Initializing a signal recovery environment, the signal recovery environment comprising: at least one first sample state sequence for acquiring sample data, and model parameters of a second neural network, wherein the model structure of the second neural network is the same as the model structure of the first neural network,
[0015] Taking a first sample state sequence as the current sample state sequence,
[0016] By interacting with the signal recovery environment, sample data for training the second neural network is obtained, wherein the sample data includes a set number of experience data, each experience data includes a current state, a current selected action, a current reward value, and a next state, and the next state in the experience data is the current state of another experience data.
[0017] The second neural network is trained using the empirical data in the sample data obtained, and the model parameters of the current second neural network in the training process are copied to the model parameters of the current first neural network according to the set interval.
[0018] Determine whether the number of sample actions selected from the sample actions fitted by the second neural network during the training of the second neural network reaches a set sample action number threshold, wherein the sample action number threshold depends on the number of states representing non-zero elements in the current sample state sequence,
[0019] If yes, stop exploring the recovery signal, take another first sample state sequence as the current sample state sequence, and return to the step of acquiring sample data for training the second neural network through environmental interaction until the training is completed.
[0020] Otherwise, return to the step of acquiring sample data for training the second neural network through environmental interaction to continue exploring the restored signal of the current sample sequence.
[0021] After the training is completed, the current first neural network is used as the trained first neural network.
[0022] As a possible implementation method, obtaining sample data for training the second neural network by interacting with the signal recovery environment includes:
[0023] The current sample state sequence is input into the second neural network, and the reward value of each first sample action is obtained through the second neural network. From each first sample valid action in each first sample action, the first sample action is selected according to the set first selection strategy to obtain the currently selected first sample action and its corresponding current state.
[0024] According to the currently selected first sample action, the current state corresponding to the currently selected first sample action and the current sample residual in the current sample state sequence are updated, and the state value of the last state in the current sample state sequence is updated according to the current sample residual and a preset second perception matrix to obtain an updated sample state sequence, wherein the sample residual is used to represent the error between the observation value of the recovered sample signal and the observation value of the sample signal to be recovered.
[0025] The first sample action currently selected is determined as the currently selected action, the state corresponding to the first sample action currently selected in the updated sample state sequence is determined as the current state, the reward corresponding to the currently selected action and the current state is determined as the current reward value, and the last state in the updated sample state sequence is determined as the next state.
[0026] The determined current selected action, current state, current reward value, and next state are used as experience data in the sample data.
[0027] The updated sample state sequence is used as the current sample state sequence, and the step of inputting the current sample state sequence into the second neural network is returned to be executed until the amount of experience data in the sample data reaches a set quantity threshold.
[0028] As a possible implementation, the method of using the empirical data in the acquired sample data to train the second neural network and copying the model parameters of the current second neural network during the training process to the model parameters of the current first neural network at set intervals includes:
[0029] For any sample data,
[0030] Based on each empirical data in the sample data, a second sample state sequence of each empirical data is obtained,
[0031] The second sample state sequence of the empirical data is used as the current second sample sequence,
[0032] Input the current second sample state sequence into the current second neural network, obtain the reward value of each second sample action through the current second neural network, select the second sample action from each second sample valid action in each second sample action according to the set second selection strategy, obtain the currently selected second sample action and its corresponding current state as the sample action of the current state,
[0033] Input the current second sample state sequence into the current first neural network, obtain the reward value of each third sample action through the current first neural network, select the third sample action from each third sample valid action in each third sample action according to the set first selection strategy, obtain the currently selected third sample action and its corresponding current state, and use them as the sample action of the next state.
[0034] Determine the current reward value based on the sample action of the current state and the state value of the last state in the current second sample sequence.
[0035] Calculate the expected reward value of the sample action in the current state based on the current reward value and the reward value of the sample action in the next state.
[0036] Update the model parameters of the current second neural network based on the loss function value between the reward value of the sample action in the current state and the expected reward value.
[0037] When the set interval is reached, the model parameters of the current second neural network are copied to the model parameters of the current first neural network.
[0038] A second sample state sequence of another empirical data is selected as the current sample state sequence, and the step of inputting the current second sample state sequence into the current second neural network is returned to be executed until the number of sample actions selected from the sample actions fitted by the second neural network reaches a sample action number threshold.
[0039] As a possible implementation, the effective action is determined in the following manner:
[0040] According to the state values of the remaining states in the current state sequence, the position information of the state used to characterize the recovered signal is determined.
[0041] According to the determined position information, the selected action is determined.
[0042] Eliminate the selected actions from the actions output by the neural network and determine the remaining actions as valid actions;
[0043] The first selection strategy, the second selection strategy, and the third selection strategy are the same.
[0044] As a possible implementation method, the selection strategy is: selecting the action with the largest reward value;
[0045] The step of obtaining a second sample state sequence of each empirical data based on each empirical data in the sample data includes:
[0046] For each empirical data in the sample data, restore the sample state sequence of the empirical data according to the empirical data, and use the sample state sequence as the second sample state sequence of the empirical data;
[0047] The set interval is determined according to the cumulative number of times the gradient of the model parameters of the second neural network changes during the training process.
[0048] As a possible implementation, determining the current reward value based on the sample action of the current state and the state value of the last state in the current sample sequence includes:
[0049] If the sample action of the current state is the same as the state value of the last state in the current sample sequence, the reward of the sample action of the current state is set to the first reward value.
[0050] If the sample action of the current state is different from the state value of the last state in the current sample sequence, the reward of the sample action of the current state is set to the second reward value.
[0051] in,
[0052] The first reward value is greater than the second reward value.
[0053] As a possible implementation, calculating the expected reward value of the sample action in the current state based on the current reward value and the reward value of the sample action in the next state includes:
[0054] The product of the discount factor and the reward value of the sample action in the next state is added to the current reward value to obtain the expected reward value of the sample action in the current state.
[0055] As a possible implementation, the reward value is a Q value;
[0056] The neural network is a multilayer perceptron, the data dimension of the output layer of the multilayer perceptron is one less than the data dimension of the input layer of the multilayer perceptron, and each action output by the output layer of the multilayer perceptron corresponds one-to-one to the remaining states in the input state sequence;
[0057] The length of the state sequence is the same as the data dimension of the multilayer perceptron input layer;
[0058] The state value of the last state is determined as follows:
[0059] Calculate the current residual based on the remaining states updated in the current state sequence,
[0060] The index value corresponding to the maximum value of the inner product of the current residual and the perception matrix is used as the state value of the last state.
[0061] A second aspect of the present application provides a compressed sensing device for sparse signal recovery, the device comprising:
[0062] a fitting module, configured to input the current state sequence for sparse signal recovery into the trained first neural network, and obtain a predicted reward value for each predicted action through inference of the first neural network;
[0063] The selection module is configured to select a predicted action from each predicted valid action in each predicted action according to a set first selection strategy, wherein the valid action includes actions excluding the selected action.
[0064] An updating module is configured to update the state value of the state corresponding to the predicted action and the current prediction residual according to the selected predicted action, and to update the state value of the last state in the current state sequence according to the current prediction residual and a preset first perception matrix to obtain an updated state sequence, wherein the prediction residual is used to characterize the error between the observation value of the recovered signal and the observation value of the sparse signal to be recovered, and return the updated state sequence as the current state sequence to the fitting module until the number of selected predicted actions reaches a set prediction action number threshold.
[0065] The signal recovery module is used to determine the recovery signal according to each selected prediction action.
[0066] The present application provides a compressed sensing method for sparse signal recovery. The method learns the predicted reward value of the predicted action for each state in the state sequence used for signal recovery through a trained first neural network, avoiding the maintenance of an explicit reward value table and the problem of a sharp increase in computational complexity caused by high-dimensional data, thereby realizing compressed sensing in a deep reinforcement learning manner; by selecting the optimal predicted action to iteratively update the state and prediction residual, the signal is gradually restored, and accurate recovery of the sparse signal is achieved. Through a Markov decision process consisting of three elements: state, action, and reward value, combined with the predicted reward value of the first neural network, the predicted action selection is guided, thereby realizing the gradual recovery of the sparse signal, which is conducive to the accurate recovery of the sparse signal. By using a multi-layer perceptron, an ε-greedy strategy is adopted for action decision-making, and empirical data and the second neural network are introduced to update the first neural network during training, which is conducive to improving the stability of model training and the accuracy of recovery. The present application does not require a priori models and data sets. With its powerful environmental interaction capabilities, it can automatically generate training data, thereby having stronger adaptability to highly dynamic communication environments and possessing strong generalization and self-adaptation capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 A flow chart of a compressed sensing method for sparse signal recovery according to an embodiment of the present application.
[0068] Figure 2a 、 2b A schematic diagram of a process for training a neural network in this embodiment.
[0069] Figure 3 Schematic diagram of the multi-layer perceptron of this embodiment.
[0070] Figure 4 A schematic diagram of the experience data stored in the experience database of this embodiment.
[0071] Figure 5 Schematic diagram of a flow chart of the compressed sensing method for sparse signal recovery according to this embodiment.
[0072] Figure 6 This is a comparison chart of the recovery accuracy of the compressed sensing method of this embodiment and other CS methods under high signal-to-noise ratio conditions with different numbers of non-zero elements k.
[0073] Figure 7 This is a comparison chart of the mean square error between the compressed sensing method of this embodiment and other CS methods under different numbers of non-zero elements k under high signal-to-noise ratio conditions.
[0074] Figure 8A schematic diagram of a compressed sensing device for sparse signal recovery according to an embodiment of the present application.
[0075] Figure 9 Another schematic diagram of a compressed sensing device for sparse signal recovery according to an embodiment of the present application. DETAILED DESCRIPTION
[0076] In order to make the purpose, technical means and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings.
[0077] The applicant discovered that the orthogonal matching pursuit compressed sensing method (OMP) is a classic greedy compressed sensing algorithm that minimizes the residual by continuously selecting a column vector in the sensing matrix as the most suitable atom, wherein the residual is used to characterize the difference between the current recovered signal observation value and the true observation value. When the recovered signal gradually approaches the original signal (i.e., the sparse signal to be recovered), the residual will gradually decrease. Inspired by this, the embodiment of the present application uses the information of the recovered signal and the information of the continuously updated residual during the recovery process to construct a state for characterizing the sparse signal index value, wherein the sparse signal index value corresponds to the sparse signal, for example, the sparse signal index value corresponds one-to-one to each element in the sparse signal matrix, so that the state can characterize the recovery state of any step in OMP and the state of the next step, so that the CS problem can be solved step by step through the state transfer in RL.
[0078] See also Figure 1 As shown, Figure 1 This is a flow chart of a compressed sensing method for sparse signal recovery according to an embodiment of the present application. The method includes:
[0079] Step 101: Input the current state sequence for sparse signal recovery into the trained first neural network, and obtain the predicted reward value of each predicted action through the reasoning of the first neural network;
[0080] in,
[0081] The position information of any state except the last state in the state sequence is used to represent the index value corresponding to the recovered signal of the sparse signal to be recovered, and the state value of the remaining state itself is used to represent whether the recovered signal is a non-zero element or a zero element;
[0082] The state value of the last state in the state sequence is used to represent the predicted action output by the first neural network based on the state values of all the remaining states.
[0083] The action is used to characterize: select the position information in the current state sequence that has the same state value as the last state as the index value corresponding to the non-zero element of the recovered signal,
[0084] The state value of the last state in the state sequence is: the index value corresponding to the maximum inner product of the current prediction residual calculated according to the remaining states updated in the current state sequence and the preset first perception matrix;
[0085] As an example, the state sequence input for the first time is a preset initialization state sequence, and the state value of the last state in the initialization state sequence is: the index value corresponding to the maximum value of the inner product of the initial residual and the first perception matrix, wherein the initial residual is the observation value of the sparse signal to be restored.
[0086] As an example, the first neural network is trained as follows:
[0087] Initializing a signal recovery environment, the signal recovery environment comprising: at least one first sample state sequence, and model parameters of a second neural network, wherein the model structure of the second neural network is the same as the model structure of the first neural network,
[0088] Taking a first sample state sequence as the current sample state sequence,
[0089] By interacting with the signal recovery environment, sample data for training the second neural network is obtained, wherein the sample data includes a set number of experience data, each experience data includes a current state, a current selected action, a current reward value, and a next state, and the next state in the experience data is the current state of another experience data.
[0090] The second neural network is trained using the empirical data in the sample data obtained to update the model parameters of the second neural network, and the model parameters of the current second neural network in the training process are copied to the model parameters of the current first neural network at set intervals to update the model parameters of the first neural network.
[0091] Determine whether the number of sample actions selected from the sample actions fitted by the second neural network during the training of the second neural network reaches a set sample action number threshold, wherein the sample action number threshold depends on the number of states representing non-zero elements in the current sample state sequence,
[0092] If yes, stop exploring the recovery signal, take another first sample state sequence as the current sample state sequence, and return to the step of acquiring sample data for training the second neural network through environmental interaction until the training is completed.
[0093] Otherwise, returning to the step of acquiring sample data for training the second neural network by interacting with the signal recovery environment to explore the recovery signal,
[0094] After the training is completed, the current first neural network is used as the trained first neural network.
[0095] Step 102: Select a predicted action from each predicted valid action according to a set first selection strategy, wherein the valid action includes actions other than the selected action.
[0096] As an example, valid actions are determined as follows:
[0097] According to the state values of the remaining states in the current state sequence, the position information of the state used to characterize the recovered signal is determined.
[0098] According to the determined position information, the selected action is determined.
[0099] The selected actions among the predicted actions output by the first neural network are excluded, and the remaining predicted actions are determined as valid actions.
[0100] Step 103: Update the state value of the state corresponding to the predicted action and the current prediction residual according to the selected predicted action, and update the state value of the last state in the current state sequence according to the current prediction residual and the preset first perception matrix to obtain an updated state sequence.
[0101] Among them, the prediction residual is used to characterize the error between the observation value of the restored signal and the observation value of the sparse signal to be restored.
[0102] Step 104: Use the updated state sequence as the current state sequence and return to step 102 until the number of selected predicted actions reaches the set predicted action number threshold.
[0103] Step 105: Determine a restoration signal according to each selected prediction action.
[0104] As an example, according to the state corresponding to each selected prediction action, the index value corresponding to the non-element in the sparse signal to be restored is determined, and the signal corresponding to the index value is determined as a non-zero element.
[0105] The embodiment of the present application gradually recovers the signal by fitting the predicted reward value of the current state sequence for sparse signal recovery through the first neural network, thereby improving the adaptability and decision-making ability of signal recovery in a dynamic environment.
[0106] To facilitate understanding of the embodiments of the present application, the following description will be given using a multilayer perceptron as a neural network and a Q value as an example of a reward value. It should be understood that the neural network and reward value of the present application are not limited thereto.
[0107] The parameters required for this embodiment are as follows.
[0108] 1. State Space
[0109] In this embodiment, the end-to-end relationship between the state and the Q value is learned through a neural network, so the state needs to be redesigned to store more information about the interaction with the signal recovery environment during the recovery process.
[0110] As an example, the state used to represent the recovered signal is set to a state sequence of length N+1, and the positions of the states of the first N bits (i.e., the remaining bits except the last bit) in the state sequence represent the index values corresponding to the recovered signals, and the state of the N+1th bit (i.e., the last bit) is used to represent the index value of the next recovered signal.
[0111] The state value of any state in the first N-bit sequence represents whether the recovered signal corresponding to the state is a zero signal or a non-zero signal. For example, when the recovered signal has a zero element, the state value is 0, and when the recovered signal has a non-zero element, the state value is 1.
[0112] The state value of the N+1th state itself represents the index value of the maximum value of the inner product of the current residual and the perception matrix, in the hope that after the neural network converges, it can make the same action as the state value of the last state in the state sequence based on the first N bits of information.
[0113] The mathematical formula is:
[0114] S=[s1,s2,...,s N ,|s N+1 ] N×1 ,in
[0115] s i Represents the i-th state of the state sequence S, and N is a natural number.
[0116] For example, the i-th state s in the state sequence i Indicates the recovered signal corresponding to the index value i. When s i = 0, indicating that the recovered signal corresponding to the index value i is a zero element. i =1, indicating that the recovered signal corresponding to the index value i is a non-zero element, and the N+1th bit is the index value corresponding to the maximum value of the inner product of the current residual and the perception matrix.
[0117] Therefore, in the state sequence S, for any state of the first N bits in the sequence, the position information of the state in the state sequence is used to represent the index information of the non-zero signal in the recovered signal, and the state value of the state itself is used to represent whether the recovered signal is a non-zero signal or a zero signal. For the state of the N+1th bit in the sequence, the state value of the state itself is used to represent the action.
[0118] For example, assuming N=10, S=01110011006, it means that the recovered signals corresponding to the 2nd, 3rd, 4th, 7th, and 8th bit states are non-zero signals, and the 11th bit state 6 indicates the action of exploring the index value 6 corresponding to the next recovered signal.
[0119] In this embodiment, since the inner product information of the current residual and the perception matrix is embedded in the state sequence, the neural network allows new states to participate in the update while storing the old state mapping relationship. Therefore, the state dimension can be designed to be larger to store more features.
[0120] 2. Action Space
[0121] In this embodiment, the action space is consistent with the RL-based CS algorithm. It can be expressed mathematically as:
[0122] A={0,1,...,N-1}
[0123] Among them, any action a in the action space A i Indicates that the i-th state in the state sequence is selected as the index value corresponding to the non-zero element.
[0124] 3. Reward Function
[0125] In this embodiment, the reward function can be simplified to a binary function. If the currently selected action is consistent with the last bit in the state sequence, the current reward value is the set first reward value. Otherwise, the current reward value is the set second reward value. The first reward value is greater than the second reward value. For example, the first reward value is 100 and the second reward value is -100, and the two are opposite numbers. It can be expressed mathematically as follows:
[0126]
[0127] Among them, Reward(s,a) represents the reward value of selecting action a when the current state is s.
[0128] Compared with the method of using the inner product of the current residual and the atom as the reward value, since the Q table updated in real time in this method limits the state dimension to not be too large, it must be processed in real time regardless of whether the state is truly useful. The binary function used in this embodiment is not limited by the state dimension.
[0129] See also Figure 2a 、2b As shown, Figure 2a 、 2b This is a flow chart of training a neural network according to this embodiment. The training process includes:
[0130] Step 201, initializing the signal recovery environment, includes:
[0131] 1) initializing at least one first sample state sequence for acquiring sample data,
[0132] As an example, the first sample state sequence is a numerical sequence having k non-zero elements. Each first sample state sequence is diverse, that is, the positions and numerical values of the non-zero elements included in different first sample state sequences are different, so as to adapt to the recovery process of different signals and avoid overfitting.
[0133] As an example, the state values of the first N bits are initially all zero, and the state value of the N+1th bit state is set to the index value corresponding to the maximum value of the inner product of the initial observation value of the first sample state sequence and the perception matrix, where, to distinguish it from the perception matrix used in the inference process, the aforementioned perception matrix is the second perception matrix.
[0134] 2) Initialize the neural network,
[0135] As an example, the neural network uses a multilayer perceptron (MLP), see Figure 3 As shown, Figure 3 A schematic diagram of a multi-layer perceptron, which includes an input layer, two hidden layers, and an output layer.
[0136] The data dimension of the input layer is the same as the length of the state sequence, which is N+1. Each dimension corresponds to a state in the state sequence. For example, x1 corresponds to state s1 in the state sequence, x2 corresponds to state s2 in the state sequence... and so on. N+1 Corresponding to state s in the state sequence N+1 ;
[0137] The data dimension of each hidden layer is 128, and each neuron introduces nonlinearity through the ReLU activation function, enabling the network to fit complex mapping relationships;
[0138] The output layer data dimension is N, and each dimension corresponds to the Q value of an action. For example, o1 corresponds to the Q value of action a1, o2 corresponds to the Q value of action a2, and so on. N Corresponding action a N Q value. Given action a i Indicates that the i-th state in the state sequence is selected as the index value corresponding to the non-zero element of the recovered signal. Therefore, any output result o of the output layeri Represents the neural network state s i The fitted action a i Q value.
[0139] In this way, the neural network can be used to fit the Q values of N actions when the current state sequence N+1 is input, so that for a state sequence of length N+1, there are N actions to choose from.
[0140] Step 202: interact with the signal recovery environment to obtain sample data for training the neural network.
[0141] As an example, this includes:
[0142] Step 2020: taking a first sample state sequence as the current sample state sequence.
[0143] Step 2021 , screening is performed according to the current state sequence to obtain the current first sample valid action to avoid repeated action selection.
[0144] Since the first N bits of the current state sequence record the selected actions, the current valid action can be determined based on the first N bits of the state sequence. That is, when any state value in the first N bits of the state sequence is updated, indicating that the state position is the selected action, the selected action is removed, and the first N bits of the state sequence are traversed, removing the selected actions one by one, thereby obtaining a set including the current valid actions.
[0145] Step 2022: Input the current state sequence into the current neural network, and use the current neural network to fit the corresponding Q values of the N first sample actions.
[0146] For the sake of convenience, the current neural network is called the second neural network to distinguish it from the first neural network used for inference.
[0147] In step 2023, according to the ε-greedy principle, a first sample action is selected from the current first sample valid actions, that is, the action corresponding to the maximum Q value is selected from the current valid actions as the currently selected first sample action and its corresponding current state.
[0148] Step 2024: Calculate the current reward value according to the reward function based on the first sample action currently selected and the last state value of the current state sequence.
[0149] Step 2025: Update the state value of the state corresponding to the currently selected action according to the currently selected first sample action, so that the first N bits of the current state sequence record the selected first sample action;
[0150] Step 2026: determine the index value of the recovered signal based on the position of the current state in the state sequence, determine the recovered signal observation value based on the index value of the recovered signal, update the current sample residual based on the error between the recovered signal observation value and the observation value of the sample signal to be recovered, and use the index value corresponding to the maximum value of the inner product of the current sample residual and the second perception matrix to update the state value of the last state in the current state sequence as the next state.
[0151] Through steps 2025 and 2026, the current state sequence is updated to obtain an updated state sequence.
[0152] In step 2027, the current state obtained in step 2023, the first sample action currently selected (i.e., the currently selected action), the current reward value calculated in step 2024, and the next state obtained in step 2026 are stored as experience data in the experience database.
[0153] In step 2028, the updated state sequence is used as the current state sequence, and the process returns to step 2021 to continue exploring the current state sequence or another first sample state sequence until the amount of experience data reaches a set threshold.
[0154] For example, assuming that N is 10, the first sample state sequence S is initialized to 00000000004.
[0155] During the first signal recovery environment interaction process, the initialized first sample state sequence is input into the second neural network to obtain 10 selectable actions. The action corresponding to the maximum Q value is selected according to the ε-greedy principle. Assuming that the current first sample action a1 is selected and the current reward value r1, the state value 0 of the state s1 is updated to the state value 1 according to the first sample action a1, and the recovered signal observation value is determined according to the index value 1 of the recovered signal, and the current sample residual is updated according to the recovered signal observation value and the observation value of the sample signal to be recovered. The index value corresponding to the maximum value of the inner product of the current sample residual and the second perception matrix is used to update the state value 4 of the last state of the current state sequence. Assuming that the updated state value of the last state is 2, the current state s1, the current first sample action a1, the current reward value r1, and the next state s2 are used as experience data. The updated first sample state sequence S=10000000002 is used as the current state sequence;
[0156] In the second environment interaction process, the current state sequence S = 10000000002 is input into the second neural network, and 10 actions are obtained for selection. According to the current state sequence S = 10000000002, it can be determined that there are 9 first sample valid actions in addition to the first sample action a1 that has been selected. According to the ε-greedy principle, the action corresponding to the maximum Q value among the first sample valid actions is selected. Assuming that the current selected first sample action is action a2 and the current reward value is r2, the state value of state s2 is updated from 0 to state value 1 according to the first sample action a2. According to the index value 2 of the restored signal, the restored signal observation value is determined, and the current sample residual is updated according to the restored signal observation value and the sample signal observation value to be restored. The index value corresponding to the maximum inner product of the current sample residual and the second perception matrix is used to update the state value 2 of the last state of the current state sequence. Assuming that the updated state value of the last state is 3, the current state s2, the current first sample action a2, the current reward value r2, and the next state s3 are used as experience data. The updated first sample state sequence S=11000000003 is used as the current state sequence;
[0157] …
[0158] Similarly, any piece of experience data t, which includes the current state s t 、Currently selected action a t , current reward value r t , next state s t+1 , where t is a natural number less than or equal to N+1.
[0159] See also Figure 4 As shown, Figure 4 This is a schematic diagram of the experience data stored in the experience database of this embodiment. Various experience data can be obtained through signal recovery environment interaction, wherein each experience data includes: current state, current selected action, current reward value, and next state. The next state in the experience data is the current state of another experience data.
[0160] Step 203: Use the experience data as sample data to train the second neural network.
[0161] As an example, this includes:
[0162] Step 2031: Select a batch of experience data from the experience database as sample data. The batch of experience data includes no less than a set number of experience data.
[0163] Step 2032: for any selected sample data, restore the second sample state sequence of each empirical data according to each empirical data in the sample data, and use the second sample state sequence of the empirical data as the current second sample sequence.
[0164] Step 2033: Input the current second sample state sequence into the current second neural network, obtain the Q value of each second sample action through the current second neural network, and select the second sample action from each second sample valid action in the second sample actions according to the set second selection strategy. For example, select the second sample valid action with the largest Q value, obtain the currently selected second sample action and its corresponding current state, and use them as the sample action of the current state.
[0165] In step 2034, the current second sample state sequence is input into the first neural network, and the Q value of each third sample action is obtained through the current first neural network. From each third sample valid action in each third sample action, the third sample action is selected according to the set first selection strategy. For example, the third sample valid action with the largest Q value is selected to obtain the currently selected third sample action and its corresponding current state as the sample action of the next state.
[0166] The first neural network is the current second neural network updated according to a set interval, and the model structure of the first neural network is the same as that of the second neural network.
[0167] Step 2035: Calculate the current reward value based on the sample action of the current state and the state value of the last state in the current second sample sequence.
[0168] Step 2036: Calculate the expected Q value of the sample action in the current state based on the current reward value and the Q value of the sample action in the next state.
[0169] As an example, the expected Q value is calculated as follows:
[0170] Q target (s,a)=r+γmax a′ Q(s′,a′)
[0171] Among them, Q target (s,a) is the expected Q value of the sample action a in the current state s, r is the current reward value of the sample action a in the current state s, γ is the discount factor, max a′ Q(s′,a′) is the maximum Q value among all actions in the next state s′. The sample action corresponding to the maximum Q value is action a′.
[0172] Step 2037: Calculate the loss function value between the Q value of the sample action in the current state and its expected Q value based on the Q value and expected Q value of the sample action in the current state.
[0173] As an example, the loss function value is calculated as follows:
[0174]
[0175] Among them, Q(s,a) is the Q value of the sample action a in the current state s.
[0176] In step 2038, based on the loss function value, back propagation is used to calculate the model parameter gradient of the current second neural network, and the model parameters of the current second neural network are updated according to the model parameter gradient.
[0177] Step 2039: Copy the model parameters of the current second neural network to the model parameters of the first neural network at the set interval to update the model parameters of the current first neural network.
[0178] As an example, the set interval can be set according to the cumulative number of times the gradient of the model parameters of the current neural network is descended. For example, when the cumulative number of times the gradient of the current neural network parameters is descended reaches a set threshold, the model parameters of the current second neural network are copied to the first neural network.
[0179] Step 204: determine whether the number of sample actions selected from the sample actions fitted by the second neural network during the training of the second neural network reaches a set sample action number threshold.
[0180] If yes, stop exploring the restored signal of the current second sample sequence, select another second sample sequence as the current sample sequence, and return to step 2033 to explore the other second sample state sequence.
[0181] Otherwise, return to step 2033 to continue exploring the restored signal of the current sample sequence;
[0182] The sample action quantity threshold is determined according to the number of non-zero elements included in the first sample state sequence, for example, the value is k.
[0183] If the training is completed, the current first neural network is used as the first neural network after training.
[0184] Otherwise, another first sample sequence is selected as the current sample sequence, and the process returns to step 2021 to obtain empirical data of the other first sample sequence.
[0185] See also Figure 5 As shown, Figure 5: This is a flow chart of a compressed sensing method for sparse signal recovery according to this embodiment. The method includes:
[0186] Step 501: Initialize various parameters.
[0187] Initialize the first perception matrix Θ, the initial observation value, initial residual, and initial state sequence corresponding to the sparse signal to be restored,
[0188] in,
[0189] Initial observation value y = Θq + n, where q is the sparse signal matrix to be recovered and n is the white noise matrix.
[0190] Initial residual residual=y,
[0191] In the initial state sequence, the state values of the first N bits are all zero, and the state value of the N+1th bit state is the index value of the maximum inner product of the initial residual and the first perception matrix;
[0192] Take the initial state sequence as the current state sequence,
[0193] Step 502: Input the current state sequence into the trained first neural network, and obtain the predicted Q values of all predicted actions through the reasoning of the first neural network.
[0194] As an example, based on the pytorch framework, after completing the training, the weights and biases of the first neural network are saved as a binary file of type .pth. During inference, a new MLP model is created, and then the .pth file is imported to restore all weights and biases to restore the trained model.
[0195] Step 503: Remove the selected action from the predicted Q values of all predicted actions, select the predicted action with the current maximum predicted Q value from the predicted Q values of the remaining predicted actions, store the predicted action in the support set, update the state value of the state corresponding to the predicted action in the current state sequence, and the current prediction residual, and use the index value corresponding to the maximum inner product of the current prediction residual and the first perception matrix to update the state value of the last state in the current state sequence.
[0196] Among them, the prediction residual is used to characterize the error between the observation value of the restored signal and the observation value of the sparse signal to be restored.
[0197] Step 504: determine whether the number of predicted actions in the support set reaches the set predicted action number threshold.
[0198] If so, determine the corresponding state of each predicted action according to the predicted action in the current support set, determine each index value according to the position of each state in the state sequence, and determine the non-zero element corresponding to each index value according to each index value to obtain the recovery signal of the sparse signal to be recovered.
[0199] Otherwise, the process returns to step 502 to determine the next state, so as to determine the next non-zero element.
[0200] This embodiment no longer requires a parallel computing framework to address the "curse of dimensionality" problem. Instead, it uses a first neural network instead of the Q-table in Q-learning, thus avoiding the need to maintain an explicit Q-table and the resulting dramatic increase in computational complexity associated with high-dimensional data. Experimental comparisons of the differences in recovery accuracy demonstrate higher recovery accuracy for highly sparse signals. The results demonstrate that in medium and high signal-to-noise ratio environments, this embodiment generally outperforms traditional methods in recovering highly sparse signals, while performing comparably well on low-sparse signal recovery tasks.
[0201] See also Figure 6 、 Figure 7 As shown, Figure 6 This is a comparison chart of the recovery accuracy of the compressed sensing method of this embodiment and other CS methods under high signal-to-noise ratio conditions with different numbers of non-zero elements k; Figure 7 The figure shows a comparison of the mean square error between the compressed sensing method of this embodiment and other CS methods under high signal-to-noise ratio conditions under different numbers of non-zero elements k. In the figure, DQSR (Deep Q-networkbased algorithm of support recovery for sparse signals) is a sparse signal support set recovery method based on a deep Q network (DQN), that is, the compressed sensing method of this embodiment, OMP (Orthogonal Matching Pursuit, OMP) is an orthogonal matching pursuit compressed sensing method, FCB (Federated Consensus-Based Algorithm for Stable Recovery of Sparse Signals) is a sparse signal stable recovery algorithm based on federated consensus, StOMP (Stagewise OMP) is a segmented orthogonal matching pursuit compressed sensing method, SP (Subspace Pursuit) is a subspace pursuit compressed sensing method, and CoSaMP (Compressive Sampling Matching Pursuit) is a compressed sampling matching pursuit compressed sensing method. Figure 6 、 7 In the equation, the noise follows a normal distribution. where δ ω =2 -6
[0202] This embodiment freely explores different signals during the training process, using a first sample state sequence of length N = 16. The value of the non-zero element in the signal is limited to 1, but the position is random to ensure that there is no repetition. The number of observations M = 10. The first perception matrix is a random Gaussian matrix, the learning rate is 0.015, the discount factor is 0.9, and 30,000 first sample state sequences are explored in each round. After 200 rounds of exploration, the neural network basically converges. During the inference process, Figure 6 、 7 50,000 signals are randomly selected for testing at each sparsity.
[0203] Depend on Figure 6 It can be seen that under high signal-to-noise ratio conditions, the CS method based on this embodiment has a slightly better recovery accuracy than other CS methods for high-sparse signals, such as k = 1, 2, 3, 4, with an average improvement of 2.7%-11.8%; for low-sparse signals, such as k = 5, 6, 7, 8, its recovery accuracy is comparable to that of other CS methods.
[0204] contrast Figure 6 and Figure 7 Under high signal-to-noise ratio conditions, the mean square error of the compressed sensing method of this embodiment is roughly consistent with the conclusion of recovery accuracy. When the sparsity is high, the mean square error of this embodiment is slightly lower than that of other CS methods. As the sparsity decreases, the mean square error of this embodiment gradually becomes equal to that of other CS methods.
[0205] See also Figure 8 As shown, Figure 8 This is a schematic diagram of a compressed sensing device for sparse signal recovery according to an embodiment of the present application. The device includes:
[0206] a fitting module, configured to input the current state sequence for sparse signal recovery into the trained first neural network, and obtain a predicted reward value for each predicted action through inference of the first neural network;
[0207] The selection module is configured to select a predicted action from each predicted valid action in each predicted action according to a set first selection strategy, wherein the valid action includes actions excluding the selected action.
[0208] An updating module is configured to update the state value of the state corresponding to the predicted action and the current prediction residual according to the selected predicted action, and to update the state value of the last state in the current state sequence according to the current prediction residual and a preset first perception matrix to obtain an updated state sequence, wherein the prediction residual is used to characterize the error between the observation value of the recovered signal and the observation value of the sparse signal to be recovered, and return the updated state sequence as the current state sequence to the fitting module until the number of selected predicted actions reaches a set prediction action number threshold.
[0209] The signal recovery module is used to determine the recovery signal according to each selected prediction action.
[0210] See also Figure 9 As shown, Figure 9 Another schematic diagram of a compressed sensing device for sparse signal recovery according to an embodiment of the present application. The device includes a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of the compressed sensing method for sparse signal recovery according to an embodiment of the present application.
[0211] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.
[0212] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
[0213] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the compressed sensing method for sparse signal recovery in the embodiment of the present application are implemented.
[0214] As for the apparatus / network-side device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0215] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
[0216] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A compressed sensing method for sparse signal recovery, characterized in that: The method includes: The current state sequence for sparse signal recovery is input into the trained first neural network, and the predicted reward value of each predicted action is obtained through the reasoning of the first neural network; wherein, the position information of the state sequence of any state except the last state in the state sequence is used to represent the index value corresponding to the recovered signal of the sparse signal to be recovered, and the state value of the remaining state itself is used to represent whether the recovered signal is a non-zero element or a zero element; the state value of the last state itself is used to represent the predicted action output by the first neural network based on the state values of all the remaining states, and the action is used to represent: selecting the position information with the same state value as the last state in the current state sequence as the index value corresponding to the non-zero element of the recovered signal, From each predicted valid action in each predicted action, a predicted action is selected according to a set first selection strategy, wherein the valid action includes actions excluding the selected action, According to the selected prediction action, the state value of the state corresponding to the prediction action and the current prediction residual are updated, and the state value of the last state in the current state sequence is updated according to the current prediction residual and the preset first perception matrix to obtain an updated state sequence, wherein the prediction residual is used to represent the error between the observation value of the recovered signal and the observation value of the sparse signal to be recovered. The updated state sequence is used as the current state sequence, and the step of inputting the current state sequence for sparse signal recovery into the trained first neural network is returned to, until the number of selected prediction actions reaches a set prediction action number threshold, A restoration signal is determined according to each selected prediction action.
2. The compressed sensing method according to claim 1, wherein: The first neural network is trained in the following manner: Initializing a signal recovery environment, the signal recovery environment comprising: at least one first sample state sequence for acquiring sample data, and model parameters of a second neural network, wherein the model structure of the second neural network is the same as the model structure of the first neural network, Taking a first sample state sequence as the current sample state sequence, By interacting with the signal recovery environment, sample data for training the second neural network is obtained, wherein the sample data includes a set number of experience data, each experience data includes a current state, a current selected action, a current reward value, and a next state, and the next state in the experience data is the current state of another experience data. The second neural network is trained using the empirical data in the sample data obtained, and the model parameters of the current second neural network in the training process are copied to the model parameters of the current first neural network according to the set interval. Determine whether the number of sample actions selected from the sample actions fitted by the second neural network during the training of the second neural network reaches a set sample action number threshold, wherein the sample action number threshold depends on the number of states representing non-zero elements in the current sample state sequence, If yes, stop exploring the recovery signal, take another first sample state sequence as the current sample state sequence, and return to the step of acquiring sample data for training the second neural network through environmental interaction until the training is completed. Otherwise, return to the step of acquiring sample data for training the second neural network through environmental interaction to continue exploring the restored signal of the current sample sequence. After the training is completed, the current first neural network is used as the trained first neural network.
3. The compressed sensing method according to claim 2, wherein: The acquiring of sample data for training the second neural network through interaction with the signal recovery environment includes: The current sample state sequence is input into the second neural network, and the reward value of each first sample action is obtained through the second neural network. From each first sample valid action in each first sample action, the first sample action is selected according to the set first selection strategy to obtain the currently selected first sample action and its corresponding current state. According to the currently selected first sample action, the current state corresponding to the currently selected first sample action and the current sample residual in the current sample state sequence are updated, and the state value of the last state in the current sample state sequence is updated according to the current sample residual and a preset second perception matrix to obtain an updated sample state sequence, wherein the sample residual is used to represent the error between the observation value of the recovered sample signal and the observation value of the sample signal to be recovered. The first sample action currently selected is determined as the currently selected action, the state corresponding to the first sample action currently selected in the updated sample state sequence is determined as the current state, the reward corresponding to the currently selected action and the current state is determined as the current reward value, and the last state in the updated sample state sequence is determined as the next state. The determined current selected action, current state, current reward value, and next state are used as experience data in the sample data. The updated sample state sequence is used as the current sample state sequence, and the step of inputting the current sample state sequence into the second neural network is returned to be executed until the amount of experience data in the sample data reaches a set quantity threshold.
4. The compressed sensing method according to claim 3, wherein: The method of training the second neural network by using the empirical data in the acquired sample data and copying the model parameters of the current second neural network during the training process to the model parameters of the current first neural network according to the set interval includes: For any sample data, Based on each empirical data in the sample data, a second sample state sequence of each empirical data is obtained, The second sample state sequence of the empirical data is used as the current second sample sequence, Input the current second sample state sequence into the current second neural network, obtain the reward value of each second sample action through the current second neural network, select the second sample action from each second sample valid action in each second sample action according to the set second selection strategy, obtain the currently selected second sample action and its corresponding current state as the sample action of the current state, Input the current second sample state sequence into the current first neural network, obtain the reward value of each third sample action through the current first neural network, select the third sample action from each third sample valid action in each third sample action according to the set first selection strategy, obtain the currently selected third sample action and its corresponding current state, and use them as the sample action of the next state. Determine the current reward value based on the sample action of the current state and the state value of the last state in the current second sample sequence. Calculate the expected reward value of the sample action in the current state based on the current reward value and the reward value of the sample action in the next state. Update the model parameters of the current second neural network based on the loss function value between the reward value of the sample action in the current state and the expected reward value. When the set interval is reached, the model parameters of the current second neural network are copied to the model parameters of the current first neural network. A second sample state sequence of another empirical data is selected as the current sample state sequence, and the step of inputting the current second sample state sequence into the current second neural network is returned to be executed until the number of sample actions selected from the sample actions fitted by the second neural network reaches a sample action number threshold.
5. The compressed sensing method according to claim 4, wherein: The valid actions are determined as follows: According to the state values of the remaining states in the current state sequence, the position information of the state used to characterize the recovered signal is determined. According to the determined position information, the selected action is determined. Eliminate the selected actions from the actions output by the neural network and determine the remaining actions as valid actions; The first selection strategy, the second selection strategy, and the third selection strategy are the same.
6. The compressed sensing method according to claim 5, wherein: The selection strategy is: select the action with the largest reward value; The step of obtaining a second sample state sequence of each empirical data based on each empirical data in the sample data includes: For each empirical data in the sample data, restore the sample state sequence of the empirical data according to the empirical data, and use the sample state sequence as the second sample state sequence of the empirical data; The set interval is determined according to the cumulative number of times the gradient of the model parameters of the second neural network changes during the training process.
7. The compressed sensing method according to claim 6, wherein: The method of determining the current reward value based on the sample action of the current state and the state value of the last state in the current sample sequence includes: If the sample action of the current state is the same as the state value of the last state in the current sample sequence, the reward of the sample action of the current state is set to the first reward value. If the sample action of the current state is different from the state value of the last state in the current sample sequence, the reward of the sample action of the current state is set to the second reward value. in, The first reward value is greater than the second reward value.
8. The compressed sensing method according to claim 7, wherein: The step of calculating the expected reward value of the sample action in the current state based on the current reward value and the reward value of the sample action in the next state includes: The product of the discount factor and the reward value of the sample action in the next state is added to the current reward value to obtain the expected reward value of the sample action in the current state.
9. The compressed sensing method according to claim 6, wherein: The return value is Q value; The neural network is a multilayer perceptron, the data dimension of the output layer of the multilayer perceptron is one less than the data dimension of the input layer of the multilayer perceptron, and each action output by the output layer of the multilayer perceptron corresponds one-to-one to the remaining states in the input state sequence; The length of the state sequence is the same as the data dimension of the multilayer perceptron input layer; The state value of the last state is determined as follows: Calculate the current residual based on the remaining states updated in the current state sequence, The index value corresponding to the maximum value of the inner product of the current residual and the perception matrix is used as the state value of the last state.
10. A compressed sensing device for sparse signal recovery, characterized in that: The device includes: a fitting module, configured to input the current state sequence for sparse signal recovery into the trained first neural network, and obtain a predicted reward value for each predicted action through inference of the first neural network; The selection module is configured to select a predicted action from each predicted valid action in each predicted action according to a set first selection strategy, wherein the valid action includes actions excluding the selected action. An updating module is configured to update the state value of the state corresponding to the predicted action and the current prediction residual according to the selected predicted action, and to update the state value of the last state in the current state sequence according to the current prediction residual and a preset first perception matrix to obtain an updated state sequence, wherein the prediction residual is used to characterize the error between the observation value of the recovered signal and the observation value of the sparse signal to be recovered, and return the updated state sequence as the current state sequence to the fitting module until the number of selected predicted actions reaches a set prediction action number threshold. The signal recovery module is used to determine the recovery signal according to each selected prediction action.