Interactive intelligent spectrum and sample collection method and system

Abstract

The invention provides an interactive intelligent spectrum and sample acquisition method and system, relates to the technical field of signal sample acquisition and artificial intelligence, and solves the limitation problems of low channel selection efficiency and insufficient intelligent technology application capability in the existing sensor signal acquisition process. The method comprises the steps that original spectrum data are acquired and preprocessed, and an acquisition channel selection model is established in a reinforcement learning mode and has multiple reinforcement learning modes; respectively optimizing a channel selection strategy corresponding to each reinforcement learning mode in model training; and inputting the spectrum data into an acquisition channel selection model, making a channel selection action and outputting a channel selection result by the model based on an optimized channel selection strategy, and synchronously presenting the spectrum data and the channel selection result to the outside in a visual mode. According to the invention, the acquisition strategy can be dynamically adjusted according to the real-time spectrum state, the acquisition process of the signal sample is optimized, and the acquisition efficiency is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of signal sample acquisition and artificial intelligence, and particularly relates to an interactive intelligent spectrum and sample acquisition method and system. Background Art

[0002] In the field of modern electromagnetic environment monitoring, low-cost sensors, as the sensing terminals of information systems, are widely used in various information systems for signal sample acquisition. These sensors provide a basis for the subsequent data aggregation and processing by collecting a large number of signal samples, thus creating conditions for realizing intelligent recognition and comprehensive situation generation. However, due to hardware resource limitations and relatively inefficient autonomous operation modes, these sensors often perform poorly in terms of signal sample acquisition efficiency and response speed, and are difficult to meet the increasingly complex electromagnetic environment monitoring requirements.

[0003] In existing technical solutions, sensors usually adopt fixed acquisition strategies or simple polling mechanisms for signal acquisition. Although this mode is simple to implement, in the face of complex and changeable electromagnetic environments, it often fails to effectively identify and utilize spectrum resources, resulting in low acquisition efficiency. In addition, due to the lack of an intelligent decision-making mechanism, sensors have a slow response speed when facing sudden signals or spectrum changes and are difficult to capture key information in real time. These problems not only affect the quality of monitoring tasks but also exacerbate the contradiction between electromagnetic spectrum monitoring tasks and personnel, increasing the frequency of manual intervention and adjustment.

[0004] With the increasing complexity of the electromagnetic environment and the increasing tension of spectrum resources, traditional signal acquisition methods are difficult to meet the requirements of modern monitoring systems. Especially in wideband scanning and multi-channel monitoring scenarios, how to efficiently select acquisition channels and optimize the signal sample acquisition process has become a technical problem to be solved urgently. Most existing technical solutions rely on predefined rules or empirical parameters and lack adaptability, unable to dynamically adjust the acquisition strategy according to real-time environmental changes, resulting in low utilization efficiency of spectrum resources and limited execution effects of monitoring tasks.

[0005] Reinforcement learning, as a machine learning method based on environmental interaction, has been widely used in the fields of intelligent decision-making and optimal control in recent years. Its core idea is to continuously interact between the intelligent agent and the environment to learn the optimal strategy to maximize the long-term cumulative reward. In the field of signal acquisition, reinforcement learning can improve the acquisition efficiency and response speed by learning channel selection strategies and dynamically adjusting the acquisition behavior of sensors. However, existing signal acquisition technologies have not fully utilized the advantages of reinforcement learning and lack an intelligent acquisition model for electromagnetic environment monitoring scenarios, resulting in unsatisfactory performance of sensors in complex environments.

[0006] The signal acquisition methods in the prior art thus have obvious deficiencies in terms of efficiency, response speed, and adaptability, and it is difficult to meet the requirements of modern electromagnetic environment monitoring. Especially in the scenarios of wideband scanning and multi-channel monitoring, how to achieve intelligent signal acquisition and optimize the utilization of spectrum resources has become an important direction for the current technological development. By introducing intelligent technologies such as reinforcement learning, it is expected to bring new breakthroughs to the field of signal acquisition and promote the development of electromagnetic environment monitoring towards a more efficient and intelligent direction. Summary of the Invention

[0007] The object of the present invention is to solve the limitations of low channel selection efficiency and insufficient application ability of intelligent technologies in the process of sensor signal acquisition. Therefore, an interactive intelligent spectrum and sample acquisition method and system are proposed. The present invention constructs a collection channel selection model for sensor signals through reinforcement learning, which can dynamically adjust the acquisition strategy according to the real-time spectrum state, optimize the acquisition process of signal samples, and effectively improve the acquisition efficiency.

[0008] The present invention adopts the following technical solutions to achieve the object:

[0009] An interactive intelligent spectrum and sample acquisition method, including the following steps:

[0010] S1. Obtain the original spectrum data of the sensor signal and perform data preprocessing;

[0011] S2. Establish a collection channel selection model for sensor signals through reinforcement learning, and the collection channel selection model has multiple reinforcement learning modes;

[0012] S3. Train the collection channel selection model, and for each reinforcement learning mode, optimize the channel selection strategy of the collection channel selection model;

[0013] S4. Input the preprocessed spectrum data into the collection channel selection model, and the model makes a channel selection action and outputs a channel selection result based on the optimized channel selection strategy;

[0014] S5. Present the preprocessed spectrum data and the channel selection result output by the model synchronously in a visual manner to complete the sample acquisition of the sensor signal.

[0015] Specifically, in step S1, the data preprocessing performed on the original spectrum data includes data reading, normalization, mixing, filtering, decimation, FFT transformation, and visualization operations, so that the original spectrum data is converted into a one-dimensional spectrum array.

[0016] Further, in step S2, the acquisition channel selection model includes the state obtained by the agent from the environment, the channel selection action made by the agent, and the reward obtained by the agent based on the made channel selection action; wherein, the state obtained by the agent from the environment is the preprocessed spectrum data; the agent optimizes the channel selection strategy through the obtained reward.

[0017] Further, the state s obtained by the agent from the environment = {S0, S1, …, S t , …, S n}; s t represents the preprocessed spectrum data received at time t;

[0018] The channel selection action A made by the agent = {A0, A1, …, A t , …, A n}, represents the acquisition frequency point required for the spectrum data at time t, represents the acquisition bandwidth required for the spectrum data at time t;

[0019] The reward obtained by the agent is the reward function at time t represents the degree of conformity between the channel selection result output by the agent based on the channel selection action and the actual acquisition result of the sensor signal, represents the influence of the channel selection result obtained manually by the domain expert on the agent, represents the score given by the domain expert to the channel selection result output by the agent;

[0020] The agent receives the input of the spectrum data in time series, continuously updates the moment values of the state, action, and reward, and obtains a historical data sequence: S t →A t →R t →S t+1 →A t+1 →R t+1 →…; The agent makes a channel selection action based on the state estimation at the current moment through the accumulation and learning of historical data, and outputs a channel selection result.

[0021] Further, in step S2, the reinforcement learning mode of the acquisition channel selection model includes an intelligent mode, an evaluation mode, and an artificial mode; in step S3, the training and optimization process corresponding to each reinforcement learning mode is as follows:

[0022] In the intelligent mode, the original spectrum data forms a one-dimensional spectrum array after preprocessing. The acquisition channel selection model outputs the corresponding channel selection result based on the real-time state of the one-dimensional spectrum array, and at the same time, optimizes the channel selection strategy in real time according to the feedback of the preset reward function.

[0023] In the evaluation mode, domain experts score the channel selection actions of the acquisition channel selection model. This mode pre-collects a preset number of spectrum data, and based on the pre-collected spectrum data and the corresponding scoring situations, the training of the existing reward function in the channel selection model is completed. During actual application, the trained reward function is used to evaluate the channel selection actions of the acquisition channel selection model, thereby optimizing the channel selection strategy.

[0024] In the manual mode, domain experts independently give the channel selection results corresponding to the spectrum data. The acquisition channel selection model updates its own discriminator based on the difference between the results given by the domain experts and the results output by its own actions, and forms a new reward function to optimize the channel selection strategy.

[0025] Preferably, the feedback of the preset reward function in the intelligent mode is given by the trained YOLO network. The YOLO network determines the corresponding channel selection results through the image recognition process of the one-dimensional spectrum array, and feeds them back to the acquisition channel selection model as the basis for optimizing the channel selection strategy.

[0026] Preferably, in the evaluation mode, the trained reward function continues to collect the scoring situations of domain experts on the channel selection actions and fine-tune its own parameters during actual application to optimize the channel selection strategy in real time.

[0027] Specifically, in step S2, the reinforcement learning algorithms used by the acquisition channel selection model include the Q-learning algorithm, DQN algorithm, DDPG algorithm, and TD3 algorithm. The acquisition channel selection model also adopts the GAIL algorithm in an interactive reinforcement learning manner in step S3. Through the adversarial training of its generator and discriminator, the agent learns the channel selection results given by domain experts, and then optimizes the channel selection strategy.

[0028] Preferably, in step S5, the original spectrum data is preprocessed and converted into a one-dimensional spectrum array. A one-dimensional Mask array of corresponding size is generated based on the one-dimensional spectrum array. According to the channel selection results output by the acquisition channel selection model, the one-dimensional Mask array is labeled. The one-dimensional spectrum arrays corresponding to multiple groups of original spectrum data are accumulated into a time-frequency waterfall two-dimensional matrix, and their corresponding one-dimensional Mask arrays are also accumulated into a two-dimensional Mask matrix. The two-dimensional Mask matrix has corresponding labeling information. The time-frequency waterfall two-dimensional matrix and the two-dimensional Mask matrix are synthesized into a visualization matrix, and the visualization matrix forms a time-frequency waterfall visualization diagram of the sensor signal through a drawing plug-in. The labeling information in the time-frequency waterfall visualization diagram represents the corresponding channel selection results, thereby realizing the synchronous external presentation in a visualization manner.

[0029] The present invention also provides an interactive intelligent spectrum and sample acquisition system, which serves as the hardware basis for the foregoing method and is used to implement the steps of the foregoing method; the functional modules of the system include:

[0030] A spectrum data preprocessing module, which is used to obtain the original spectrum data of the sensor signal and perform data preprocessing;

[0031] A reinforcement learning modeling module, which is used to establish an acquisition channel selection model for the sensor signal through the reinforcement learning method; the reinforcement learning modes of the acquisition channel selection model include an intelligent mode, an evaluation mode, and an artificial mode;

[0032] A model training module, which is used to train the acquisition channel selection model and, corresponding to the intelligent mode, the evaluation mode, and the artificial mode, respectively optimize the channel selection strategy of the acquisition channel selection model;

[0033] A sensor signal acquisition module, which is used to input the preprocessed spectrum data into the acquisition channel selection model. The model makes a channel selection action and outputs a channel selection result based on the optimized channel selection strategy, so as to realize the acquisition of the sensor signal;

[0034] A signal acquisition visualization module, which is used to synchronously present the preprocessed spectrum data and the channel selection result output by the model in a visual manner.

[0035] To sum up, due to the adoption of the present technical solution, the beneficial effects of the present invention are as follows:

[0036] By introducing the signal intelligent acquisition technology based on reinforcement learning, the present invention significantly improves the signal acquisition efficiency and response speed of the sensor in electromagnetic environment monitoring. Traditional signal acquisition methods rely on fixed acquisition strategies or simple polling mechanisms, and it is difficult to adapt to complex and changeable electromagnetic environments and make optimal channel selections, resulting in low utilization efficiency of spectrum resources. However, through the intelligent channel selection model of the present invention, the acquisition strategy can be dynamically adjusted according to the real-time spectrum state, thereby optimizing the acquisition process of signal samples and effectively improving the acquisition efficiency.

[0037] The present invention also realizes the multi-dimensional optimization of the acquisition strategy by combining three modes of intelligent, evaluation, and artificial. In the intelligent mode, the system can predict the effective frequency band according to the real-time spectrum data and continuously optimize the strategy through the feedback of the reward function; in the evaluation mode, the reward function is trained and fine-tuned by introducing human feedback to ensure the rationality and practicability of the strategy; in the artificial mode, the expert demonstration provides guidance for the intelligent agent to further improve the accuracy and reliability of the strategy. This multi-mode collaborative optimization method not only enhances the adaptive ability of the system but also significantly reduces the frequency of manual intervention and alleviates the contradiction between the electromagnetic spectrum monitoring task and personnel.

[0038] The present invention thus significantly improves the performance of sensors in complex electromagnetic environments through intelligent and adaptive signal acquisition technologies, providing strong support for the development of future intelligent signal acquisition devices. Brief Description of the Drawings

[0039] Figure 1 It is a schematic diagram briefly showing the overall process of the method of the present invention;

[0040] Figure 2 It is a schematic diagram outlining the reinforcement learning algorithm in the method of the present invention;

[0041] Figure 3 It is a principle block diagram of the reinforcement learning algorithm;

[0042] Figure 4 It is a flow block diagram of the GAIL algorithm in the method of the present invention;

[0043] Figure 5 It is a schematic diagram of the interface of the graphical software GUI in the system of the present invention;

[0044] Figure 6 It is a schematic diagram of the signal sample acquisition interface of the system of the present invention in intelligent mode;

[0045] Figure 7 It is a schematic diagram of the signal sample acquisition interface of the system of the present invention in evaluation mode;

[0046] Figure 8 It is a schematic diagram of the signal sample acquisition interface of the system of the present invention in manual mode. Detailed Description of the Preferred Embodiments

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Generally, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0049] Embodiment 1

[0050] An interactive intelligent spectrum and sample acquisition method, Figure 1The overall process of the method is briefly shown. For synchronous reference, the key steps of the method are summarized as follows:

[0051] S1. Obtain the original spectrum data of the sensor signal and perform data preprocessing;

[0052] S2. Establish an acquisition channel selection model for the sensor signal through reinforcement learning. The acquisition channel selection model has multiple reinforcement learning modes;

[0053] S3. Train the acquisition channel selection model. Corresponding to each reinforcement learning mode, optimize the channel selection strategy of the acquisition channel selection model respectively;

[0054] S4. Input the preprocessed spectrum data into the acquisition channel selection model. The model makes a channel selection action based on the optimized channel selection strategy and outputs the channel selection result;

[0055] S5. Present the preprocessed spectrum data and the channel selection result output by the model in a visual way synchronously to complete the sample acquisition of the sensor signal.

[0056] This embodiment will introduce the detailed content of each step in detail according to the above step sequence.

[0057] First, in step S1, the preprocessing of the original spectrum data can be one of the child threads in the main process of the whole method and is started by the user by operating the GUI interface. The data preprocessing performed on the original spectrum data includes data reading, normalization, mixing, filtering, decimation, FFT transformation, and visualization (the visualization here is for the visualization processing in step S5 and may specifically involve spectrum cropping, shifting, decimation, etc.) operations, so that the original spectrum data is converted into a one-dimensional spectrum array. The obtained one-dimensional spectrum array is the input data for the subsequent training of the acquisition channel selection model or the input data processed by the acquisition channel selection model during actual signal sample acquisition.

[0058] The establishment of the acquisition channel selection model in step S2 is also a child thread in the main process. It establishes a reinforcement learning environment through a specific reinforcement learning algorithm and at the same time supports the training and saving functions of the model in step S3; the process of establishing and applying the reinforcement learning algorithm is also associated with the data preprocessing in step S1, the visualization in step S5, and other contents.

[0059] The acquisition channel selection model of this embodiment includes the state obtained by the agent from the environment, the channel selection action made by the agent, and the reward obtained by the agent based on the made channel selection action; among them, the state obtained by the agent from the environment is the preprocessed spectrum data; the agent optimizes the channel selection strategy by obtaining the reward.

[0060] Meanwhile, in this embodiment, as can be seen from Figure 2 , the reinforcement learning modes for collecting channel selection models include intelligent mode, evaluation mode, and manual mode. For the specific content included in the collection channel selection model, it can be introduced separately as follows:

[0061] The state S obtained by the agent from the environment is S = {S0, S1, …, S t , …, S n}; S t represents the preprocessed spectrum data received at time t;

[0062] The channel selection action A made by the agent is A = {A0, A1, …, A t , …, A n}, represents the acquisition frequency point required for the spectrum data at time t, represents the acquisition bandwidth required for the spectrum data at time t;

[0063] The reward obtained by the agent is the reward function at time t represents the degree of conformity between the channel selection result output based on the channel selection action by the agent and the true acquisition result of the sensor signal, represents the score given by the domain expert to the channel selection result output by the agent, represents the influence of the channel selection result obtained manually by the domain expert on the agent; W1, W2, and W3 represent the corresponding weights respectively;

[0064] The agent receives the input of spectrum data in a time series, continuously updates the moment values of the state, action, and reward, and obtains a historical data sequence: S t →A t →R t →S t+1 →A t+1 →R t+1 →…; Through the cumulative learning of historical data, the agent then makes a channel selection action based on the state estimation at the current moment and outputs a channel selection result.

[0065] Combined with the specific content of the collection channel selection model, in step S3 of this embodiment, the training and optimization processes corresponding to each reinforcement learning mode of this model are as follows:

[0066] In the intelligent mode, the original spectrum data is preprocessed to form a one-dimensional spectrum array. The acquisition channel selection model outputs the corresponding channel selection result based on the real-time state of the one-dimensional spectrum array, and at the same time, according to the feedback of the preset reward function, it optimizes the channel selection strategy in real time. At the same time, the feedback of the preset reward function in this mode is given by the trained YOLO network. The YOLO network determines the corresponding channel selection result through the image recognition process of the one-dimensional spectrum array and feeds it back to the acquisition channel selection model as the basis for optimizing the channel selection strategy.

[0067] In the evaluation mode, domain experts score the channel selection actions of the acquisition channel selection model. This mode pre-collects a preset number of spectrum data, and based on the pre-collected spectrum data and the corresponding scoring situation, completes the training of the existing reward function in the channel selection model. In actual application, through the trained reward function, it evaluates the channel selection actions of the acquisition channel selection model, thereby optimizing the channel selection strategy. At the same time, the trained reward function in this mode continues to collect the scoring situation of domain experts on the channel selection actions and fine-tunes its own parameters in actual application to optimize the channel selection strategy in real time.

[0068] In the manual mode, domain experts independently give the channel selection results corresponding to the spectrum data. The acquisition channel selection model updates its own discriminator based on the difference between the results given by the domain experts and the results output by its own actions, and forms a new reward function to optimize the channel selection strategy. The acquisition channel selection model can continuously learn through this mode until it is familiar with the results demonstrated manually.

[0069] In this embodiment, the core of the acquisition channel selection model lies in its reinforcement learning algorithm. The principle block diagrams of this algorithm in the intelligent mode and the evaluation mode can be seen in Figure 3 the schematic. Reinforcement learning emphasizes the interaction between the agent and the environment. Through continuous trial-and-error learning, it obtains information, learns experience, and updates the control strategy by maximizing the cumulative reward. The reinforcement learning problem is usually modeled as a Markov decision process MDP, which can be represented by a tuple M = {S, A, P, R, γ}. The process of the agent continuously interacting with the environment can be briefly described in the MDP framework as follows: at different times t, the agent based on the current state s t ∈S, selects an action a t ∈A according to the current policy π∈Π, and after executing this action, enters the next state s t+1 ∈S and obtains the corresponding reward signal Thus, one interaction with the environment is completed. The detailed principle explanation of the reinforcement learning algorithm is well-known to those skilled in the art, and this embodiment will not elaborate here.

[0070] This embodiment only introduces and explains the specific reinforcement learning algorithm types preferably used in the acquisition channel selection model. The reinforcement learning algorithms used in the acquisition channel selection model include the Q-learning algorithm, DQN algorithm, DDPG algorithm, and TD3 algorithm. By selecting the algorithms listed above in this embodiment, a better signal sample acquisition effect can be achieved.

[0071] The Q-learning algorithm is the most effective and general reinforcement learning algorithm. It is a temporal difference (TD) algorithm and is offline. By updating the valuation of its Q-table, it is easy to find the optimal path and the best strategy in a simple environment; the specific implementation method is also relatively simple and a mature technology, which is suitable for the resource-constrained sensor application environment in this embodiment.

[0072] The DQN algorithm uses value function approximation and combines the method of deep neural network. Through the target network and experience replay for network training, and by using the deep neural network representation to fit the value function, it solves the problem that the learning speed of the Q-Learning algorithm slows down in the high-dimensional state space, and calculates the Q value through a fixed deep neural network structure, thus reducing the requirement for storage space.

[0073] The DDPG algorithm is a deep deterministic policy gradient method proposed on the basis of DPG, which is a policy learning method that integrates neural networks into DPG. It simultaneously establishes a Q-value function (Critic) and a policy function (Actor). The Q-value function is updated by the temporal difference method, the same as the Q-learning algorithm. The policy function uses the estimation of the Q-value function and is updated by the policy gradient method.

[0074] The TD3 algorithm introduces the concepts of double Q-network and delayed update to improve the stability and convergence of the algorithm. The double Q-network is used to estimate the target Q-value, thereby reducing the overestimation of the target Q-value and improving the stability of training. At the same time, the update of the target policy and the target value function is delayed to reduce the variance in the training process and improve the stability and convergence speed of the algorithm.

[0075] In addition to using the above 4 specific algorithms for the application of the deep learning algorithm in this embodiment, as Figure 4 shown, the acquisition channel selection model is also based on the interactive reinforcement learning method. In step S3, the GAIL method is adopted. Through the adversarial training of its generator and discriminator, the intelligent agent learns the channel selection results given by the domain expert, and then optimizes the channel selection strategy.

[0076] For the application of the GAIL method, in each round of interaction of the acquisition channel selection model, the environment is the one-dimensional spectrum after signal processing. The agent generates a trajectory according to the existing policy, makes a frequency band selection, and uses it together with the demonstration trajectory given manually to update the discriminator. After the update, the discriminator outputs a new reward to the agent to further update the agent's policy. The above steps are continuously repeated, and the generator and the discriminator continuously optimize their respective performances through adversarial training until an ideal policy similar to the manually labeled result is learned.

[0077] In this embodiment, in the intelligent mode and the evaluation mode, the basic process of the reinforcement learning algorithm differs in the way of obtaining the reward function. In the intelligent mode, the signal spectrum in the environment is sent into the trained YOLO network to obtain the reward for the frequency band, while in the evaluation mode, manual scoring of the choices made by the agent is required to affect the reward. The difference here only involves how to obtain the reward function, so the aforementioned 4 specific algorithms and the GAIL method based on interactive reinforcement learning can be used.

[0078] In step S5 of this embodiment, the original spectrum data is preprocessed and converted into a one-dimensional spectrum array. A one-dimensional Mask array of corresponding size is generated based on the one-dimensional spectrum array, and the one-dimensional Mask array is labeled according to the channel selection result output by the acquisition channel selection model; the one-dimensional spectrum arrays corresponding to multiple groups of original spectrum data are accumulated into a time-frequency waterfall two-dimensional matrix, and the corresponding one-dimensional Mask arrays are also accumulated into a two-dimensional Mask matrix. The two-dimensional Mask matrix has corresponding labeling information; the time-frequency waterfall two-dimensional matrix and the two-dimensional Mask matrix are synthesized into a visualization matrix, and the visualization matrix forms a time-frequency waterfall visualization diagram of the sensor signal through a plotting plug-in. Here, relevant operations such as inversion and color mapping can be performed on the visualization matrix according to the characteristics and visual features of the plotting plug-in; the labeling information in the time-frequency waterfall visualization diagram represents the corresponding channel selection result, thereby realizing the synchronous external presentation in a visualization manner.

[0079] In summary, in the process of intelligent spectrum and sample acquisition of sensor signals in this embodiment, the parameters of the spectrum data are used as the state S t and are received by the agent. The agent makes a channel selection according to the real-time policy. At the same time, the state of the signal is also sent into the interactive reinforcement learning module to obtain corresponding rewards through the intelligent, evaluation, and manual modes. Finally, the reward after combining the rewards of each mode, the effective duration and weight of the acquired signal, is fed back to the agent to update the selection policy. After accumulating and learning historical data, the reinforcement learning algorithm realizes the estimation of the current state, thereby outputting a suitable selected channel, achieving the effect of improving the efficiency of signal sample acquisition.

[0080] Embodiment 2

[0081] Based on Embodiment 1, this embodiment provides an interactive intelligent spectrum and sample acquisition system, which serves as the hardware basis for the method in Embodiment 1 and is used to implement the steps of its method. The functional modules of the system include:

[0082] A spectrum data preprocessing module, which is used to obtain the original spectrum data of the sensor signal and perform data preprocessing;

[0083] A reinforcement learning modeling module, which is used to establish an acquisition channel selection model for the sensor signal by means of reinforcement learning. The reinforcement learning modes of the acquisition channel selection model include an intelligent mode, an evaluation mode, and an artificial mode;

[0084] A model training module, which is used for the training of the acquisition channel selection model. Corresponding to the intelligent mode, the evaluation mode, and the artificial mode, it optimizes the channel selection strategy of the acquisition channel selection model respectively;

[0085] A sensor signal acquisition module, which is used to input the preprocessed spectrum data into the acquisition channel selection model. The model makes a channel selection action and outputs a channel selection result based on the optimized channel selection strategy to realize the acquisition of the sensor signal;

[0086] A signal acquisition visualization module, which is used to synchronously present the preprocessed spectrum data and the channel selection result output by the model in a visual manner.

[0087] In this embodiment, for the signal acquisition visualization module, it can correspondingly form a graphical software for front-end display. This graphical software is developed using the Python programming language and the PyQt5 library. The recommended operating system is Windows 7 and above. The graphical software will mainly include a GUI interface display unit to implement functions related to drawing data. At the same time, this software is controlled by the user, which is convenient for the user to perform automatic signal acquisition, view and control the intelligent spectrum monitoring process. The GUI interface presented by the GUI interface display unit can provide various status prompts in a timely manner, and the user can perform various operations quickly through the GUI interface. The control of the GUI interface can become the main thread in the main process of the entire method, thereby starting the remaining sub-modules and executing the corresponding sub-threads.

[0088] As Figure 5 shown, this embodiment shows an interface example of the GUI interface for intelligent spectrum and sample acquisition of a certain video file. Figure 5The interface in it is divided into 7 functional areas: Area A is a one-dimensional spectrogram visualization drawing area for displaying the preprocessed one-dimensional spectrum array; Area B is the control buttons for the manual mode; Area C is a two-dimensional time-frequency waterfall diagram visualization area for displaying the synthesized visualization matrix, marked with the corresponding channel selection results; Area D is the working control buttons for the drawing area, responsible for controlling the working operation status of the entire signal preprocessing, signal visualization, and signal processing; Area E is for file selection, mainly used to read signal files through this area; Area F is for signal file parameter control, and the parameters of the preprocessing process are controlled through the buttons in this area to meet the preprocessing requirements of different types of signal files; Area G is the reinforcement learning mode selection interaction area for the acquisition channel selection model, used to select the intelligent mode, evaluation mode, and manual mode, and provide a score slider that can be controlled in the evaluation mode.

[0089] Figure 6 It shows the schematic diagram of the channel selection result of the system for the input signal file in the intelligent mode; Figure 7 Then it is the corresponding schematic diagram in the evaluation mode, and the expert score is given; Figure 8 It is the interface schematic diagram in the manual mode. In Figures 6 to 8 In the two-dimensional time-frequency waterfall diagram visualization area corresponding to Area C, the green box represents the action situation of the agent, while the blue box is the demonstration annotation manually carried out by the domain expert in the manual mode to guide the acquisition channel selection model to update its strategy.

Claims

1. An interactive intelligent spectrum and sample acquisition method, characterized in that It includes the following steps: S1. Obtain the original spectrum data of the sensor signal and perform data preprocessing; S2. Establish a collection channel selection model for the sensor signal by means of reinforcement learning. The collection channel selection model has multiple reinforcement learning modes; S3. Train the collection channel selection model. Corresponding to each reinforcement learning mode, optimize the channel selection strategy of the collection channel selection model respectively; S4. Input the preprocessed spectrum data into the collection channel selection model. The model makes a channel selection action and outputs a channel selection result based on the optimized channel selection strategy; S5. Present the preprocessed spectrum data and the channel selection result output by the model in a visual way synchronously to the outside, and complete the sample collection of the sensor signal.

2. The interactive intelligent spectrum and sample acquisition method according to claim 1, characterized in that: In step S1, the data preprocessing performed on the original spectrum data includes data reading, normalization, mixing, filtering, decimation, FFT transformation and visualization operations, so that the original spectrum data is converted into a one-dimensional spectrum array.

3. The interactive intelligent spectrum and sample acquisition method according to claim 1, characterized in that: In step S2, the collection channel selection model includes the state obtained by the agent from the environment, the channel selection action made by the agent, and the reward obtained by the agent based on the made channel selection action; wherein, the state obtained by the agent from the environment is the preprocessed spectrum data; the agent optimizes the channel selection strategy by the obtained reward.

4. The interactive intelligent spectrum and sample collection method according to claim 3, wherein: The state S obtained by the agent from the environment is S = {S0, S1, …, S t , …, S n}; S t represents the preprocessed spectrum data received at time t; The channel selection action A made by the agent = {A0, A1, …, A t , …, A n}, represents the acquisition frequency points required for the spectrum data at time t, represents the acquisition bandwidth required for the spectrum data at time t; The reward obtained by the agent is the reward function at time t represents the degree of conformity between the channel selection result output by the agent based on the channel selection action and the actual acquisition result of the sensor signal, represents the impact of the channel selection result obtained manually by the domain expert on the agent, represents the score given by the domain expert to the channel selection result output by the agent; The agent receives the input of spectrum data in a time series, continuously updates the moment values of the state, action, and reward, and obtains a historical data sequence: S t →A t →R t →S t+1 →A t+1 →R t+1 →…; Through the accumulation and learning of historical data, the agent makes a channel selection action based on the state estimation at the current moment and outputs the channel selection result.

5. The interactive intelligent spectrum and sample acquisition method according to claim 1, wherein: In step S2, the reinforcement learning modes of the collection channel selection model include an intelligent mode, an evaluation mode and an artificial mode; in step S3, the training optimization process corresponding to each reinforcement learning mode is as follows: In the intelligent mode, the original spectrum data forms a one-dimensional spectrum array after preprocessing. The collection channel selection model outputs the corresponding channel selection result based on the real-time state of the one-dimensional spectrum array, and at the same time optimizes the channel selection strategy in real time according to the feedback of the preset reward function; In the evaluation mode, the domain expert scores the channel selection actions of the collection channel selection model. This mode pre-collects a preset number of spectrum data, and based on the pre-collected spectrum data and the corresponding scoring situation, completes the training of the existing reward function in the channel selection model; In actual application, the trained reward function is used to evaluate the channel selection actions of the collection channel selection model, so as to optimize the channel selection strategy; In the artificial mode, the domain expert independently gives the channel selection result corresponding to the spectrum data. The collection channel selection model updates its own discriminator according to the difference between the result given by the domain expert and the result output by its own action, and forms a new reward function to optimize the channel selection strategy.

6. The interactive intelligent spectrum and sample acquisition method according to claim 5, wherein: The feedback of the preset reward function in the intelligent mode is given by the trained YOLO network. The YOLO network determines the corresponding channel selection result through the image recognition process of the one-dimensional spectrum array, and feeds it back to the collection channel selection model as the basis for optimizing the channel selection strategy.

7. The interactive intelligent spectrum and sample acquisition method according to claim 5, wherein: In the evaluation mode, the trained reward function continues to collect the scoring situation of the domain expert on the channel selection action and fine-tune its own parameters in actual application, and optimizes the channel selection strategy in real time.

8. The interactive intelligent spectrum and sample acquisition method according to claim 1, wherein: In step S2, the reinforcement learning algorithms used by the acquisition channel selection model include the Q-learning algorithm, DQN algorithm, DDPG algorithm, and TD3 algorithm; the acquisition channel selection model is also based on the interactive reinforcement learning method. In step S3, the GAIL algorithm is adopted, and through the adversarial training of its generator and discriminator, the agent learns the channel selection results given by domain experts, thereby optimizing the channel selection strategy.

9. The interactive intelligent spectrum and sample acquisition method according to claim 1, characterized in that: In step S5, the original spectrum data is preprocessed and converted into a one-dimensional spectrum array. A one-dimensional Mask array of corresponding size is generated based on the one-dimensional spectrum array. According to the channel selection results output by the acquisition channel selection model, the one-dimensional Mask array is labeled; the one-dimensional spectrum arrays corresponding to multiple groups of original spectrum data are accumulated into a time-frequency waterfall two-dimensional matrix, and their corresponding one-dimensional Mask arrays are also accumulated into a two-dimensional Mask matrix. The two-dimensional Mask matrix has corresponding labeling information. The time-frequency waterfall two-dimensional matrix and the two-dimensional Mask matrix are synthesized into a visualization matrix. The visualization matrix forms a time-frequency waterfall visualization diagram of the sensor signal through a plotting plugin. The labeling information in the time-frequency waterfall visualization diagram represents the corresponding channel selection results, thereby realizing the synchronous external presentation in a visual way.

10. An interactive intelligent spectrum and sample acquisition system, characterized in that, This system is the hardware basis for the method described in any one of claims 1-9 and is used to implement the steps of the method; the functional modules of the system include: A spectrum data preprocessing module, which is used to obtain the original spectrum data of the sensor signal and perform data preprocessing. A reinforcement learning modeling module, which is used to establish an acquisition channel selection model for the sensor signal through the reinforcement learning method; the reinforcement learning mode of the acquisition channel selection model includes an intelligent mode, an evaluation mode, and an artificial mode. A model training module, which is used to train the acquisition channel selection model. Corresponding to the intelligent mode, evaluation mode, and artificial mode, it optimizes the channel selection strategy of the acquisition channel selection model respectively. A sensor signal acquisition module, which is used to input the preprocessed spectrum data into the acquisition channel selection model. The model makes a channel selection action and outputs the channel selection results based on the optimized channel selection strategy, thereby realizing the acquisition of sensor signals. A signal acquisition visualization module, which is used to synchronously present the preprocessed spectrum data and the channel selection results output by the model in a visual way.

Images