Unknown electromagnetic data processing method and device based on artificial intelligence technology

Abstract

The invention discloses an unknown electromagnetic data processing method and device based on an artificial intelligence technology. The method comprises the following steps: acquiring electromagnetic information to be processed; performing strategy generation processing on the to-be-processed electromagnetic information to obtain initial spectrum strategy information; the initial spectrum strategy information comprises M pieces of first spectrum strategy information; and performing game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an unknown electromagnetic data processing method and device based on artificial intelligence technology. Background Art

[0002] The agent game process usually involves sequential decision-making tasks, which are very challenging due to highly random environment transitions and delayed rewards. In recent years, a large number of imitation learning works have found that, in experiments, the effect of generative adversarial imitation learning algorithms is better than that of behavioral cloning algorithms. However, the theoretical explanation behind this phenomenon has not been fully understood. The main problem of behavioral cloning algorithms is compound error: due to the characteristics of Markov decision processes, the error of a single step will accumulate with the number of decision steps. Although some work has studied the error accumulation problem in Markov decision processes of finite length, theoretical analysis of the error accumulation problem in Markov decision processes of infinite length is still lacking. In addition, theoretical analysis of why generative adversarial imitation learning algorithms perform better than behavioral cloning algorithms needs further study. Therefore, a method and device for processing unknown electromagnetic data based on artificial intelligence technology is provided to improve the accuracy of optimal strategy generation by learning the value difference between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device for processing unknown electromagnetic data based on artificial intelligence technology, which is beneficial to improve the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0004] In order to solve the above technical problem, a first aspect of an embodiment of the present invention discloses a data processing method, the method comprising:

[0005] Acquiring electromagnetic information to be processed;

[0006] Performing strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information; the initial spectrum strategy information includes M first spectrum strategy information;

[0007] The initial spectrum strategy information is subjected to game screening processing to obtain target spectrum strategy information.

[0008] A second aspect of an embodiment of the present invention discloses a data processing device, the device comprising:

[0009] An acquisition module, used for acquiring electromagnetic information to be processed;

[0010] A first processing module is used to perform strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information; the initial spectrum strategy information includes M first spectrum strategy information;

[0011] The second processing module is used to perform game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information.

[0012] A third aspect of the present invention discloses another data processing device, the device comprising:

[0013] A memory storing executable program code;

[0014] a processor coupled to the memory;

[0015] The processor calls the executable program code stored in the memory to execute part or all of the steps in the data processing method disclosed in the first aspect of the embodiment of the present invention.

[0016] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute part or all of the steps in the data processing method disclosed in the first aspect of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a schematic diagram of a scenario of a data processing system provided by an embodiment of the present invention;

[0019] Figure 2 It is a flowchart of a data processing method disclosed in an embodiment of the present invention;

[0020] Figure 3 is a structural schematic diagram of a data processing device disclosed in an embodiment of the present invention;

[0021] Figure 4 is a structural schematic diagram of another data processing device disclosed in an embodiment of the present invention;

[0022] Figure 5 It is a structural schematic diagram of a strategy generation model disclosed in an embodiment of the present invention;

[0023] Figure 6It is a structural diagram of a 5-state MDP disclosed in an embodiment of the present invention;

[0024] Figure 7 It is a schematic diagram of a test effect on a Mujoco environment disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0027] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0028] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0029] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0030] The embodiments of the present application provide a data processing method, apparatus, computer device, and computer-readable storage medium, which are described in detail below.

[0031] See also Figure 1 , Figure 1 Schematic diagram of a data processing system provided in an embodiment of the present application. The data processing system may include a computer device 100, in which a data processing device is integrated, such as Figure 1 Computer equipment in.

[0032] In the embodiment of the present application, the computer device 100 is mainly used to obtain electromagnetic information to be processed;

[0033] Performing strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information; the initial spectrum strategy information includes M first spectrum strategy information;

[0034] The initial spectrum strategy information is subjected to game screening processing to obtain target spectrum strategy information.

[0035] It can improve the accuracy of optimal strategy generation by learning a large amount of value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0036] In the embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0037] It is understandable that the computer device 100 used in the embodiments of the present application may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may also be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0038] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the data processing system may also include one or more other services, which are not specifically limited here.

[0039] In addition, if Figure 1 As shown, the data processing system may further include a memory 200 for storing data, such as image data, location information, and the like.

[0040] It should be noted that Figure 1 The scenario diagram of the data processing system shown is merely an example. The data processing system and scenario described in the embodiments of the present application are intended to more clearly illustrate the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided in the embodiments of the present application. A person of ordinary skill in the art will appreciate that with the evolution of the data processing system and the emergence of new business scenarios, the technical solution provided in the embodiments of the present application is equally applicable to similar technical problems.

[0041] The present invention discloses an unknown electromagnetic data processing method and device based on artificial intelligence technology, which is beneficial to improve the accuracy of optimal strategy generation by learning the value difference between expert strategy and imitation strategy, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation. The following are detailed descriptions.

[0042] Embodiment 1

[0043] See also Figure 2 , Figure 2 : is a flow chart of a data processing method disclosed in an embodiment of the present invention. Figure 2 The data processing method described is applied to a management system, such as a local server or a cloud server for management, and the embodiments of the present invention do not limit this. Figure 2 As shown, the data processing method may include the following operations:

[0044] 101. Obtain electromagnetic information to be processed.

[0045] 102. Perform strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information.

[0046] In the embodiment of the present invention, the initial spectrum strategy information includes M first spectrum strategy information.

[0047] 103. Perform game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information.

[0048] It should be noted that the data processing method of the present application is applied to fields such as electromagnetic space security. It is combined with artificial intelligence technologies such as reinforcement learning to enable the algorithm to achieve a gaming effect that is equivalent to or even exceeds the level of gaming experts by learning the value differences of a large number of expert strategies and imitation strategies. The embodiments of the present invention are not limited to this.

[0049] It should be noted that the above-mentioned first spectrum strategy information and target spectrum strategy information represent the quantized values ​​corresponding to the transmission / reception actions of the electromagnetic spectrum, that is, each quantized value corresponds to an action number, which is not limited in the embodiment of the present invention.

[0050] It should be noted that after determining the target spectrum strategy information, the present application may also introduce an infinite length discounted Markov decision process, that is, an infinite length discounted Markov decision process may be represented by a 6-tuple:

[0051]

[0052] Where S is the target spectrum strategy information, AA is the action space, and M * is the state transition probability, R is the reward function, γ is the discount factor, and d0 is the electromagnetic information to be processed.

[0053] It should be noted that the above-mentioned electromagnetic information to be processed is the received unclear and unknown electromagnetic information, that is, the electromagnetic wave information that needs to be analyzed, which may include radio waves, microwaves, infrared rays, visible light, ultraviolet rays, X-rays and gamma rays, etc., and the embodiments of the present invention do not limit this.

[0054] Assume that the state space and action space corresponding to the target spectrum strategy information are finite. The Markov decision process runs as follows: At time step t, the agent observes state s from the environment t , then perform action a t , the environment will return a reward r(s) to the agent according to the reward function R t ,a t ), the agent is based on the state transition probability M* (·|s t ,a t )Transfer to the next state s t+1 Without loss of generality, we assume that the reward function is bounded, that is, A steady-state policy π(·|s) describes the distribution of actions in a given state s. The quality of a policy is measured by its policy value:

[0055]

[0056] The goal of reinforcement learning is to find an optimal strategy π * To maximize the value of the strategy:

[0057] π * = argmax π V π

[0058] Note that under the infinite length discounted Markov decision process, due to the discount factor, the policy value mainly depends on the reward obtained in the first H steps. Specifically, the policy value cutoff after H steps can be defined as:

[0059]

[0060] In order to obtain an ε-approximate policy value, that is,

[0061] |V π -V π,H |≤ε

[0062] The truncation length H can be set to:

[0063]

[0064] After ignoring the logarithmic term in H, we get the effective trajectory length 1 / (1-γ). We can see that the effective trajectory length plays an important role in the error analysis of the imitation learning algorithm. To help the subsequent analysis, the discounted steady-state state distribution is introduced:

[0065]

[0066] and the discounted steady-state state-action distribution:

[0067]

[0068] Figuratively speaking, the discounted steady-state state distribution and state-action distribution measure the frequency of visiting a state and a state-action, respectively. Below, we introduce the main methods in the field of imitation learning. Imitation learning learns strategies from expert examples. In general, it is assumed that the expert strategy is a near-optimal strategy, so a good performance can be obtained by imitating the expert. Use π E To represent the expert strategy, Π is used to represent the strategy space to be selected. In imitation learning, the performance indicator of interest is the strategy value difference A small difference in policy value indicates that the imitation policy can achieve performance close to that of the expert policy. In order to minimize the difference in policy value, different imitation learning methods are designed according to different criteria. Below, we briefly introduce the two imitation learning methods that this article focuses on: behavioral cloning and generative adversarial imitation learning algorithms. Behavioral cloning imitates expert strategies by minimizing the difference in action distribution. In the simplest case, behavioral cloning minimizes the KL distance between the expert action distribution and the imitation strategy action distribution. This objective function can also be viewed as maximizing likelihood estimation in supervised learning:

[0069]

[0070] Note that the above objective function is defined under the state-action distribution of the expert strategy. In fact, only The finite samples collected from independent and identical distribution Therefore, the objective function above can be optimized by a finite number of samples in D:

[0071]

[0072] Note that behavior cloning does not need to interact with the environment, and the above objective function can be optimized offline. Generative adversarial imitation learning follows the idea of ​​generative adversarial networks, aiming to imitate expert strategies through adversarial learning. In generative adversarial imitation learning, the discriminator D is used to determine whether a state-action pair comes from an expert example. At the same time, the generator imitates the expert strategy by maximizing the reward output by the discriminator. This optimization problem is defined as follows:

[0073]

[0074] Different from the behavior cloning algorithm, generative adversarial imitation learning requires interaction with the environment to update the strategy in each iteration. When the discriminator reaches the optimal value, that is,

[0075]

[0076] It can be concluded that generative adversarial imitation learning is to minimize the JS distance between the state-action distribution of the expert strategy and the imitation strategy:

[0077]

[0078] Model-based reinforcement learning is a method for training policies on a learned experience transfer model. Here we focus on the problem of environmental imitation, that is, how to efficiently learn an experience model from data collected in the real environment. The environmental imitation problem is very critical to model-based reinforcement learning methods: the error of the restored experience transfer model highly affects the performance of the trained policy. If the error of the experience transfer model is too high, the performance of the policy trained on the transfer model will also be poor. Most current model-based reinforcement learning algorithms restore the experience transfer model by simply fitting a single-step transfer:

[0079]

[0080] where π D is the strategy used to collect data. This objective function is consistent with the principle of behavior cloning methods in imitation learning. Therefore, these methods still have the problem of error accumulation on model bias, because fitting single-step transitions does not exploit the temporal structure of the Markov decision process.

[0081] Given a random expert strategy π E and the imitation strategy π I satisfy (which can be achieved through behavioral cloning algorithm), then this application has

[0082]

[0083] To prove this result, this application proposes an error propagation framework. In this framework, this application first links the policy value difference with the state-action distribution difference. Then, this application proves that the policy distribution difference is an upper bound of the state-action distribution difference. Note that Theorem 1 considers the case of wireless samples. In the case of finite samples, this application can use statistical learning theory to obtain an upper bound on the generalization error ∈.

[0084] This application is for represents an expert example, where It is collected independently and identically from the state-action distribution of the expert strategy. This application assumes that the expert strategy π E and the imitation strategy π I is a deterministic strategy, the set of candidate strategies Π is finite and contains expert strategies. For the imitation strategy π obtained by behavior cloning I , for any confidence level With at least 1-δ probability, this application has

[0085]

[0086] The inference shows that the error bound of the behavior cloning algorithm grows at a square rate with respect to the effective trajectory length, describing the error accumulation problem of the behavior cloning algorithm. Furthermore, the present application obtains that the generalization error bound is inversely proportional to the number of expert examples.

[0087] Below, this application considers the theoretical analysis of strategy imitation using generative adversarial imitation learning.

[0088] Given a random expert strategy π E and the imitation strategy π I satisfy (can be implemented by generative adversarial imitation learning algorithm), then this application has

[0089]

[0090] The above discussion shows that the value difference of the generative adversarial imitation learning algorithm grows linearly with the length of the effective trajectory.

[0091] Consider a discriminator set The discriminators in D are all Δ-bounded, that is, Given a random expert strategy π E and the imitation strategy π I satisfy

[0092]

[0093] Then, for any δ∈(0,1), with probability at least 1-δ, the application has

[0094]

[0095] in

[0096]

[0097] The state-action distribution is collected independently and identically. For any algorithm Alg:D→π I , then there exists a constant δ0∈(0,1 / 10],C, a and a deterministic expert strategy π E So that, for any δ∈(0,δ0), with probability 1-δ, the difference in strategy value has a lower bound

[0098]

[0099] This application is for represents an expert example, where is collected from the state-action distribution of the expert strategy independently and identically distributed. For any algorithm Alg:D→π I , then there exists a constant δ0∈(0,1 / 10],C, an MDP and a deterministic expert strategy π E So that, for any δ∈(0,δ0), with probability 1-δ, the difference in strategy value has a lower bound

[0100]

[0101] The error bound for using the behavior cloning algorithm to imitate the environment is given below.

[0102] Given a true Markov decision process, the transition probability model is denoted as M * , a data collection strategy π D , a learned transfer model M θ satisfy

[0103] (This can be achieved through behavioral cloning), for an arbitrary strategy, satisfying max s D KL (π(·|s),π D (·|s))≤∈ π , then the upper bound of the strategy evaluation error is

[0104]

[0105] This application considers using generative adversarial imitation learning to imitate the environment

[0106] Given a true Markov decision process, the transition probability model is denoted as M * , a data collection strategy π D , a learned transfer model M θ satisfy (This can be achieved through generative adversarial imitation learning), for an arbitrary strategy, satisfying max s D KL (π(·|s),π D (·|s))≤∈ π , then the upper bound of the strategy evaluation error is

[0107]

[0108] In the scenario of spectrum gaming, a very important task is to achieve automatic control and decision-making of spectrum parameters. For example, how to achieve autonomous control of unmanned platforms in the process of spectrum gaming. Reinforcement learning, as a solution to sequential decision-making problems, has received more and more attention in practical applications, and has also demonstrated excellent control and decision-making effects. Under this mechanism, the collected data is not used directly for training, but is first stored in the replay pool, and then the data is obtained by sampling in the replay pool for training. This application studies previous principles based on these two algorithms. This application constructs an MDP, such as Figure 6 This MDP has 5 states, s0 is the initial state, s T is the terminal state. The agent can choose to go left or right. Going left will get a reward of +2 and reach the terminal state, and going right will get a reward of +1 and reach the next state. It can be seen that the optimal strategy is to start from s0, keep going right until s3, and finally return to s T , so a total reward of +5 can be obtained. In this MDP, this application tried the PER and DisCor algorithms respectively, and compared them with the simplest value iteration method. The experimental results are shown in 6b. Figure 6 The left picture in b is the result of PER. Figure 6 The right figure in b is the result of DisCor. Note the blue and red lines in the figure. The blue line represents value iteration, and the red line represents PER and DisCor respectively. It can be seen that the performance of PER and DisCor is not as good as the simplest value iteration. However, this application looks at the green dotted line in the figure. Figure 6 The left graph in b shows that PER is indeed better if we only focus on TD error. Figure 6 The right figure in b shows that DisCor is indeed better if we only focus on whether we have learned a better value function. This result shows that the mismatch between the current sampling strategy and the reinforcement learning goal is real and will significantly affect the performance of the reinforcement learning algorithm. In order to solve this problem, this application first constructs an optimization problem. This application uses S to represent the state space of MDP, A to represent the action space of MDP, η(π) to represent the expected return of strategy π, and π * represents the optimal strategy. * represents the Bellman optimal operator, that is, for the function f∈R S×A ,have

[0109] (B * f)(s,a)=r(s,a)+γmax a′ f(s′,a′)

[0110] To avoid the above-mentioned goal mismatch problem, this application constructs the following optimization problem:

[0111]

[0112] QUR k =argmin Q E μ [w k (s,a)·(QB * Q k-1 ) 2 (s,a)],

[0113] E μ [w k (s,a)]=1,w k (s,a)≥0,

[0114] Where k represents the kth round. In this optimization problem, the present application takes the difference in cumulative rewards between the current strategy and the optimal strategy as the optimization target, and the amount of optimization is the weight of the samples taken from the replay pool. In other words, this optimization problem is to find out what is the sample weight that maximizes the cumulative reward of the current strategy of the present application. In this way, the present application matches the goal of using the replay pool with the goal of reinforcement learning. By solving this optimization problem, the present application obtains the following results:

[0115] w k (s,a)=(E k (s,a)+∈ k (s,a)) / Z *

[0116] in

[0117]

[0118] First, consider ∈ k According to its definition, there are two cases where (s,a) is relatively small: one is that the probability of returning to the previous state is small, and the other is that the number of steps required to return to the previous state is long. Intuitively, most reinforcement learning tasks will not let the agent frequently experience the same state. This application also conducted experiments in the Atari game environment. The experimental results show that in most environments its value is less than 1 / 10 of the previous item. The experimental results also confirm the intuition of this application. In this way, E plays a dominant role in the result. k (s,a) This term, the error term mentioned above can be ignored. The main term consists of four parts, which tell this application that samples with the following properties should be given higher weights:

[0119] 1. Closer to the current strategy (the first item). More efficient optimization requires that the data collected by this application is closer to the current strategy, that is, it is necessary to focus on states that are more easily accessed by the current strategy. This principle has been considered by the LFIW and BCQ algorithms.

[0120] 2. More accurate value function estimation (the third term). This term means that data with inaccurate Q values ​​should be optimized less. Intuitively, an incorrect Q value may lead the Q value to a more wrong direction after Bellman iteration. This principle is exactly the principle of DisCor.

[0121] 3. Higher a posteriori TD error (fourth term). Q k is the estimate of Q after Bellman iteration. This term describes the difference between the estimated Q after Bellman iteration and the Bellman objective. This term is similar to the objective of PER.

[0122] 4. Smaller action probability (second term). This term offsets the effect of the first term to a certain extent, which is similar to the ∈-greedy approach.

[0123] From this result, the present application can see that some previous methods are just a puzzle of the conclusion of the present application. This also explains to some extent why the previous algorithms are effective, because they partially realize the optimal utilization principle of the present application. Based on this theory, the present application designs a practical algorithm. For the first item, it is the so-called importance weight, which the present application can directly solve with the method of LFIW. LFIW divides the playback pool into fast pool D f and slow pool D s Two parts. Next, it will initialize a neural network κ ψ (s,a) and optimize the neural network according to the following loss function:

[0124]

[0125] where f′ and f * are the derivative and convex conjugate of f respectively. The updated κ ψ This is the importance weight that this application wants. For the second and fourth items, this application uses a method similar to DisCor to find their lower bounds and uses the lower bounds as estimates of these two items. Although this method of using the lower bound may give lower weights to some samples that should be given high weights, it will not mistakenly overestimate those low-weighted samples, thus achieving a certain conservative effect. The third item is a more critical one, and because Q *This application cannot obtain this item in advance, which makes it difficult to estimate. This application proposes two solutions for this item. The first solution is to imitate the approach of DisCor and use the cumulative TD error as an estimate of this item. This solution is called REMN in this application. However, this solution requires the use of a neural network, which means that the estimation process needs to go through a complex optimization process, resulting in a long algorithm time-consuming and not particularly accurate.

[0126] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0127] In an optional embodiment, the game screening process is performed on the initial spectrum strategy information to obtain the target spectrum strategy information, including:

[0128] Mapping the initial spectrum strategy information to obtain mapping strategy information; the mapping strategy information includes N second spectrum strategy information; each second spectrum strategy information includes L third spectrum strategy information; N×L is not less than M;

[0129] The mapping strategy information is calculated and processed to obtain target spectrum strategy information.

[0130] It should be noted that the above N represents the number of mapping strategy information generated by evolving the first spectrum strategy information in the initial spectrum strategy. Further, the above N also represents the number of inputs of network nodes in the game screening process, that is, when the second conversion calculation model is used to calculate the mapping strategy information, each network node will select a third spectrum strategy information from each second spectrum strategy information, which is not limited in the embodiment of the present invention.

[0131] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0132] In another optional embodiment, mapping processing is performed on the initial spectrum strategy information to obtain mapping strategy information, including:

[0133] Acquire candidate mapping fuzzy information; the candidate mapping fuzzy information includes K first mapping information;

[0134] For any first spectrum strategy information in the initial spectrum strategy information, based on the first spectrum strategy information and the alternative mapping fuzzy information, determine the target mapping fuzzy information corresponding to the first spectrum strategy information; the target mapping fuzzy information includes L second mapping information; each second mapping information corresponds to one first mapping information;

[0135] Based on the first spectrum strategy information and the target mapping ambiguity information, second spectrum strategy information corresponding to the first spectrum strategy information is determined.

[0136] It should be noted that the first mapping information represents several preset membership functions, such as sine, Gaussian, etc., which is not limited in the embodiment of the present invention.

[0137] It should be noted that the above-mentioned fuzzy information based on the first spectrum strategy information and the alternative mapping can be based on fuzzy reasoning, such as using if-then fuzzy rules. For example, for the fifth first spectrum strategy information, the fuzzy rule corresponding to the first spectrum strategy information is:

[0138] If A1 is LS1 and A2 is LS2 and A3 is LS3 and A4 is LS1

[0139] Zhen A5 is LS2

[0140] Furthermore, the above A1, A2, A3 and A4 represent the first spectrum strategy information ranked first, second, third and fourth respectively; A5 represents the fifth first spectrum strategy information; LS1, LS2, LS3 represent the first, second and third first mapping information ranked first, second and third, which is not limited in the embodiments of the present invention.

[0141] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0142] In yet another optional embodiment, determining second spectrum strategy information corresponding to the first spectrum strategy information based on the first spectrum strategy information and the target mapping ambiguity information includes:

[0143] For any second mapping information in the target mapping fuzzy information, the first spectrum strategy information and the second mapping information are calculated and processed by using the first conversion calculation model to obtain third spectrum strategy information corresponding to the second mapping information;

[0144] Among them, the first conversion calculation model is:

[0145]

[0146] In the formula, BB represents the third spectrum strategy information; A represents the first spectrum strategy information; b1 represents the first mapping coefficient in the second mapping information; and b2 represents the second mapping coefficient in the second mapping information.

[0147] It should be noted that the first mapping coefficient represents the mean value of the Gaussian function corresponding to the second mapping information, which is not limited in the embodiment of the present invention.

[0148] It should be noted that the second mapping coefficient represents the variance of the Gaussian function corresponding to the second mapping information, which is not limited in the embodiment of the present invention.

[0149] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0150] In yet another optional embodiment, the mapping strategy information is calculated and processed to obtain the target spectrum strategy information, including:

[0151] The mapping strategy information is calculated and processed by using the second conversion calculation model to obtain first calculation strategy information; the first calculation strategy information includes P first sub-calculation strategy information;

[0152] Among them, the second conversion calculation model is:

[0153]

[0154] Where CC represents the first sub-computation strategy information; B i A third spectrum strategy information representing the i-th second spectrum strategy information;

[0155] For any first sub-computing strategy information in the first computing strategy information, use the third conversion computing model to perform computing processing on the first sub-computing strategy information to obtain second sub-computing strategy information corresponding to the first sub-computing strategy information;

[0156] Among them, the third conversion calculation model is:

[0157]

[0158] In the formula, DD j represents the second sub-computation strategy information corresponding to the j-th first sub-computation strategy information; CC j and CC h Respectively represent the j-th first sub-computation strategy information and the h-th first sub-computation strategy information;

[0159] Using the fourth conversion calculation model to calculate and process all the second sub-calculation strategy information to obtain target spectrum strategy information;

[0160] Among them, the fourth conversion calculation model is:

[0161]

[0162] Where, E represents the target spectrum strategy information; DD o and DD p Respectively represent the oth second sub-computation strategy information and the pth second sub-computation strategy information; x o Characterizes the conversion coefficient corresponding to the o-th second sub-computation strategy information.

[0163] It should be noted that the above conversion coefficient can be set by the user or determined by the system according to historical parameters, and its value is between [0, 1], which is not limited in the embodiment of the present invention.

[0164] It should be noted that the above-mentioned calculation and processing of the mapping strategy information using the second conversion calculation model is to take out a third spectrum strategy information from each second spectrum strategy information in turn and multiply them to obtain the first sub-computation strategy information. Furthermore, all the third spectrum strategy information multiplied between each two first sub-computation strategy information are not completely consistent, and the embodiment of the present invention does not limit this.

[0165] like Figure 7 As shown, this application was tested on the MuJoCo environment. Figure 6 As can be seen from the figure, when the conversion coefficient increases, the effective trajectory length increases, and the generation strategy effect is better.

[0166] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0167] In an optional embodiment, the above-mentioned performing strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information includes:

[0168] Performing vector conversion processing on the electromagnetic information to be processed to obtain electromagnetic vector information;

[0169] The strategy generation model is used to perform strategy generation processing on the electromagnetic vector information to obtain initial spectrum strategy information.

[0170] It should be noted that the above-mentioned use of the strategy generation model to perform strategy generation processing on the electromagnetic vector information can generate three first spectrum strategy information, that is, using different network structures to autonomously generate strategies for the same electromagnetic vector information to generate multiple different strategies, thereby increasing the autonomy and flexibility of spectrum strategy generation, thereby providing multiple strategy source data for the next step of strategy game selection, further improving the accuracy and reliability of strategy generation, and the embodiments of the present invention are not limited to this.

[0171] It should be noted that the above-mentioned vector conversion processing of the electromagnetic information to be processed is to convert the electromagnetic information to be processed into a 1×18 vector to facilitate efficient processing of the subsequent model. It can be achieved by segmenting it with a word segmenter and then converting it into binary data, or it can be achieved by a deep learning algorithm, which is not limited in the embodiments of the present invention.

[0172] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0173] In another optional embodiment, Figure 5 As shown, the strategy generation model includes a first connection module, a second connection module, a third connection module, a fourth connection module, a fifth connection module, a sixth connection module, a first activation module, a second activation module, a third activation module, a fourth activation module, a fifth activation module, a sixth activation module, a seventh activation module, an eighth activation module, a first convolution module, a second convolution module, a timing processing module and a first conversion module; wherein,

[0174] The input end of the first connection module, the input end of the first conversion module and the input end of the fifth connection module are all configured to receive model input information of the policy generation model; the output end of the first connection module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second connection module is connected to the input end of the third connection module; the output end of the third connection module is connected to the input end of the third activation module; the output end of the third activation module is configured to output the first model output information of the policy generation model; the output end of the first conversion module is connected to the input end of the first convolution module; the output end of the first convolution module is connected to the input end of the fourth activation module; The output end of the fourth activation module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the fifth activation module; the output end of the fifth activation module is connected to the input end of the fourth connection module; the output end of the fourth connection module is connected to the input end of the sixth activation module; the output end of the sixth activation module is configured to output the second model output information of the strategy generation model; the output end of the fifth connection module is connected to the input end of the seventh activation module; the output end of the seventh activation module is connected to the input end of the timing processing module; the output end of the timing processing module is connected to the input end of the sixth connection module; the output end of the sixth connection module is connected to the input end of the eighth activation module; the output end of the eighth activation module is configured to output the third model output information of the strategy generation model.

[0175] It should be noted that the first activation module and the second activation module are constructed based on the tanh activation function, which is not limited in the embodiment of the present invention. The third activation module, the sixth activation module and the eighth activation module are constructed based on the softmax activation function, which is not limited in the embodiment of the present invention. The fourth activation module, the fifth activation module and the seventh activation module are constructed based on the relu activation function, which is not limited in the embodiment of the present invention.

[0176] It should be noted that the above-mentioned first connection module, second connection module, third connection module, fourth connection module, fifth connection module and sixth connection module are constructed based on the fully connected layer. Furthermore, the number of neurons in the above-mentioned first connection module, second connection module and fifth connection module is 32, and the number of neurons in the third connection module, fourth connection module and sixth connection module is 36, which is not limited in the embodiment of the present invention.

[0177] It should be noted that the first conversion module is constructed based on the reshape function. Furthermore, the first conversion module converts a 1×18 vector into a 3×6 matrix, which is not limited in the embodiment of the present invention.

[0178] It should be noted that the first convolution module and the second convolution module are constructed based on 3×3 and 1×3 convolution kernels respectively, which is not limited in the embodiment of the present invention.

[0179] It should be noted that the above-mentioned time series processing module is constructed based on the long short-term memory network, which is not limited in the embodiment of the present invention. Further, the data processing based on the time series processing module can associate the historical strategy information, thereby improving the accuracy of spectrum strategy generation, which is not limited in the embodiment of the present invention.

[0180] It can be seen that implementing the data processing method described in the embodiment of the present invention is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0181] Embodiment 2

[0182] See also Figure 3 , Figure 3 is a schematic diagram of the structure of a data processing device disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited in the embodiments of the present invention. Figure 3 As shown, the device may include:

[0183] An acquisition module 201 is used to acquire electromagnetic information to be processed;

[0184] The first processing module 202 is used to perform strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information; the initial spectrum strategy information includes M first spectrum strategy information;

[0185] The second processing module 203 is used to perform game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information.

[0186] It can be seen that implementation Figure 3 The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0187] In another optional embodiment, Figure 3 As shown, the first processing module 202 performs game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information, including:

[0188] Mapping the initial spectrum strategy information to obtain mapping strategy information; the mapping strategy information includes N second spectrum strategy information; each second spectrum strategy information includes L third spectrum strategy information; N×L is not less than M;

[0189] The mapping strategy information is calculated and processed to obtain target spectrum strategy information.

[0190] It can be seen that implementation Figure 3 The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0191] In yet another optional embodiment, Figure 3 As shown, the first processing module 202 performs mapping processing on the initial spectrum strategy information to obtain mapping strategy information, including:

[0192] Acquire candidate mapping fuzzy information; the candidate mapping fuzzy information includes K first mapping information;

[0193] For any first spectrum strategy information in the initial spectrum strategy information, based on the first spectrum strategy information and the alternative mapping fuzzy information, determine the target mapping fuzzy information corresponding to the first spectrum strategy information; the target mapping fuzzy information includes L second mapping information; each second mapping information corresponds to one first mapping information;

[0194] Based on the first spectrum strategy information and the target mapping ambiguity information, second spectrum strategy information corresponding to the first spectrum strategy information is determined.

[0195] It can be seen that implementation Figure 3The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0196] In yet another optional embodiment, Figure 3 As shown, the first processing module 202 determines the second spectrum strategy information corresponding to the first spectrum strategy information based on the first spectrum strategy information and the target mapping fuzzy information, including:

[0197] For any second mapping information in the target mapping fuzzy information, the first spectrum strategy information and the second mapping information are calculated and processed by using the first conversion calculation model to obtain third spectrum strategy information corresponding to the second mapping information;

[0198] Among them, the first conversion calculation model is:

[0199]

[0200] In the formula, B represents the third spectrum strategy information; A represents the first spectrum strategy information; b1 represents the first mapping coefficient in the second mapping information; and b2 represents the second mapping coefficient in the second mapping information.

[0201] It can be seen that implementation Figure 3 The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0202] In yet another optional embodiment, Figure 3 As shown, the first processing module 202 calculates and processes the mapping strategy information to obtain target spectrum strategy information, including:

[0203] The mapping strategy information is calculated and processed by using the second conversion calculation model to obtain first calculation strategy information; the first calculation strategy information includes P first sub-calculation strategy information;

[0204] Among them, the second conversion calculation model is:

[0205]

[0206] Where C represents the first sub-computation strategy information; B i A third spectrum strategy information representing the i-th second spectrum strategy information;

[0207] For any first sub-computing strategy information in the first computing strategy information, use the third conversion computing model to perform computing processing on the first sub-computing strategy information to obtain second sub-computing strategy information corresponding to the first sub-computing strategy information;

[0208] Among them, the third conversion calculation model is:

[0209]

[0210] Where D j Represents the second sub-computation strategy information corresponding to the j-th first sub-computation strategy information; C j and C h Respectively represent the j-th first sub-computation strategy information and the h-th first sub-computation strategy information;

[0211] Using the fourth conversion calculation model to calculate and process all the second sub-calculation strategy information to obtain target spectrum strategy information;

[0212] Among them, the fourth conversion calculation model is:

[0213]

[0214] Where, E represents the target spectrum strategy information; D o and D p Respectively represent the oth second sub-computation strategy information and the pth second sub-computation strategy information; x o Characterizes the conversion coefficient corresponding to the o-th second sub-computation strategy information.

[0215] It can be seen that implementation Figure 3 The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0216] In yet another optional embodiment, Figure 3 As shown, the second processing module 203 performs strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information, including:

[0217] Performing vector conversion processing on the electromagnetic information to be processed to obtain electromagnetic vector information;

[0218] The strategy generation model is used to perform strategy generation processing on the electromagnetic vector information to obtain initial spectrum strategy information.

[0219] It can be seen that implementation Figure 3 The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0220] In yet another optional embodiment, Figure 3As shown, the strategy generation model includes a first connection module, a second connection module, a third connection module, a fourth connection module, a fifth connection module, a sixth connection module, a first activation module, a second activation module, a third activation module, a fourth activation module, a fifth activation module, a sixth activation module, a seventh activation module, an eighth activation module, a first convolution module, a second convolution module, a timing processing module and a first conversion module; wherein,

[0221] The input end of the first connection module, the input end of the first conversion module and the input end of the fifth connection module are all configured to receive model input information of the policy generation model; the output end of the first connection module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second connection module is connected to the input end of the third connection module; the output end of the third connection module is connected to the input end of the third activation module; the output end of the third activation module is configured to output the first model output information of the policy generation model; the output end of the first conversion module is connected to the input end of the first convolution module; the output end of the first convolution module is connected to the input end of the fourth activation module; The output end of the fourth activation module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the fifth activation module; the output end of the fifth activation module is connected to the input end of the fourth connection module; the output end of the fourth connection module is connected to the input end of the sixth activation module; the output end of the sixth activation module is configured to output the second model output information of the strategy generation model; the output end of the fifth connection module is connected to the input end of the seventh activation module; the output end of the seventh activation module is connected to the input end of the timing processing module; the output end of the timing processing module is connected to the input end of the sixth connection module; the output end of the sixth connection module is connected to the input end of the eighth activation module; the output end of the eighth activation module is configured to output the third model output information of the strategy generation model.

[0222] It can be seen that implementation Figure 3 The described data processing device is beneficial to improving the accuracy of optimal strategy generation by learning the value differences between expert strategies and imitation strategies in large quantities, thereby improving the efficiency and reliability of electromagnetic spectrum strategy generation.

[0223] Embodiment 3

[0224] See also Figure 4 , Figure 4 is a structural diagram of another data processing device disclosed in an embodiment of the present invention. Figure 4 The described device can be applied to a management system, such as a local server or a cloud server for management, etc., which is not limited in the embodiments of the present invention. Figure 4 As shown, the device may include:

[0225] A memory 301 storing executable program codes;

[0226] a processor 302 coupled to the memory 301;

[0227] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the data processing method described in the first embodiment.

[0228] Embodiment 4

[0229] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the data processing method described in the first embodiment.

[0230] Embodiment 5

[0231] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps in the data processing method described in the first embodiment.

[0232] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0233] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0234] Finally, it should be noted that the unknown electromagnetic data processing method and device based on artificial intelligence technology disclosed in the embodiment of the present invention only discloses the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Acquiring electromagnetic information to be processed; Performing strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information; the initial spectrum strategy information includes M first spectrum strategy information; The initial spectrum strategy information is subjected to game screening processing to obtain target spectrum strategy information.

2. The data processing method according to claim 1, characterized in that: The performing game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information includes: Mapping the initial spectrum strategy information to obtain mapping strategy information; the mapping strategy information includes N second spectrum strategy information; each second spectrum strategy information includes L third spectrum strategy information; N×L is not less than M; The mapping strategy information is calculated and processed to obtain target spectrum strategy information.

3. The data processing method according to claim 2, characterized in that: The mapping process is performed on the initial spectrum strategy information to obtain mapping strategy information, including: Acquire candidate mapping fuzzy information; the candidate mapping fuzzy information includes the K first mapping information; For any of the first spectrum strategy information in the initial spectrum strategy information, based on the first spectrum strategy information and the alternative mapping fuzzy information, determine the target mapping fuzzy information corresponding to the first spectrum strategy information; the target mapping fuzzy information includes the L second mapping information; each second mapping information corresponds to one first mapping information; Based on the first spectrum strategy information and the target mapping ambiguity information, second spectrum strategy information corresponding to the first spectrum strategy information is determined.

4. The data processing method according to claim 3, characterized in that: The determining, based on the first spectrum strategy information and the target mapping fuzzy information, second spectrum strategy information corresponding to the first spectrum strategy information includes: For any of the second mapping information in the target mapping fuzzy information, use the first conversion calculation model to calculate and process the first spectrum strategy information and the second mapping information to obtain third spectrum strategy information corresponding to the second mapping information; Wherein, the first conversion calculation model is: In the formula, B represents the third spectrum strategy information; A represents the first spectrum strategy information; b1 represents the first mapping coefficient in the second mapping information; b2 represents the second mapping coefficient in the second mapping information.

5. The data processing method according to claim 2, characterized in that: The calculating and processing the mapping strategy information to obtain the target spectrum strategy information includes: The mapping strategy information is calculated and processed by using a second conversion calculation model to obtain first calculation strategy information; the first calculation strategy information includes P first sub-calculation strategy information; Wherein, the second conversion calculation model is: Wherein, C represents the first sub-computation strategy information; B i The third spectrum strategy information representing an i-th piece of the second spectrum strategy information; For any of the first sub-computing strategy information in the first computing strategy information, use the third conversion computing model to perform computing processing on the first sub-computing strategy information to obtain second sub-computing strategy information corresponding to the first sub-computing strategy information; Wherein, the third conversion calculation model is: Where D j Indicates the second sub-computing strategy information corresponding to the j-th first sub-computing strategy information; C j and C h respectively represent the j-th first sub-computing strategy information and the h-th first sub-computing strategy information; Using a fourth conversion calculation model to calculate and process all of the second sub-calculation strategy information to obtain target spectrum strategy information; Wherein, the fourth conversion calculation model is: Wherein, E represents the target spectrum strategy information; D o and D p Respectively represent the oth second sub-computing strategy information and the pth second sub-computing strategy information; x o A conversion coefficient corresponding to the o-th second sub-computation strategy information is represented.

6. The data processing method according to claim 1, characterized in that: The performing strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information includes: Performing vector conversion processing on the electromagnetic information to be processed to obtain electromagnetic vector information; The strategy generation model is used to perform strategy generation processing on the electromagnetic vector information to obtain initial spectrum strategy information.

7. The data processing method according to claim 6, characterized in that: The strategy generation model includes a first connection module, a second connection module, a third connection module, a fourth connection module, a fifth connection module, a sixth connection module, a first activation module, a second activation module, a third activation module, a fourth activation module, a fifth activation module, a sixth activation module, a seventh activation module, an eighth activation module, a first convolution module, a second convolution module, a timing processing module and a first conversion module; wherein, The input end of the first connection module, the input end of the first conversion module and the input end of the fifth connection module are all configured to receive the model input information of the policy generation model; the output end of the first connection module is connected to the input end of the first activation module; the output end of the first activation module is connected to the input end of the second connection module; the output end of the second connection module is connected to the input end of the second activation module; the output end of the second connection module is connected to the input end of the third connection module; the output end of the third connection module is connected to the input end of the third activation module; the output end of the third activation module is configured to output the first model output information of the policy generation model; the output end of the first conversion module is connected to the input end of the first convolution module; the output end of the first convolution module is connected to the input end of the fourth activation module; The output end of the fourth activation module is connected to the input end of the second convolution module; the output end of the second convolution module is connected to the input end of the fifth activation module; the output end of the fifth activation module is connected to the input end of the fourth connection module; the output end of the fourth connection module is connected to the input end of the sixth activation module; the output end of the sixth activation module is configured to output the second model output information of the policy generation model; the output end of the fifth connection module is connected to the input end of the seventh activation module; the output end of the seventh activation module is connected to the input end of the timing processing module; the output end of the timing processing module is connected to the input end of the sixth connection module; the output end of the sixth connection module is connected to the input end of the eighth activation module; the output end of the eighth activation module is configured to output the third model output information of the policy generation model.

8. A data processing device, characterized in that: The device comprises: An acquisition module, used for acquiring electromagnetic information to be processed; A first processing module is used to perform strategy generation processing on the electromagnetic information to be processed to obtain initial spectrum strategy information; the initial spectrum strategy information includes M first spectrum strategy information; The second processing module is used to perform game screening processing on the initial spectrum strategy information to obtain target spectrum strategy information.

9. A data processing device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the data processing method according to any one of claims 1 to 7.

Images