A Simulated Soccer Robot Evaluation Method Based on Genetic Algorithm-Optimized LSTM Network

Abstract

The application provides a simulation soccer robot evaluation method based on a genetic algorithm optimized LSTM network, and comprises the following steps: based on a simulation soccer platform, a soccer robot team program is established; a basic device evaluation module is constructed; original training data of the soccer robot team is acquired, and a training set and a test set are constructed after preprocessing; based on a long short-term memory network algorithm framework, an evaluation network model of the soccer robot team program is built; a genetic algorithm optimization model is constructed, and a fitness function is defined to guide genetic operation; the evaluation network model is continuously iteratively optimized by using the genetic algorithm; and an optimized evaluator module is obtained to help the soccer robot select optimal action parameters in real time. The genetic algorithm is combined with the long short-term memory network, so that the evaluator can more accurately reflect the decision-making requirements of the soccer robot in a complex dynamic environment, the accuracy of overall decision-making is improved, and the common adaptability problem in the traditional evaluation method is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of simulation soccer robot movement, more particularly, to a simulation soccer robot evaluation method based on a genetic algorithm optimized LSTM network. BACKGROUND

[0002] A simulation soccer robot is a complex system integrating multiple technical fields. In a simulation soccer match, the soccer robot needs to perform various tasks such as attack and defense. In a soccer robot team, these tasks are mainly completed by an evaluator. The evaluation mechanism of a traditional evaluator usually relies on pre-set rules, but this method often has problems such as dependence on an expert system, inability to cope with complex situations, insufficient adaptability, and limitations in optimization. Therefore, there is an urgent need for an evaluation mechanism that can improve the decision-making ability of a simulation soccer robot in complex situations, enhance adaptability, and be easy to optimize.

[0003] After searching, there is insufficient existing research on the optimization of a simulation soccer robot evaluator. For example, Chinese patent application No. CN201910541349.9, entitled "Football robot defense strategy based on reinforcement learning", mainly studies the problem based on a half-field attack platform (HFO) environment and does not deeply study the evaluator mechanism, which is crucial in a soccer robot match. Therefore, there is still a need for an evaluation method that is not highly dependent on an expert system, can cope with complex situations, has strong adaptability, and is easy to optimize. SUMMARY

[0004] The present application aims to provide a simulation soccer robot evaluation method based on a genetic algorithm optimized LSTM network, which solves the technical problems of dependence on an expert system, inability to cope with complex situations, poor adaptability, and difficulty in optimization in a simulation soccer robot team.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solution:

[0006] The simulation soccer robot evaluation method based on a genetic algorithm optimized LSTM network comprises the following steps:

[0007] S1. Based on a simulation soccer platform, a soccer robot team program is established, wherein the soccer robot team program is composed of two teams, and each team includes 11 heterogeneous robot agents;

[0008] S2. Based on the simulation soccer platform, the soccer robot team program, and according to the platform characteristics and the original evaluation mechanism, a basic evaluator module is constructed;

[0009] The input parameters of the basic evaluator module include the currently predicted preferred action and the sequence of possible subsequent actions, and the output is the evaluation score of the currently predicted preferred action.

[0010] S3. Based on the interaction between the simulated football platform, the football robot team program, and the basic evaluator module, the original training data of the football robot team is obtained, and the original training data is processed for missing values, dimensionality reduction, and data augmentation to construct training and test sets.

[0011] S4. Based on the Long Short-Term Memory network algorithm framework, build an evaluation network model for the soccer robot team program, and define the input and output dimensions, number of network layers, number of hidden units, number of fully connected layers, number of neurons in fully connected layers, learning rate, optimizer, training batch size, number of iterations, loss function, and activation function of the evaluation network model.

[0012] S5. Based on the evaluation network model, construct a genetic algorithm optimization model. Use the number of network layers, the number of neurons in each layer, and the parameters of the fully connected layers of the evaluation network model as inputs to the optimization model. In the genetic algorithm optimization model, define a fitness function to guide genetic operations, including the execution of selection, crossover, and mutation. Update the structural parameters of the evaluation network model by evaluating the fitness of individuals.

[0013] S6. The evaluation network model is updated based on the network structure parameters optimized by the genetic algorithm. That is, the evaluation network model is reconstructed and trained according to the updated network structure parameters. The optimal evaluation network model is continuously updated through iterative training in a set number of rounds.

[0014] S7. Based on the evaluation weights output by the finally selected optimal evaluation network model, interact with the basic evaluator module to obtain the optimized evaluator module. The optimized evaluator module will be used in the soccer robot competition to help the soccer robot select the optimal action parameters in real time during the competition.

[0015] Furthermore, the training set described in step S3 includes at least the real-time position parameters of the simulated soccer robot, the real-time velocity parameters of the simulated soccer robot, the real-time position parameters of the ball, the real-time velocity parameters of the ball, the real-time action parameters of the simulated soccer robot, and the real-time physical parameters of the simulated soccer robot.

[0016] Furthermore, the motion features in the original training data of the soccer robot team in step S3 need to be processed by a multi-classifier based on support vector machine (SVM) and synthetic minority oversampling technique (SMOTE). The synthetic minority oversampling technique is used to oversample the minority class samples to generate new samples to balance the class distribution. Then, the balanced dataset is subjected to multi-class classification operation by support vector machine.

[0017] Based on the classification results, the training data that best matches the features of the current evaluation network model is selected and labeled as positive samples to enhance the evaluation network's ability to learn target features, thereby improving the model's training performance.

[0018] The aforementioned multi-classifier based on support vector machines is a method that extends the support vector machine algorithm to multi-classification tasks. It uses a "one-to-many" (OvR) approach for extension, and its objective function is as follows:

[0019]

[0020] Where i represents the current classifier, j represents the j-th data sample in the training set, and y ij w is the label of sample j with respect to classifier i. i It is the weight of the i-th class, b i is the bias of the i-th class, and C is the regularization parameter used to balance the complexity of the model and the penalty for classification errors;

[0021] The final decision function of the multi-classifier based on support vector machines is as follows:

[0022]

[0023] The synthetic minority class oversampling technique described above generates new minority class samples through interpolation. The specific method is as follows:

[0024] x new =x m +λ·(x n -x m )

[0025] Where, x m and x n It is a minority class sample, x n It is x m The nearest neighbor sample, λ is a random number, obtained by x m and x n Linear interpolation is performed on the vectors between them, and new sample points x are generated according to a random proportion of λ. new This expands the number of minority class samples in the training dataset.

[0026] Furthermore, the evaluation network model described in step S4 includes short-term memory units and long-term memory units;

[0027] The short-term memory unit is used to capture short-term time or state changes that occur during program execution, i.e., short-term information centered on the current moment. This short-term information includes at least: short-term player movement, ball passing, short-term changes in player stamina, and short-term atomic actions taken by the player, including turning, sprinting, kicking, tackling, and diving.

[0028] The long-term memory unit is used to store accumulated information or trends over a longer period of time during program execution, i.e., long-term information on a longer timescale than the current moment. This long-term information includes at least: the player's cumulative running distance, the player's cumulative physical exertion, and the cumulative number of offensive or defensive cycles, i.e., the number of cycles a player spends in the opponent's half and our own half.

[0029] Furthermore, the loss function used to evaluate the network model in step S4 is the sparse classification cross-entropy loss function, which is implemented as follows:

[0030]

[0031] Among them, y j This represents the true label of the j-th sample. Let N represent the probability distribution of the network's prediction for the j-th sample, where N is the number of samples.

[0032] Furthermore, the optimizer used to evaluate the network model in step S4 is the Adam optimizer, wherein the parameter update of the Adam optimizer is implemented as follows:

[0033]

[0034] Average shift of gradient:

[0035] m t =β1·m t-1 +(1-β1)·g t

[0036]

[0037] Bias correction:

[0038]

[0039] Where t represents the time step, θ t This represents the model parameters for evaluating the network model at the current time, g. t This represents the gradient of the loss function with respect to the parameters in the current training round, where 'a' represents the learning rate, which determines the magnitude of the change in network parameters for each update.t , v t are the exponential moving averages of the first and second moments of the gradient calculated at the current moment, respectively, and are used to smooth and adjust the update speed of the gradient.

[0040] Furthermore, the genetic algorithm described in step S5 acts on the hyperparameter optimization of the evaluation network model through selection, crossover, mutation, and population update, so as to find the optimal structure configuration of the evaluation network model;

[0041] The goal of the selection operation is to select individuals with high fitness from the current population according to the value of the fitness function to generate a new population, which is specifically expressed as follows:

[0042]

[0043] where P(k) is the probability that individual k is selected, DNA k represents the chromosome of the k-th individual, and each DNA k chromosome represents specific network structure parameters. Each chromosome contains several gene loci, and f(DNA k ) is the fitness of individual k, and POP size is the population size;

[0044] The crossover operation is used to combine the chromosomes of two individuals to generate new offspring, and the specific process is expressed as follows:

[0045]

[0046] where DNA child represents the chromosome of the generated offspring individual, and the chromosome is the network structure parameter. DNA parent1 and DNA parent2 represent two parent individuals used to generate the offspring individual. Mask is a binary vector, and the value of the vector being 1 means inheriting the chromosome from parent1, and being 0 means inheriting the chromosome from parent2;

[0047] The mutation operation increases the diversity of the population by randomly changing the gene loci in the individual chromosome, and is specifically expressed as follows:

[0048]

[0049] where rand(range) represents randomly generating a new value within the specified range range. When rand() < MUTATION_RATE, the mutation operation is triggered, and the gene locus of the offspring is set to rand(range). MUTATION_RATE is the mutation probability defined when initializing the genetic algorithm parameters.

[0050] Furthermore, the next generation of the population in the aforementioned population update operation consists of individuals resulting from selection, crossover, and mutation. This process is specifically represented as follows:

[0051] POP new =Selection(POP) old )+Crossover(POP selected )+Mutation(POP crossed )

[0052] Among them, POP new Represents a new generation of population, Selection(POP) old Crossover (POP) represents a highly fit individual selected from the previous generation of the population. selected Mutation(POP) represents performing a crossover operation on selected individuals, pairing them up to generate new individuals. crossed ) indicates a mutation operation, which randomly changes the gene loci of generated individuals, increasing the diversity of the population.

[0053] Compared with existing technologies, the simulated soccer robot evaluation method based on LSTM network optimized by genetic algorithm described in this invention has the following advantages:

[0054] (1) By combining genetic algorithm with long short-term memory network, this invention effectively optimizes the evaluator, enabling the evaluator to more accurately reflect the decision-making needs of soccer robot in complex dynamic environment and improve the overall decision-making accuracy.

[0055] (2) By employing short-term memory units and long-term memory units in a long short-term memory network, the evaluator can capture short-term and long-term information in a simulated soccer game, ensuring that the soccer robot has higher adaptability in decision-making under different scenarios, and avoiding the common problem of insufficient adaptability in traditional evaluation methods.

[0056] (3) In this invention, the present invention uses a genetic algorithm to continuously iterate and optimize the network structure, so that the parameter configuration of the evaluator model can be automatically adjusted according to the changes in the competition environment, which simplifies the system optimization process, reduces the dependence on expert systems, and solves the bottleneck that is difficult to optimize in the existing technology. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The drawings are not intended to be drawn to scale, and for clarity, not every component will be labeled in each drawing. The drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] in:

[0059] Fig. 1 A flowchart illustrating the simulated soccer robot evaluation method based on a genetic algorithm-optimized LSTM network provided in this embodiment of the invention;

[0060] Fig. 2 This is a schematic diagram of the backbone network structure of the LSTM network optimized by the genetic algorithm in an embodiment of the present invention;

[0061] Fig. 3 This is a complete framework diagram of the simulated soccer platform, simulated soccer robot program, network model, and genetic algorithm optimization module in this embodiment of the invention. Detailed Implementation

[0062] This invention provides a simulated soccer robot evaluation method based on a genetic algorithm-optimized LSTM network, addressing the technical problems of existing simulated soccer robot team evaluation methods, such as reliance on expert systems, inability to handle complex situations, poor adaptability, and difficulty in optimization.

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and do not limit the scope of the invention.

[0064] This invention provides a method for evaluating simulated soccer robots based on LSTM networks optimized by genetic algorithms, combined with... Figs. 1-3 The method includes:

[0065] S1. Based on a simulated soccer platform, a soccer robot team program is established. The robot team program consists of two teams, each of which includes 11 heterogeneous robot players.

[0066] The simulated football platform is RCSSSERVER. Based on the basic parameter interfaces of football fields and players provided by the simulated football platform, a simulated football robot team program is built.

[0067] S2 is based on a simulated soccer platform and a soccer robot team program, and a basic evaluator module is built according to the platform characteristics and the original evaluation mechanism.

[0068] The scoring mechanism of the basic evaluator module of the soccer robot includes at least the following scenarios: 1. When the ball is in the opponent's half and the current soccer robot player is in control of the ball, a positive evaluation score is returned based on the position parameters of the opponent's soccer robot player and the player's own soccer robot player at this time; 2. When the ball is in the player's own half and the current soccer robot player is defending the player in control of the ball, a negative evaluation score is returned based on the position parameters of the player's own soccer robot player acting as the goalkeeper and other players involved in the defense; 3. When the ball is in front of the opponent's goal and the current soccer robot player is in control of the ball, a very large positive evaluation score is returned based on the position parameters of the opponent's soccer robot player and the player's own soccer robot player at this time; 4. When the ball is in front of the player's own goal and the current soccer robot player is defending the player in control of the ball, a very small negative evaluation score is returned based on the position parameters of the player's own soccer robot player acting as the goalkeeper and other players involved in the defense.

[0069] S3, the simulated football platform, the football robot team program, and the basic evaluator module interact to obtain raw training data, perform missing value processing, dimensionality reduction, and data augmentation on the raw data, and construct training and test sets.

[0070] The training data utilizes a multi-classifier based on Support Vector Machines (SVM) and Synthetic Minority Oversampling Technique (SMOTE) to process action features. SMOTE oversamples minority class samples to generate new samples that balance the class distribution. Subsequently, SVM performs multi-class classification on the balanced dataset. Based on the classification results, the training data that best matches the features of the current evaluation network is selected and labeled as positive samples. Increasing the proportion of positive samples in the training dataset enhances the evaluation network's ability to learn target features, thereby improving the model's training performance.

[0071] S4. Based on the Long Short-Term Memory (LSTM) network algorithm framework, an evaluation network model for the soccer robot team program is built. The input and output dimensions, number of network layers, number of hidden units, number of fully connected layers, number of neurons in fully connected layers, learning rate, optimizer, training batch size, number of iterations, loss function, and activation function of the network model are defined.

[0072] The number of network layers, hidden units, fully connected layers, and neurons in fully connected layers are initialized in the chromosomes of the genetic algorithm. The number of network layers is initially random. The optimizer used is the Adam optimizer, the loss function is the sparse classification cross-entropy loss function, and the activation function is the ReLU activation function.

[0073] S5. Based on the aforementioned evaluation network model, a genetic algorithm optimization model is constructed, using the number of network layers, the number of neurons per layer, and the parameters of fully connected layers as model inputs. During the execution of the genetic optimization algorithm, a fitness function is defined to evaluate the performance of each candidate network structure. The output of the fitness function determines the retention and reproduction probability of an individual in genetic operations (selection, crossover, mutation). Specifically, by evaluating the fitness of each individual, the genetic algorithm determines which network structure parameters are more likely to be retained for iterative training of the next generation. Ultimately, the genetic algorithm continuously optimizes and updates the network structure parameters.

[0074] The selection operation uses a roulette wheel method, calculating the probability of selection based on each individual's fitness value, and then randomly selecting individuals from the existing population to form a new population. The probability of selection is directly proportional to fitness. The crossover operation uses single-point crossover, randomly selecting a chromosome for crossover, and then using a Boolean value of True or False to indicate whether the corresponding gene loci on the chromosome are exchanged. The probability of crossover occurring is determined by the parameter CROSS_RATE. The mutation operation uses random mutation, where each gene locus has a certain probability of mutating, determined by the parameter MUTATION_RATE. The specific operation of mutation is to randomly change the value of the gene locus.

[0075] S6, the evaluation network is updated based on the network structure parameters optimized by the genetic algorithm. The evaluation network is reconstructed and trained according to these network structure parameters. Through iterative training in a set number of rounds, the optimal evaluation network is continuously updated based on its performance on the test set each time.

[0076] The reconstruction and training of the evaluation network are primarily accomplished using the fitness function. In each generation, the genetic algorithm generates a new set of network structure parameters, which are the final output chromosomes. For each individual in each generation, an evaluation network is reconstructed using its corresponding network structure parameters and retrained. The training set provided in step 3 is used to train the network, with the goal of minimizing the loss function. After training, the network performance is evaluated using the test set provided in step 3 to obtain the fitness value. With multiple iterations, the network structure parameters are continuously optimized, and the performance of the evaluation network gradually improves.

[0077] S7. Based on the evaluation weights output by the finally selected optimal evaluation network, the module interacts with the basic evaluator module to obtain an optimized evaluator module. This optimized evaluator module will be used in soccer robot matches to help the soccer robot select optimal action parameters in real time during the match, thereby providing more accurate decision support.

[0078] In summary, the simulated soccer robot evaluation method based on a genetic algorithm-optimized LSTM network provided in this invention establishes a soccer robot team program including multiple heterogeneous soccer robot players through a simulated soccer platform. Based on the simulated soccer platform, the soccer robot team program, platform characteristics, and basic evaluation mechanisms, a basic evaluation module for the soccer robot is constructed. The simulated soccer platform interacts with the soccer robot team to obtain raw data. Through missing value handling, dimensionality reduction, and data augmentation, training and test datasets are obtained. An evaluation network model is built based on a Long Short-Term Memory (LSTM) network model, defining its input / output dimensions, number of network layers, number of hidden units, number of fully connected layers, number of neurons in fully connected layers, learning rate, optimizer, training batch size, number of iterations, loss function, and activation function. Based on the evaluation network model, a genetic algorithm optimization model is constructed, using the number of network layers, the number of neurons in each layer, and the parameters of the fully connected layers as model inputs. The fitness function evaluates the performance of each candidate network structure. The output of the fitness function determines the retention and reproduction probability of an individual in genetic operations (selection, crossover, mutation), thereby continuously optimizing and updating the network structure parameters. The evaluation network is updated based on the optimized network structure parameters optimized by the genetic algorithm. The evaluation network is rebuilt and trained based on these network structure parameters. Through iterative training with set rounds, the optimal evaluation network is continuously updated based on its performance on the test set each time. The evaluation weights output by the finally selected optimal evaluation network interact with the basic evaluator module to obtain the optimized evaluator module. The optimized evaluator module will be used in soccer robot games to help the soccer robot select the optimal action parameters in real time during the game, thereby providing more accurate decision support for the soccer robot. This simplifies the optimization difficulty of the soccer robot evaluator and solves the technical problems of existing simulation soccer robot evaluation methods, such as reliance on expert systems, inability to cope with complex situations, poor adaptability, and difficulty in optimization.

Claims

1. A simulation soccer robot evaluation method based on a genetic algorithm optimized LSTM network, characterized in that, The method comprises the following steps: S1, based on the simulation football platform, a football robot team program is established, wherein the football robot team program is composed of two teams, and each team includes 11 heterogeneous robot agents; S2, based on the simulation football platform, the football robot team program, and according to the platform characteristics and the original evaluation mechanism, a basic evaluator module is constructed; The input parameters of the basic evaluator module include the current predicted preferred action and the subsequent possible action sequence, and the output is the evaluation score of the current predicted preferred action; S3, based on the simulation football platform and the football robot team program, the basic evaluator module is interacted, the football robot team original training data is acquired, and the original training data is processed for missing value, dimension reduction and data enhancement, so as to construct a training set and a test set; S4, based on the long short-term memory network algorithm framework, an evaluation network model of the football robot team program is built, and the input and output dimensions, the number of network layers, the number of hidden units, the number of full connection layer layers, the number of full connection layer neurons, the learning rate, the optimizer, the training batch size, the iteration number, the loss function and the activation function of the evaluation network model are defined; S5, based on the evaluation network model, a genetic algorithm optimization model is constructed, the number of network layers, the number of neurons in each layer and the parameters of the full connection layer of the evaluation network model are taken as the input of the optimization model, in the genetic algorithm optimization model, the fitness function is defined to guide the execution of genetic operations including selection, crossover and mutation, and the fitness of individuals is evaluated to update the structure parameters of the evaluation network model; S6, the evaluation network model is updated based on the network structure parameters updated by the genetic algorithm optimization, that is, the evaluation network model is reconstructed and trained according to the updated network structure parameters, and the optimal evaluation network model is obtained by continuously updating through setting the iteration training of rounds; S7, based on the evaluation weight output by the finally selected optimal evaluation network model, the basic evaluator module is interacted to obtain an optimized evaluator module, which will act on the football robot competition and help the football robot to select the optimal action parameters in real time during the competition.

2. The simulation football robot evaluation method based on the genetic algorithm optimized LSTM network according to claim 1, wherein the training set in step S3 includes at least the following features: real-time position parameters of the simulation football robot, real-time speed parameters of the simulation football robot, real-time position parameters of the ball, real-time speed parameters of the ball, action parameters taken by the simulation football robot in real time, and real-time physical parameters of the simulation football robot.

3. The simulation football robot evaluation method based on the genetic algorithm optimized LSTM network according to claim 2, wherein the action features in the football robot team original training data in step S3 need to be processed based on the support vector machine (SVM) multi-classifier and the synthetic minority over-sampling technique (SMOTE). ​ ​ The minority class samples are oversampled by using a synthetic minority oversampling technique to generate new samples to balance the class distribution, and then a support vector machine is used for multi-classification of the balanced dataset; According to the classification result, the training data most matched with the current evaluation network model features are selected and marked as positive samples to enhance the learning ability of the evaluation network for target features and improve the training effect of the model.

4. The simulation soccer robot evaluation method based on the LSTM network optimized by the genetic algorithm according to claim 3, characterized in that, The objective function of the support vector machine-based multi-classifier is as follows: where i denotes the current classifier, j denotes the jth data sample in the training set, y ij is the label of sample j for classifier i, w i is the weight of the ith class, b i is the bias of the ith class, and C is a regularization parameter used to trade off the model's complexity and the penalty of classification errors. The final decision function of the support vector machine-based multi-classifier is as follows: The synthetic minority oversampling technique generates new minority class samples by interpolation, and the specific method is as follows: x new = x m + λ · (x n - x m ) where x m and x n are minority class samples, x n is the nearest neighbor of x m , and λ is a random number. The number of minority class samples in the training dataset is expanded by generating a new sample point x m by linearly interpolating the vectors between x n and x new in a random proportion of λ.

5. The simulation soccer robot evaluation method based on the LSTM network optimized by the genetic algorithm according to claim 1, characterized in that, The evaluation network model in step S4 includes a short-term memory unit and a long-term memory unit; The short-term memory unit is used to capture the time or state changes occurring in a short period during program execution, i.e., short-term information centered on the current time, wherein the short-term information at least includes the short-term movement of the player, the passing of the ball, the short-term physical change of the player, and the atomic action taken by the player in a short period, and the atomic action includes turning, sprinting, kicking, tackling, and diving; The long-term memory unit is used to save accumulated information or trends over a longer period during program execution, i.e., long-term information at a longer time scale from the current time, wherein the long-term information at least includes the cumulative running amount of the player, the cumulative physical consumption of the player, and the cumulative cycle number in attack or defense, i.e., the cycle number of the player in the opponent's half and the home team's half.

6. The simulation soccer robot evaluation method based on the LSTM network optimized by the genetic algorithm according to claim 5, characterized in that, The loss function of the evaluation network model in step S4 adopts a sparse classification cross-entropy loss function, and the specific implementation is as follows: where y j represents the true label of the jth sample, represents the prediction probability distribution of the jth sample by the evaluation network, and N is the number of samples.

7. The simulation soccer robot evaluation method based on the LSTM network optimized by the genetic algorithm according to claim 6, characterized in that, The optimizer of the evaluation network model in step S4 adopts an Adam optimizer, wherein the parameter update of the Adam optimizer is implemented as follows: Average movement of gradient: m t = β1·m t-1 +(1-β1)·g t Bias correction: where t represents the time step, θ t represents the model parameters of the evaluation network model at the current time, g t represents the gradient of the loss function with respect to the parameters in the current round of training, a represents the learning rate, which determines the magnitude of the change in the evaluation network parameters in each update, m t , v t are the exponential moving averages of the first and second moments of the calculated gradient at the current time, respectively, used to smooth and adjust the update speed of the gradient.

8. The simulation soccer robot evaluation method based on the LSTM network optimized by the genetic algorithm according to claim 1, characterized in that, The genetic algorithm in step S5 optimizes the hyperparameters of the evaluation network model through selection, crossover, mutation, and population update, so as to find the optimal structure configuration of the evaluation network model; The selection operation aims to select individuals with high fitness from the current population according to the value of the fitness function to generate a new population, and the specific representation is as follows: where P(k) is the probability of individual k being selected, DNA k represents the chromosome of the kth individual, each DNA k chromosome encodes a specific network structure parameter, each chromosome contains several gene loci, f(DNA k ) is the fitness of individual k, POP size is the population size; The crossover operation is used to combine the chromosomes of two individuals together to generate new offspring, and the specific representation of the process is as follows: wherein DNA child represents the chromosome of the generated offspring individual, i.e. the network structure parameter, DNA parent1 and DNA parent2 represents the two parent individuals used to generate the offspring individual, mask is a binary vector, the value of the vector being 1 indicates inheriting the chromosome from parent 1 and 0 indicates inheriting the chromosome from parent 2; The mutation operation increases the diversity of the population by randomly changing the gene sites of the chromosomes of the individuals, and is specifically represented as follows: where rand(range) represents a new value randomly generated in the specified range range, when rand()<MUTATION_RATE, the mutation operation is triggered, and the gene site of the chromosome of the offspring is set to rand(range), MUTATION_RATE being a mutation probability defined when the genetic algorithm parameters are initialized.

9. The simulation soccer robot evaluation method based on the LSTM network optimized by the genetic algorithm according to claim 8, characterized in that, The next generation population in the population updating operation is composed of the individuals after selection, crossover and mutation, and the process is specifically represented as follows: POP new = Selection(POP old ) + Crossover(POP selected ) + Mutation(POP crossed ) wherein POP new represents a new generation population, Selection(POP old ) represents individuals with high fitness selected from the previous generation population, Crossover(POP selected ) represents that the selected individuals are paired two by two to generate new individuals, and Mutation(POP crossed ) represents a mutation operation to randomly change the gene sites of the chromosomes of the generated individuals to increase the diversity of the population.

Images