123results about How to "Improve exploration ability" patented technology

The invention provides an unmanned ship autonomous navigation control method and system based on artificial intelligence, and relates to the technical field of control, and the method comprises the steps: collecting the real-time navigation data of an unmanned vehicle through a multi-source sensor, inputting the navigation environment feature data into a pre-trained reinforcement learning model, and enabling the reinforcement learning model to adopt a dual Q network structure, performing iterative optimization on the navigation strategy based on the environment reward function to generate a navigation strategy optimization parameter; constructing a multi-target route optimization model according to the navigation strategy optimization parameters, performing global optimization on a route by adopting an improved ant colony algorithm, and outputting optimal navigation path data; a hierarchical navigation control model is established according to the optimal navigation path data, the hierarchical navigation control model comprises a strategic layer, a tactical layer and an execution layer, a navigation control instruction is output through the hierarchical navigation control model, and autonomous navigation control of the unmanned vehicle is achieved.

The invention discloses an integrated energy system economic dispatching model method based on deep reinforcement learning, and the method comprises the steps: constructing a park equipment environment model, and carrying out the mathematical description of a park integrated energy system low-carbon economic dispatching problem through taking economy and carbon emission as targets; secondly, the scheduling problem is expressed as a Markov decision process framework, and a state space, an action space and a reward mechanism of the system are reasonably designed; then, a near-end strategy optimization algorithm is adopted to carry out low-carbon economic dispatching decision making, and an action noise dynamic updating strategy is provided to gradually improve the exploration capability and learning capability of a network model; according to the method, the action space, the state space, the reward mechanism, the neural network structure and the like of the intelligent agent are designed in detail, the minimization of the system operation cost is considered on the basis, and a deep reinforcement learning method is adopted for scheduling simulation. The method provided by the invention can perform optimal scheduling according to the load demand of the system, and has good scheduling performance and self-learning ability.

The invention discloses an electrical automatic control method and system. The method comprises the following steps: acquiring temperature data and load current data in an electrical cabinet; denoising the temperature data by adopting a wavelet transform algorithm; filtering the load current data by adopting a Kalman filtering algorithm; establishing a temperature rise compensation model based on a long short-term memory (LSTM) neural network, wherein the output of the temperature rise compensation model is a temperature rise predicted value in the electrical cabinet; setting a multi-objective function of temperature rise control in the electrical cabinet; solving by adopting a multi-target ant colony algorithm to obtain an optimal control strategy of the cooling fan; and starting, stopping and speed adjustment of the cooling fan are controlled according to the optimal control strategy of the cooling fan. Aiming at frequent starting and stopping of a cooling fan of an electrical cabinet in the prior art, a temperature rise compensation model is established based on a long short-term memory (LSTM) neural network, a multi-objective optimization function including temperature rise deviation and starting and stopping times of the cooling fan is set, an improved multi-objective ant colony algorithm is adopted for solving, and starting and stopping of the cooling fan are reduced.

The invention relates to the technical field of fault detection for a wireless communication network, and discloses a fault detection method for the wireless communication network. By collecting data of base station signal strength, network delay, data packet loss rate and user equipment signal quality, the problem of insufficient fault detection caused by a single index is solved. Through multi-dimensional data acquisition, the system can comprehensively evaluate network performance, the fault detection accuracy is improved, and different fault types can be effectively identified especially in a complex fault mode. By storing the data according to the time sequence, the problem of time sequence chaos is avoided, and accurate detection of time-varying faults is ensured. The comprehensive data vector generated by each time of sampling comprises a plurality of network performance indexes, so that the fault detection and positioning capability is improved. Abnormal values are eliminated through data preprocessing, the data quality is ensured, an accurate basis is provided for subsequent analysis, and the reliability of a detection result is ensured.

The invention relates to an optimization method for improving a whale algorithm based on chaos multi-strategy fusion, and belongs to the technical field of mechanical engineering. The method comprises the following steps: firstly, constructing a tension / compression spring design problem model, secondly, fusing multiple improvement strategies to improve a whale optimization algorithm, then, optimizing a mathematical model by using the improved whale optimization algorithm, and finally, outputting an optimal variable combination and an optimal target value of a tension / compression spring model. According to the whale optimization algorithm, multiple improvement strategies are fused, the initial population quality and population diversity of the whale optimization algorithm are improved, the convergence speed and exploration capacity are improved, the situation that a stretching / compression spring design method is caught in local optimum is avoided, and the algorithm has better optimization accuracy when solving the stretching / compression spring design problem.

The invention belongs to the technical field of automatic driving, and discloses an autonomous driving decision near-end strategy optimization method based on distributed exploration enhancement. According to the invention, a distributed architecture of the reinforcement learning decision module of the autonomous vehicle is designed, through parallelization and distributed processing, multiple agents can execute strategy training in different environments at the same time, and strategy updating and empirical data are shared in real time. In addition, the distributed architecture allows the method to operate in a larger-scale data set and a complex environment, and the adaptability and generalization ability of autonomous driving decision making are improved. In addition, a random network exploration (RND) module is also combined, the diversity and effectiveness of exploration are further enhanced, and a near-end strategy optimization algorithm used by the reinforcement learning decision module is perfected. By combining the distributed architecture and the RND module, the training process is remarkably accelerated, the exploration efficiency is improved, the decision-making quality of the automatic driving system is comprehensively improved, and the reliability and efficiency of automatic driving decision-making are improved.

The invention discloses an unmanned ship multi-path optimization method for large-scale dynamic search and rescue tasks, and belongs to the technical field of unmanned ship path planning. According to the method, a multi-track optimization framework combined with a deep reinforcement learning algorithm is introduced, so that the efficiency and accuracy of path planning are effectively improved. Through a multi-trajectory sampling technology, the problem of trajectory data sampling deviation in a traditional method is solved, and the robustness and adaptability of an algorithm in a dynamic rescue environment are enhanced. Meanwhile, an encoder-decoder structure is designed, path exploration of the unmanned ship is optimized by utilizing reinforcement learning, and the exploration capability of the model to a complex solution space is greatly improved. According to the method, the optimal searching path can be quickly found, and the unmanned ship is prevented from falling into a local optimal solution. According to the invention, the unmanned ship can intelligently adapt to a large-scale emergency rescue scene, the search time is greatly shortened, the task success rate is improved, and key technical support is provided for the application of the unmanned ship in emergency rescue.

The invention relates to a data calibration method suitable for an atmospheric pollutant sensor deployed in an embedded terminal. The method comprises the following steps: acquiring original response data of different sensors with different concentrations in different atmospheric environments after temperature compensation, and corresponding temperature, humidity, air pressure data and sampling time; collecting real concentration values of detection objects corresponding to different sensors at corresponding moments given by a national control station or a provincial control station; preprocessing the obtained data to obtain a final data set; a random forest algorithm is used as a teacher model, a gradient elevator algorithm is used as a student model, and hyper-parameters of the gradient elevator algorithm and the random forest algorithm are optimized by using a hedera helix algorithm; substituting a hyper-parameter optimal solution found by optimization into a corresponding algorithm, and retraining the teacher model and the student model by using hyper-parameter combinations; and deploying, updating and displaying the model in the embedded equipment. And the model size is reduced to realize the deployment of the embedded end on the premise of ensuring the accuracy.

The invention discloses a micro-grid system planning and design method and device based on double-layer optimization, and relates to the field of micro-grid system planning and design, and the method comprises the following steps: S1, collecting the operation data of wind power, photovoltaic, hydropower, an energy storage power station, an electrolytic cell and a hydrogen storage tank in a micro-grid system, and the load data of the micro-grid system; s2, constructing a micro-grid system double-layer optimization model based on a double-layer optimization method; the upper-layer model is a capacity optimization layer, and the maximum total income within the operating age limit is used as a target function; the lower-layer model is a scheduling optimization layer, the maximum operation net income in a scheduling period is taken as an objective function, and system power balance, equipment operation and capacity are taken as constraint conditions; and S3, solving the double-layer optimization model by using an improved grey wolf algorithm and a solver. The microgrid system is planned and designed through a double-layer optimization method, a clean energy power generation and energy storage device capacity configuration scheme is reasonably configured, and an energy scheduling strategy is optimized.

The invention discloses a metasurface reverse design method of a generative adversarial network based on anchor point network control. The method comprises the following steps: collecting metasurface design parameters and spectral response data, preprocessing, and encoding the data into a format which can be processed by a deep learning model; and analyzing the training data by using a K-means clustering algorithm to generate a structured control vector. And constructing an AcGAN framework, and integrating the spectrum similarity measurement SOC into a loss function of a generator and a discriminator to enhance the spectrum matching precision. Through an adversarial training method, a generator is trained to generate a design conforming to target spectral characteristics, and meanwhile, a discriminator and an anchor network AncorNet are optimized to quickly predict and evaluate spectral response. Candidate designs are generated using the trained generator in combination with control vectors and random noise. And screening the candidate design to select the optimal design. According to the method, the spectrum regulation and control precision and the design efficiency of the metasurface design are effectively improved, and the challenge faced by a traditional method when a complex spectrum requirement is processed is solved.

The embodiment of the invention provides a verification method and device based on a genetic algorithm, and the method comprises the steps: randomly selecting a plurality of groups of initial key parameters from a preset key parameter value range, and calculating the fitness of each group of initial key parameters in a processor verification process, processing the plurality of groups of initial key parameters according to the fitness, a preset selection probability, a preset crossover probability and a preset mutation probability to finally obtain a plurality of groups of target key parameters, taking the plurality of groups of target key parameters as the plurality of groups of initial key parameters, executing the steps again until a preset termination condition is met, and stopping iteration to obtain a plurality of target key parameters; and calculating and outputting a processor verification coverage rate according to the plurality of groups of target key parameters and the key parameter value range. The target key parameters are obtained based on the genetic algorithm, local optimum is avoided, the parameters are continuously optimized in the iteration process, the high processor verification coverage rate can be obtained, and the processor verification efficiency and quality are improved.

The invention discloses a multi-parameter identification method for a permanent magnet synchronous motor. The method comprises the following steps: constructing a full-rank equation set based on a mathematical model of a PMSM in a d-q axis coordinate system; comparing a parameter output value of the motor in actual operation with a theoretical output value, and establishing a fitness function; parameter setting is carried out by adopting a Logistic chaotic mapping uniform initialization population strategy, and parameter optimization is carried out by utilizing an improved red-mouth blue-magpie optimization algorithm; and correcting the error of the fitness function until the maximum number of iterations is reached, and finally outputting an optimal identification parameter. Simulation results show that the improved Red-beak-Blue-Tapie optimization algorithm can identify the parameters of the permanent magnet synchronous motor more accurately and quickly, and the results are very close to actual motor parameter values.

The invention relates to the field of artificial intelligence, in particular to a digital intelligent substation parallel control management system based on artificial intelligence, which comprises a data acquisition module, a data processing module, an artificial intelligence analysis module, a parallel control module, a digital twinning module and a communication module, a window-by-window attention long and short term memory model is introduced into the artificial intelligence analysis module, multi-dimensional data streams are processed more efficiently through window-by-window processing, and the data analysis capability is improved; the whale optimization algorithm is used in the window-by-window attention long and short term memory model, the population initial diversity and global search ability are improved through chaotic mapping and random reverse learning, the dynamic updating ability of the window-by-window attention long and short term memory model in different stages is enhanced through golden sine, and adaptive disturbance is added to enhance the exploration ability.

The invention provides a mold surface repairing method and system based on laser cladding, and relates to the technical field of laser cladding, the method comprises the following steps: selecting a matrix material with a proper size, and pretreating the surface of the matrix material; according to the actual process requirements, the value range of laser cladding process parameters is determined, and a response curved surface model of the microhardness and the dilution rate is established; optimizing the response curved surface model, and obtaining an optimized process parameter combination by taking maximization of microhardness and minimization of dilution rate as targets; according to the optimized technological parameter combination, the laser cladding process is carried out, and a cladding layer is formed on the pretreated base material; and performing performance detection on the formed cladding layer, and adjusting process parameters according to a performance detection result until an optimal process parameter combination is determined. By means of the method, it can be guaranteed that the hardness of the cladding layer is stabilized within the ideal range, the mold repairing quality is remarkably improved, and the service life of the mold is remarkably prolonged.

The invention discloses a multi-agent experience exploration cooperation method based on a curiosity mechanism. The method comprises the following steps: constructing a multi-agent decision framework; a curiosity mechanism suitable for a multi-agent decision-making framework is constructed; constructing an experience exploration component suitable for the multi-agent decision-making architecture; constructing an experience playback pool and a state playback pool; a multi-agent cooperative confrontation environment is adapted; training the obtained deep neural network formed by the multi-agent experience exploration decision architecture based on the curiosity mechanism to obtain a multi-agent experience exploration collaborative strategy model; and in the multi-agent experience exploration collaborative strategy model, solving the multi-agent collaborative confrontation strategy in the current confrontation scene. Through the above mode, the method can balance the exploration and utilization process of a multi-agent algorithm during training, alleviates the influence caused by reward sparsity in a multi-agent confrontation environment, helps the multi-agent to carry out experiential exploration, searches an optimal strategy for confrontation, and improves the learning efficiency at the same time.

The invention discloses a line path planning scheme generation method and system based on deep reinforcement learning, and the method comprises the steps: firstly constructing a grid map and an evaluation index system of a planning region, determining the index weight based on a Critic method to construct a grid comprehensive cost evaluation model, and then constructing a path planning model with the minimum total cost as a target, the method comprises the following steps of: firstly, extracting an optimal power transmission line planning path, describing the optimal power transmission line planning path as a Markov decision process, then introducing mixed noise and an epsilon-greedy strategy into a depth deterministic strategy gradient algorithm, improving a priority experience playback mechanism to obtain an improved depth deterministic strategy gradient algorithm, and finally training an intelligent agent based on the algorithm and outputting an optimal power transmission line planning path. According to the method, mixed noise is added, the exploring ability of the intelligent agent is improved, a greedy strategy is introduced to accelerate the learning efficiency of the initial stage of the algorithm and reduce the exploring frequency so as to reduce unnecessary trial and error, and finally, an instant reward value is fused into a priority experience playback mechanism to help the intelligent agent to accurately distinguish the importance of different experiences and accelerate the learning efficiency of the intelligent agent.

The invention discloses an improved DS-PPO reinforcement learning method for intelligent game deduction. In order to enable an intelligent agent to make a more accurate decision in a complex war game deduction environment, the DS-PPO firstly introduces a dual-state input mechanism and an action mask mechanism according to the characteristics of large-scale state-action space and environment asymmetry, high randomness and the like of the war game deduction environment; therefore, the intelligent agent can better perceive global environment information and avoid invalid actions. Secondly, an original loss function is redesigned, masks for processing illegal actions and self-defined entropy loss are introduced, exploratory performance of a strategy is enhanced, an intelligent agent is prevented from falling into a local optimal solution, and instability of strategy updating is reduced;

The invention relates to a relationship perception learning method of a time sequence knowledge graph based on an attention mechanism, which comprises the following steps of: firstly, acquiring a target relationship, a neighbor entity and an embedding vector of time sequence information of the neighbor entity through an embedding layer, and dynamically adjusting a weight according to the similarity of the target relationship and the neighbor relationship in a time dimension by utilizing the attention mechanism; therefore, the time sequence information of the neighbors is finely aggregated. Then, a fitness function is introduced to score negative samples, and the most challenging negative sample is selected to realize local optimization; in addition, a specially designed positive and negative sample processing mechanism effectively improves the time sequence discrimination capability of the relation perception embedding model on the negative sample, so that the modeling effect on the time sequence relation is enhanced.

The invention relates to the technical field of energy optimization scheduling, in particular to a multi-source microgrid optimization scheduling method and system. The method comprises the following steps: acquiring a mathematical model of each distributed power supply and an energy conversion device of a micro-grid in a to-be-optimized region, and constructing a target function by taking weighted minimization of fuel cost and operation maintenance cost of the micro-grid, pollutant discharge treatment cost of the micro-grid and load control compensation cost of the micro-grid as a target; setting a power balance constraint, an energy storage charge state constraint, a generation power constraint, each distributed power supply output constraint and a ramp rate constraint as constraint conditions, and constructing a day-ahead optimization model; the day-ahead optimization model is solved through the whale-bat algorithm, and the whale-bat algorithm replaces the search stage of the whale algorithm with the improved bat algorithm to solve the day-ahead optimization scheduling strategy. The optimal scheduling strategy obtained by the invention can respond to dynamic changes more quickly while ensuring high precision.

The invention discloses an evaporation cooling air conditioner energy-saving optimization control method and system based on a mechanism model, and relates to the technical field of evaporation cooling air conditioner control, and the method comprises the steps: carrying out preliminary partitioning according to the distribution condition of evaporation cooling air conditioners in a warehouse, and obtaining cold supply partitions; equipment operation states, personnel activities and lighting systems in the warehouse are monitored in real time, and heat source distribution values of all subareas in the cold supply subarea are obtained; setting a regional heat source prediction algorithm, and predicting a heat source distribution prediction value in the partition in combination with the goods density; and setting a self-adaptive evaporation cooling air conditioner parameter adjusting strategy, and dynamically adjusting the cooling water temperature, the air inlet amount, the air supply temperature and the air supply direction according to the heat source distribution predicted value and the monitored humidity level. The problems that under the conditions of high humidity and high goods density, an air conditioning system is insufficient in robustness and high in energy consumption are solved, and it is ensured that efficient and stable operation can still be achieved when the load changes.

The invention discloses an artificial intelligence-based enterprise credit risk assessment method and system. The method comprises the steps of data acquisition, data preprocessing, credit risk assessment model construction, hyper-parameter optimization and enterprise credit risk assessment. The invention relates to the technical field of enterprise credit risk assessment, in particular to an artificial intelligence-based enterprise credit risk assessment method and system, and the method comprises the steps: obtaining credit assessment original data through data collection; a data preprocessing method of data cleaning, data labeling, data coding, data normalization and data set segmentation is adopted; an improved kernel extreme learning machine model is adopted to carry out enterprise credit risk assessment, a two-way gating loop structure is utilized to efficiently learn a dependency relationship in time sequence data, and the accuracy and stability of assessment are improved; a waterfowl optimization algorithm is adopted to carry out hyper-parameter optimization, higher calculation efficiency and exploration capability are achieved, and hyper-parameters can be optimized in real time in a dynamic learning environment.

The invention provides a fault diagnosis method for an underwater vector propeller, and relates to the technical field of fault diagnosis. The method specifically comprises the following steps: generating a historical fault data set; preprocessing the historical fault data set; constructing a fusion model of the multi-source heterogeneous data; inputting the trained data into a data fusion model, and outputting a fusion result; inputting data from the training set into three hybrid deep learning models combined with a self-attention module for training, then taking prediction results of the three hybrid deep learning models as input of ensemble learning, and generating a final prediction result by using a weighted ensemble strategy; adding a white whale optimization algorithm of a dynamic pinhole imaging strategy to optimize hyper-parameters in the neural network model; and performing fault diagnosis. According to the technical scheme, the problems that in the prior art, an underwater vehicle is low in reliability and safety, high in maintenance cost and short in equipment service life are solved.

The invention discloses a mechanical arm grabbing method and system based on action branch architecture value advantage learning. The method comprises the steps that a mechanical arm grabbing task is modeled as a Markov decision problem; based on the action space, a VA learning algorithm is introduced, and a main network and a target network of an action branch structure are built; based on a greedy strategy, training the main network and the target network, and accumulating an experience playback pool; and performing setting according to an actual scene, and combining a greedy strategy and the experience playback pool to complete a target capture task. The system comprises a modeling module, a network construction module and a mechanical arm training module. According to the method, the deep learning algorithm based on the action branch architecture is used for exploring the deterministic strategy, the discrete state action space of the mechanical arm can be better processed, and then the control performance is improved. The method can be widely applied to the field of mechanical arm control.

The invention discloses a multi-target and multi-component integrated optimization method and equipment for a counter-rotating axial flow fan, and belongs to the technical field of turbomachinery optimization design. According to the method, parametric modeling is carried out on multiple components such as two rows of blades, an inlet current collector and an outlet diffuser of the contra-rotating axial flow fan, geometric parameter combinations of all the components are selected as multi-dimensional decision variables, and a multi-dimensional decision space is determined; obtaining decision sample sets according to the multi-dimensional decision space, solving a plurality of aerodynamic performances of the counter-rotating fan corresponding to different decision sample sets through numerical simulation, and forming a multi-target aerodynamic sample set; combining a genetic algorithm to optimize a weight and a threshold to train a neural network agent model; performing global search and nonlinear optimization through a genetic algorithm based on a neural network agent model by taking the optimal multi-target aerodynamic performance as an optimization target, and outputting a Pareto optimal solution set; and determining a global optimal solution and carrying out numerical simulation verification to obtain an optimized geometric model of the counter-rotating fan. The pneumatic performance of the contra-rotating axial flow fan can be effectively improved.

An automatic generation method for a digital construction scheme of a power transmission line project comprises the steps that the digital construction scheme of the power transmission line project is decomposed, an automatic generation model of the digital construction scheme of the power transmission line project is constructed, and the automatic generation model comprises a state space, an action space, an intelligent agent and a reward function; the intelligent agent comprises a strategy network and a value network, the strategy network is used for outputting action probability distribution, the value network is used for outputting a state value, and the reward function comprises an efficiency reward, a quality reward and a security penalty; training the intelligent agent by using a PPO algorithm, and optimizing parameters of the strategy network by minimizing a loss function; and then obtaining an optimal digital construction scheme of the power transmission line project based on the trained automatic generation model. According to the method, deep reinforcement learning and automatic generation of the digital construction scheme of the power transmission line project are combined, and the obtained optimal scheme can improve the construction efficiency, improve the project quality and ensure the construction safety.

The invention relates to the technical field of mobile robot path planning, in particular to a robot multi-target path planning method based on a self-learning evolutionary algorithm. Comprising the steps of initializing algorithm parameters; initializing a population; in an employed bee stage, performing path crossing, path variation, path shortening and path safety operation on each solution in the population; calculating a target value of each solution, and dividing the population into a non-dominated solution set and a dominated solution set; in the following bee stage, a learning mechanism based on cooperation acts on a non-dominated solution set, and a learning mechanism based on dominated guidance acts on a dominated solution set; combining the non-dominated solution set and the dominated solution set; in the reconnaissance bee stage, an individual body weight starting strategy is used for acting on a solution with the number of continuous evolution failures exceeding a set maximum threshold value; and updating the non-dominated solution set, and judging whether a termination condition is met or not. The application of the invention can effectively improve the path planning efficiency and enhance the path quality.

The invention discloses a weapon-target allocation optimization method based on an improved golden sine algorithm. The method comprises the following steps: establishing a weapon-target damage probability matrix; aiming at a weapon-target allocation problem, establishing a mathematical model comprising a weapon allocation matrix, a target function and a constraint function; generating an original population satisfying the constraint; and optimizing the population by adopting an improved golden sine algorithm, optimizing the population by updating a golden section coefficient and an individual position and using a dynamic random reverse learning strategy, and searching a better individual until iteration is ended. Compared with the prior art, the method provided by the invention avoids falling into a local optimal solution in the WTA problem optimization process by improving the position updating formula of the golden sine algorithm, and meanwhile, the population diversity is improved by fusing the dynamic random reverse learning strategy, and the algorithm is prevented from premature convergence, so that the algorithm has relatively strong global search capability, and the algorithm is more suitable for being popularized and used. And meanwhile, the method has relatively high capability of jumping out of a local optimal solution.

The invention requests to protect a reinforcement learning mixing method applied to assembly sequence planning, the assembly sequence planning can be essentially understood as a sorting of product parts, the excellent assembly sequence can shorten the production cycle of products, and the production cost is reduced. In the invention, a reinforcement learning algorithm, i.e., a Greedy-QS algorithm, which combines a Q-learning algorithm and a Sarsa algorithm is provided to solve the problem of assembly sequence planning. In the work of the method, firstly, the to-be-assembled product is reasonably abstracted into a mathematical model expressed by several Boolean matrixes, the complexity of the assembly model is reduced, the operability of the algorithm for the assembly sequence planning problem is improved, and then the optimal solution is solved by using the Greedy-QS algorithm. Experiments prove that compared with a Q-learning algorithm, a Sarsa algorithm and a Q-SARSA algorithm, the Greedy-QS algorithm has a faster convergence speed in the assembly sequence planning problem, the obtained reward value of the optimal assembly sequence is higher, and the advantages of the Q-learning algorithm and the Sarsa algorithm are both achieved.

The invention discloses a cold-chain logistics vehicle planning system. The system comprises the following steps of initial population creation, explosion and spark generation, evaluation and selection, genetic manipulation, pheromone updating, termination judgment and optimal solution output. According to the method, a dynamic pheromone updating and path reward and punishment mechanism is introduced through improvement of pheromone updating, so that the convergence speed of the algorithm is accelerated, and the quality of a solution is improved; the exploration depth of a solution space is increased by improving the crossover operation and considering the length and quality of an introduced path; improvement of mutation operation depends on a self-adaptive mutation strategy, the exploration capability of the algorithm is enhanced, multi-objective optimization is introduced into path planning through the improved firework algorithm to consider transportation cost, temperature control and energy consumption management at the same time, and a dynamic adjustment mechanism is combined to cope with frequently changing terrain and climate conditions in the karst landform, so that the development of the karst landform is facilitated. Therefore, the adaptability of the algorithm in a complex environment is improved.

The invention discloses an unmanned aerial vehicle communication system strategy optimization method based on deep reinforcement learning, and the method comprises the steps: building an unmanned aerial vehicle communication system model which comprises an unmanned aerial vehicle, an RIS and a plurality of ground receiving users, carrying out the optimization of a maximum rate C through an improved TD3 algorithm improvement-TD3 by employing the deep reinforcement learning (DRL) technology under the condition of giving specific channel state information (CSI), and carrying out the optimization of the maximum rate C through the improved TD3 algorithm improvement-TD3. And solving an optimal beam forming matrix and an optimal phase shift matrix. According to the method, entropy regularization is introduced into an existing TD3 algorithm, so that exploratory performance is enhanced, an intelligent agent is promoted to explore more environments, the relation between exploration and utilization is balanced, finally, an optimal transmitting beam forming matrix and RIS phase shift are obtained, falling into a local optimal solution is avoided, and learning robustness and generalization ability are improved.