quantum optimization parameter adjustment method for distributed deep learning under a Spark framework

Abstract

The invention discloses a quantum optimization parameter adjustment method for distributed deep learning under a Spark framework. The method comprises the following steps of data is collected, preprocessed and grouped and generative adversarial network structure parameters are determined; deep neural network is preliminary constructed, including, number of layers,, node number of each layer, weight and learning rate and the Spark master node broadcasts the parameter information to the slave node; a generative adversarial network model is trained in a data parallel mode; initializing Spark-based distributed quantum particle swarm optimization algorithm parameter information; and each slave node performs distributed independent evolution, summarizes the evolutions through the master node, calculates a fitness function value corresponding to each particle according to the individual optimal and global optimal update connection weights in the previous iteration, and evaluates the performance of the deep learning model. The invention can provide a reference method for how to find the optimal parameter for the distributed deep learning model, and can avoid the problems of priori knowledge required by manual parameter adjustment of the deep learning model and low efficiency.

Description

technical field [0001] The present invention relates to a digital computing device or a data processing device or a data processing method suitable for a specific function, in particular to a quantum optimization parameter adjustment method for distributed deep learning under the Spark framework. Background technique [0002] With the advent of the era of big data, artificial intelligence has developed rapidly. As an important branch of artificial intelligence, machine learning, represented by deep learning, has attracted widespread attention. In practical applications, problems such as insufficient data samples, lack of truly valuable data, and lack of prior knowledge are often faced. Therefore, deep learning has gradually shown its limitations, especially in the problem of relying on large-scale labeled data and a large amount of prior knowledge to adjust parameters. How to adjust parameters to improve the performance of deep learning models has become the current deep lea...

AI Technical Summary

Problems solved by technology

In practical applications, we often face problems such as insufficient data samples, lack of truly valuable data, and lack of prior knowledge.

Images