90 results about "Vanishing gradient problem" patented technology

The invention discloses an image super-resolution method based on dense connection network. By increasing the depth of a convolution neural network and introducing a large quantity of jumping connection in the deep network, the image super-resolution method based on dense connection network effectively solves the problem that the gradient disappears during the reverse propagation of the deep network, optimizes flowing of information on the network, and improves the super-resolution reconstruction capability of the convolution neural network. At the same time, the image super-resolution method based on dense connection network is effectively combined with the bottom layer characteristic and the high layer abstract characteristic, and can reduce the model parameters and compress the deep network model so as to improve the reconstruction efficiency of the image super-resolution. Besides, by introducing a deep monitoring technology, the image super-resolution method based on dense connection network can reconstruct the super-resolution image at different depth of network, thus not only optimizing training of the deep network, but also being able to selecting a suitable network depth to reconstruct a high definition image according to the calculation capability of the test terminal during the testing process. Finally, the image super-resolution method based on dense connection network utilizes an image ser having a plurality of amplification factors to train, so that the obtained model can perform image super-resolution on a plurality of dimensions and does not need to train different models for every amplification factor.

The invention belongs to the technical field of image segmentation, in particular to an image segmentation method, a system and an electronic device based on depth learning. The image segmentation method based on depth learning comprises the following steps: step A: normalizing the original image; B, inputting the normalized image into a ResUNet network model, extracting a feature map containing global semantic information from the input image by the ResUNet network model, upsampling the feature map and stacking the feature map to obtain a final feature map; step c: classifying pixel by pixelthe feature map after upsampling and stacking processing, and outputting the image segmentation result. The present application solves the common problem of gradient disappearance in a convolution neural network, and also enables the network to be more easily trained and converged to a better segmentation result.

The invention provides an infant brain magnetic resonance image partitioning method based on a fully convolutional network. The main content of the method comprises the multi-stream three-dimensionalfully convolutional network (FCN) with jump connection, partial transfer learning, training and testing and evaluation. According to the process of the method, first, a probability graph of each pieceof brain tissue is learned from a multi-modal magnetic resonance image; second, initial partitions of different brain tissue are obtained from the probability graphs and used for calculating a distance graph of each piece of brain tissue; third, spatial context information is simulated according to the distance graphs; and last, final partitioning is realized by use of spatial correlation information and the multi-modal magnetic resonance image, wherein the training process mainly comprises training data increasing, training patch preparation and iterative training, and various detection values are used for evaluation after testing is performed. Through the partitioning method, a white matter area, a grey matter area and a cerebrospinal fluid area are successfully divided, the potential gradient vanishing problem of multi-level deep supervision is relieved, training efficiency is improved, and partitioning performance is greatly enhanced.

The invention relates to an image classification method based on a multi-scale dense convolutional neural network and a spectral attention mechanism. The multi-scale dense convolutional neural networkis constructed by using a dense connection mechanism, and the dense connection mechanism can effectively alleviate the problem of gradient disappearance, enhance feature propagation, encourage feature multiplexing, greatly reduce the number of parameters, and reduce the demand for training samples in the network training process; In addition, the network is combined with a spectrum attention mechanism, and the characteristic utilization of the spectrum direction is more effective. According to the method, autonomous extraction and high-precision classification of hyperspectral image depth features are realized under the condition of small samples. Compared with an existing hyperspectral image classification method based on deep learning, the hyperspectral image classification method has the advantages that the sample demand is lower, and the precision is higher.

The invention discloses a video compression artifact adaptive removing method based on depth learning, which adopts a depth-dense connected convolution network to automatically extract the compressioncharacteristics of a video frame, and can effectively avoid the shortcomings caused by the manual design of a filter in the traditional method. The invention acts on the post-processing stage of thevideo, and does not affect the processing flow and the real-time performance of the existing video coding and decoding algorithm. A new image quality prediction model is proposed to realize automaticselection of compression artifacts with different intensities, which has strong adaptive ability. Deep-connected convolution network is used to remove video compression artifacts, which can effectively alleviate the problem of gradient disappearance, deepen the network structure and enhance the network non-linear expression ability. At the same time, the network can also make full use of the characteristics of the middle layer, which not only enhances the propagation and multiplexing of features, but also greatly reduces the network parameters.

The invention discloses a software reliability prediction model based on a depth CG-LSTM neural network, and belongs to the technical field of computer software. The model comprises a model training part and a model prediction part, wherein the model training part is used for performing data normalization processing on a software failure data set; and the software reliability prediction model based on the depth CG-LSTM neural network is trained by using software failure data set has subjected to normalization processing to acquire a prediction model. The model prediction part is used for acquiring the current software failure data and performing data normalization processing, and then the acquired prediction model is input to predict future software failure to acquire a prediction result.According to the model provided by the invention, the problems of gradient vanishing and poor generalization ability of the software reliability prediction model based on the traditional neural network are overcome, the model is higher in prediction accuracy and wider in applicability.

The invention provides a GRU based recurrent neural network multi-label learning method, which comprises the steps of S1, initializing a system parameter [theta]=(W, U, B); S2, inputting a sample {xi,yi}<i=1><N>, calculating a hidden state hT of the output of an RNN (Recurrent Neural Network) at each moment, wherein the sample xi belongs to R<M*1>, yi is a multi-label vector of the sample xi, andyi belongs to R<C*1>; S3, calculating a context vector hT and output zi of an output layer; S4, calculating the predicted output yi^, calculating the loss Li, and determining an objective function J;S5, solving the gradient of the system parameter [theta]=(W, U, B) according to a gradient descent method and a BPTT (Back-propagation Through Time) algorithm; S6, determining a learning rate [eta],and updating each weight gradient W=W-[eta]*[delta]W; S7, judging whether the neural network reaches stable or not, if so, executing a step S8, if not, returning the step S2, and iteratively updatingmodel parameters; and S8, outputting an optimization model. According to the invention, effective feature representation of the sample can be obtained by fully utilizing the RNN so as to improve the accuracy of multi-label classification. In addition, a problem of gradient disappearance is not easy to occur in back propagation.

The invention discloses a hyperspectral image classification method based on a global attention residual network, and the method comprises: constructing an overall network, which comprises a multi-scale feature extraction network, a global attention module and an improved residual network module; performing multi-scale feature extraction to extract hierarchical features of the hyperspectral image; constructing a spatial and spectral dependency relationship of global pixel points by the global attention module through combination of the spatial attention module and the spectral attention module; fusing the improved residual error network module and the global attention module to form a novel global attention residual error network; and sending an output result into a classifier through global pooling for final classification, and outputting a result. According to the method, rich spatial-spectral features are obtained at the same time by introducing a multi-scale receptive field and a global attention module, and an improved residual network is added to relieve the gradient disappearance problem and accelerate network convergence, so that the classification precision is improved, and a good and stable classification effect is ensured.

The invention discloses a short-term residence load prediction method based on Attention-GRU. The method comprises the following steps: data preprocessing; the first step of carrying out load prediction by using a deep learning model being to prepare data in a proper format and evaluate the consistency of daily power distribution by using a density-based noise application space clustering (DBSCAN)technology; in the secondary step, constructing a training set and a test set. According to the method, two algorithms of artificial intelligence in natural language processing are combined to construct a short-term residence load prediction model, and the model uses a GRU algorithm to not only overcome the defects of a recurrent neural network of a traditional intelligent prediction algorithm, but also solve the problems of gradient explosion and gradient disappearance of RNN. The Attention layer is used for endowing the input vector in the next time step with the feature weight learned by the model, and highlighting the influence of the key feature on the predicted load.

The invention discloses a remote sensing image cloud identification method based on depth learning. The method comprises the following steps: automatically acquiring remote sensing cloud images, making remote sensing images into training sets and expanding the existing training sets, and making labels in the training sets, constructing a deep convolution neural network based on SegNet neural network structure with multi-scale convolution kernel and high symmetry and for restoring the feature map i by deconvolution layer finally, to avoid over-fitting, under-fitting and gradient disappearance in network training, adopting the method of subsection training, after the training, extracting the feature of the remote sensing image by using the obtained weight file, and carrying out cloud detection at the pixel level. The remote sensing image cloud identification depth convolution neural network of the invention improves the retrieval accuracy by utilizing multi-scale convolution and high symmetry.

The invention relates to a video semantic segmentation method based on a ConvLSTM convolutional neural network. The method comprises the following steps: A, constructing and training a video semanticsegmentation network: (1) acquiring a data set; (2) constructing a video semantic segmentation network; (3) training a video semantic segmentation network; (4) testing the segmentation accuracy of thevideo semantic segmentation network; B, performing video semantic segmentation through the trained video semantic segmentation network structure. According to the method, the ConvLSTM module is adopted to consider the correlation between the adjacent frames of the video, so the accuracy of semantic segmentation of the video is improved. In addition, a densely-connected cavity space pyramid pooling module with densely-connected blocks is adopted, so the transmission of features and gradients is more effective, the problem of gradient disappearance in the deep network training process is solved, multi-scale context information can be systematically aggregated, and the receptive field is expanded.

The present invention relates to a learning method of generative adversarial network (GAN) with multiple generators for image denoising, and provides a generative adversarial network with three generators. Such generators are used for removing Poisson noise, Gaussian blur noise and distortion noise respectively to improve the quality of low-dose CT (LDCT) images; the generators adopt the residual network structure. The mapped short connection used in the residual network can avoid the vanishing gradient problem in a deep neural network and accelerate the network training; the training of GAN is always a difficult problem due to the unreasonable measure between the generative distribution and real distribution. The present invention can stabilize training and enhance the robustness of training models by limiting the spectral norm of a weight matrix.

The invention relates to the technical field of information retrieval, and provides a semantic document retrieval method for searching topic type query in TED speech. The method comprises: training aneural network model by utilizing the existing query and document, and learning neural network model parameters; when a user inputs a query, a query likelihood retrieval model is used to obtain a preliminary retrieval result; and inputting the preliminary retrieval result into the neural network model with the fixed parameters for reordering, and determining a final retrieval result. According tothe method, the problem that a good effect cannot be achieved due to the fact that a traditional retrieval method in topic type query retrieval lacks semantic contact between queries and documents issolved; respectively modeling the topic type query and the speech document by introducing a neural network, and obtaining the semantic level correlation between the query and the document. In the neural network part, a recurrent neural network and a convolutional neural network are connected in series, and in addition, in order to solve the problem of gradient disappearance, a currently popular LSTM module is adopted.

The invention discloses a three-dimensional atmospheric temperature profile inversion method and system based on a DenseNet convolutional neural network, and belongs to the field of atmospheric microwave remote sensing. The method comprises the following steps: constructing a training data set according to a two-dimensional atmospheric observation brightness temperature image and a three-dimensional atmospheric temperature profile of an oxygen absorption frequency band; based on the training data set, training the DenseNet convolutional neural network until the DenseNet convolutional neural network is converged to obtain a trained network; and inputting a brightness temperature image to be inverted into the trained network, and outputting a three-dimensional atmospheric temperature profileobtained by inversion. A data set used for training takes two-dimensional brightness temperature images as units, each brightness temperature image covers a certain area on the earth, the whole dataset spans a long time interval, the generalization ability is greatly improved, and the inversion error is also reduced. The DenseNet convolutional neural network is large in layer number, the problemof gradient disappearance in the training process is avoided due to the dense connection structure of the DenseNet convolutional neural network, the DenseNet convolutional neural network is suitablefor complex inversion, data of three scenes of clear sky, cloud and rainy days can be directly inverted together, and time consumption is reduced.

The invention discloses a power transmission line icing thickness prediction method based on CEEMDAN-QFOA-LSTM, and relates to the field of combination of power transmission line state evaluation anddeep learning. The method comprises the following steps o: (1) carrying out data acquisition and preprocessing; (2) carrying out CEEMDAN decomposition on an icing thickness historical data sequence (12); (3) optimizing hyper-parameters of the LSTM by a quantum drosophila melanogaster algorithm; (4) carrying out LSTM model training (14); and (5) predicting the icing thickness of a power transmission line and analyzing a result (15). According to the method, the CEEMDAN decomposition algorithm is used, a sequence which is difficult to directly predict is converted into a plurality of predictablecomponent sequences; a neural network can more accurately grasp the law of the sequence according to multi-dimensional feature information obtained through decomposition; a QFOA optimization algorithm is used for obtaining the hyper-parameters, a complex manual parameter adjustment process is avoided, and a network model is trained more effectively; the used LSTM neural network does not have theproblem of gradient disappearance of a general network, so that optimal convergence of the model is ensured, and the problem of short-term and long-term time sequence prediction is effectively solved.

The invention provides a facial expression recognition method based on multi-channel fusion and a lightweight neural network, and aims to solve the problems that in a traditional facial expression recognition learning method, the feature extraction process is complex, and deeper high semantic features and deep features cannot be obtained from an original image. According to the facial expression recognition method based on the multi-channel fusion and the lightweight neural network, a three-channel feature image after multi-channel fusion is used as the input of the constructed lightweight neural network, the design idea of deep separable convolution is adopted to reduce the number of parameters and the amount of calculation, and meanwhile, a residual connection mechanism is adopted to construct a network model to solve the problems of network performance degradation and gradient disappearance, so that good performance can be ensured while a deeper network is trained. Experiments show that the model provided by the invention can effectively extract facial expression features and classify expressions, and has good accuracy and robustness.

In order to accelerate the convergence speed of a Dep Belief Network (DBN) and improve the acquisition precision of situation elements under a small sample condition, the invention provides a networksecurity situation element acquisition mechanism based on BN-DBN. On one hand, BN is added into the deep neural network to solve the gradient disappearance problem; on the other hand, an improved active Learning (IAL) algorithm is put forward through the deep neural network output layer, and the deep belief network is finely adjusted in the reverse direction through the algorithm, and the algorithm balances the sample types by actively selecting training samples in each iteration. Theoretical analysis and instance data simulation results show that the mechanism can solve the problems that theconvergence speed of the deep neural network is too slow or the gradient disappears, and samples are accurately classified. And the acquisition precision, the convergence rate and the algorithm complexity of the method are superior to those of other situation element acquisition mechanisms listed in the text.

The invention provides an evapotranspiration data downscaling method based on multi-source data and deep learning. The method comprises the steps of obtaining low-spatial-resolution satellite surface evapotranspiration data, low-spatial-resolution atmosphere reanalysis data and high-spatial-resolution satellite remote sensing data; carrying out data preprocessing; based on a built deep learning regression network, establishing a surface evapotranspiration inversion model; and performing downscaling inversion on high-spatial-resolution surface evapotranspiration through the surface evapotranspiration inversion model established on the low-spatial-resolution surface evapotranspiration. According to the method, the earth surface evapotranspiration inversion precision is improved by comprehensively considering earth surface evapotranspiration related influence factors. The nonlinear complex relation between remote sensing earth surface parameters and atmospheric data and earth surface evapotranspiration is deeply analyzed based on deep learning. The relation between the remote sensing earth surface parameters and atmospheric data and earth surface evapotranspiration is learned by adopting BN and a dynamic learning rate. The BN processing avoids the gradient disappearance problem, the training speed is greatly increased, and the dynamic learning rate enables the network to converge to the optimal solution better.

The invention discloses a video event recognition method based on a deep residual long-short term memory network, which comprises 1) the design of spatial-temporal feature data connection layer, thatis, spatial-temporal feature data is synchronously parsed through a long-short term memory (LSTM) unit and then forms a spatial-temporal feature data connection unit DLSTM (double-LSTM), and the consistency of spatial and temporal information is highlighted; (2) the design of a DU-DLSTM (dual unidirectional DLSTM) structure which expands the width of the network and increases the feature selectionrange; (3) the design of an RDU-DLSTM (residual dual unidirectional DLSTM) module which solves a deeper problem of network gradient disappearance; and 4) the design of a 2C-softmax objective functionwhich reduces the distance within classes while expanding the distance between the classes. The video event recognition method has the advantages that the problem of gradient disappearance is solvedthrough constructing the deep residual network framework, and the video event recognition accuracy is improved by using the consistency fusion of temporal network and spatial network features at the same time.