684 results about "Stochastic gradient descent" patented technology

The invention discloses an automatic insect image identification method based on a depth convolutional neural network. The method comprises the following steps: (1), collecting an original image and carrying out pretreatment to form a training set, and calculating a mean value image of the training set; (2), constructing a depth convolutional neural network; (3), collecting a sub image block randomly from a training sample of the training set and carrying out pre training on the depth convolutional neural network by using the sub image block; (4), training the depth convolutional neural network by using the training set and combining a mini-batch-based random gradient descent algorithm; and (5), carrying out pretreatment on a to-be-measured insect image to form a test sample, and using the trained depth convolutional neural network to identify the test sample after subtracting the mean value image of the training set from the test sample. Therefore, the identification precision is high; the identification types are diversified; the insect within-class change robustness is enhanced; and the insect inter-class similarity sensitivity is high.

The invention discloses a graph-based vision SLAM (simultaneous localization and mapping) method. According to the method, the matching relation between an image and visual feature can be obtained based on the natural feature probability vector representation of the image, and the relative pose between two interframes can be calculated by utilizing the space geometry relation of images. Data association of visual odometry is obtained by utilizing the corresponding relation of continuous images, so that all constraints in an image sequence can be obtained. The camera relative pose is taken as a node in a map, the space constrained relation of image interframes is taken as an edge, so that an estimated track map based on the camera relative pose is constructed. Finally, a maximum likelihood method is employed for optimizing the map, and optimized pose estimation is obtained through a random gradient descent method. Related experiments are performed in the laboratory environment based on the provided method, also the moving track of a robot is displayed, and the validity of the algorithm is confirmed.

The invention discloses a full convolution neural network (FCN)-based monocular image depth estimation method. The method comprises the steps of acquiring training image data; inputting the training image data into a full convolution neural network (FCN), and sequentially outputting through pooling layers to obtain a characteristic image; subjecting each characteristic image outputted by a last pooling layer sequentially to amplification treatment to obtain a new characteristic image the same with the dimension of a characteristic image outputted by a previous pooling layer, and fusing the twocharacteristic images; sequentially fusing the outputted characteristic image of each pooling layer from back to front so as to obtain a final prediction depth image; training the parameters of the full convolution neural network (FCN) by utilizing a random gradient descent method (SGD) during training; acquiring an RGB image required for depth prediction, and inputting the RGB image into the well trained full convolution neural network (FCN) so as to obtain a corresponding prediction depth image. According to the method, the problem that the resolution of an output image is low in the convolution process can be solved. By adopting the form of the full convolution neural network, a full-connection layer is removed. The number of parameters in the network is effectively reduced.

The invention discloses a semi-supervised image classification method based on a generative adversarial network, which mainly solves the problems that the existing unsupervised learning classificationprecision is low and semi-supervised learning needs a large number of accurate labels, and comprises the following implementation steps of: 1) selecting and downloading a standard image training sample and a test sample; 2) setting relevant parameters of network supervised learning, and establishing a generative adversarial network consisting of a generator network, a discriminator network and anauxiliary classifier in parallel; 3) training the generative adversarial network by using a random gradient descent method; and 4) inputting the test sample to be classified into the trained generative adversarial network model, and outputting the category of the image to be detected. The method improves the image classification precision of unsupervised learning, can obtain a very good image classification effect on a sample set only containing a small amount of accurate annotation samples, and can be used for target classification in an actual scene.

The invention discloses an image significance detection method which uses confrontation training to generate a convolution neural network model, which belongs to the field of computer vision and image processing. The method comprises the steps of data preprocessing, network structure, suitable parameter selecting, and training with a random gradient descending method and an impulse unit. According to data preprocessing, a large amount of collected data and labels are preprocessed. According to network structure, a network structure and a specific kernel function are designed. Suitable parameters including learning rate, a momentum factor and the number of images inserted into the network each time are selected. The random gradient descending method and the impulse unit are used for training to reduce the possibility of network over-fitting. According to the invention, a significance map can be accurately acquired.

The invention relates to a remote-sensing image object detection method based on deep learning. The method comprises: using remote-sensing images to construct a related data set, namely an image dataset after classification and labeling on the remote-sensing images and class labels generated through labeling work; building a panchromatic sharpening model based on generative adversarial networks (GAN); building an object detection model based on a deep convolutional neural network, and carrying out end-to-end training on the model through methods of back propagation, random gradient descent and the like; and carrying out end-to-end testing on the built model. The method of the invention has the advantage of high accuracy.

A method and a device for training a DNN model includes: at a device including one or more processors and memory: establishing an initial DNN model; dividing a training data corpus into a plurality of disjoint data subsets; for each of the plurality of disjoint data subsets, providing the data subset to a respective training processing unit of a plurality of training processing units operating in parallel, wherein the respective training processing unit applies a Stochastic Gradient Descent (SGD) process to update the initial DNN model to generate a respective DNN sub-model based on the data subset; and merging the respective DNN sub-models generated by the plurality of training processing units to obtain an intermediate DNN model, wherein the intermediate DNN model is established as either the initial DNN model for a next training iteration or a final DNN model in accordance with a preset convergence condition.

New and efficient protocols are provided for privacy-preserving machine learning training (e.g., for linear regression, logistic regression and neural network using the stochastic gradient descent method). A protocols can use the two-server model, where data owners distribute their private data among two non-colluding servers, which train various models on the joint data using secure two-party computation (2PC). New techniques support secure arithmetic operations on shared decimal numbers, and propose MPC-friendly alternatives to non-linear functions, such as sigmoid and softmax.

The invention discloses a face recognition method based deep learning and a face recognition device thereof and electronic equipment. The method comprises the steps that a convolutional neural network model is constructed, and the convolutional neural network model comprises a first convolution unit, a first pooling layer, multiple convolution combinations, a second pooling layer and full connection layers which are connected in turn, wherein the first convolution unit comprises a first convolution layer, a batch normalization layer and an excitation function layer, the excitation function layer simultaneously uses a ReLU function and a NReLU function to act as the excitation function, and the adjacent convolution combinations are connected by the short circuit layer of the residual network; and the convolutional neural network model is trained, the training data are inputted to the convolutional neural network model and training is performed by using the stochastic gradient descent method, and the last full connection layer is removed out of the trained convolutional neural network model and then only forward propagation is performed so as to act as the face feature data required for face recognition. ReLU + NReLU are used as the excitation function so that the computational burden can be reduced, the accuracy can be guaranteed, the model size can be reduced and the operation speed can be enhanced.

The invention relates to a fine granularity vehicle multi-property recognition method based on a convolutional neural network and belongs to the technical field of computer visual recognition. The method comprises the steps that a neural network structure is designed, including a convolution layer, a pooling layer and a full-connection layer, wherein the convolution layer and the pooling layer areresponsible for feature extraction, and a classification result is output by calculating an objective loss function on the last full-connection layer; a fine granularity vehicle dataset and a tag dataset are utilized to train the neural network, the training mode is supervised learning, and a stochastic gradient descent algorithm is utilized to adjust a weight matrix and offset; and a trained neural network model is used for performing vehicle property recognition. The method can be applied to multi-property recognition of a vehicle, the fine granularity vehicle dataset and the multi-propertytag dataset are utilized to obtain more abstract high-level expression of the vehicle through the convolutional neural network, invisible characteristics reflecting the nature of the to-be-recognizedvehicle are learnt from a large quantity of training samples, therefore, extensibility is higher, and recognition precision is higher.

The invention discloses a method and a device for acquiring knowledge graph vectoring expression. The method comprises the following steps of labeling an entity, existed in and belonging to a knowledge graph, in a given auxiliary text corpus by utilization of an entity labeling tool according to a to-be-processed knowledge graph so as to obtain an entity-labeled text corpus; constructing a co-occurrence network comprising words and entities on the basis of the text corpus so as to relate text information of the auxiliary text corpus to entity information of the knowledge graph, and then learning to obtain a text context embedded expression; respectively modeling the embedded expression of the entity and relation in the knowledge graph according to the text context embedded expression so as to obtain an embedded expression model of the knowledge graph; training the embedded expression model by utilization of a stochastic gradient descent algorithm so as to obtain the embedded expression of the entity and relation in the knowledge graph. The method and the device disclosed by the invention have the advantages that not only can the expression capability of the relation be improved, but also the problem of insufficient expression effect caused by sparseness of the knowledge graph can be effectively solved.

The invention relates to an image classification method capable of effectively preventing a convolutional neural network from being overfit. The image classification method comprises the following steps: obtaining an image training set and an image test set; training a convolutional neural network model; and carrying out image classification to the image test set by adopting the trained convolutional neural network model. The step of training the convolutional neural network model comprises the following steps: carrying out pretreatment and sample amplification to image data in the image training set to form a training sample; carrying out forward propagation to the training sample to extract image features; calculating the classification probability of each sample in a Softmax classifier; according to the probability yi, calculating to obtain a training error; successively carrying out forward counterpropagation from the last layer of the convolutional neural network by the training error; and meanwhile, revising a network weight matrix W by SGD (Stochastic Gradient Descent). Compared with the prior art, the invention has the advantages of being high in classification precision, high in rate of convergence and high in calculation efficiency.

The invention discloses an identification method of emotional tendency of network comment texts and a convolutional neutral network model. The method comprises the steps as follows: grabbed network comment texts constitute a data set; word segmentation and text preprocessing are performed; all words subjected to text preprocessing are trained, and word vector representation of all words is obtained; the convolutional neutral network model is constructed and trained on a training set selected from the data set, and network parameters are updated with a back-propagating algorithm; in each step of training, noise is added to word vectors of an input layer for construction of adversarial samples, adversarial training is performed, and network parameters are updated with a random gradient descent algorithm; a classification model is obtained through repeated iteration to identify the emotional tendency of the network review texts. The convolutional neutral network model is used in the method and comprises the input layer, a convolution layer, a pooling layer and a classification layer. The adversarial samples can be classified correctly and the identification accuracy is improved.

The invention discloses an infrared target instance segmentation method based on feature fusion and a dense connection network, and the method comprises the steps: collecting and constructing an infrared image data set required for instance segmentation, and obtaining an original known infrared tag image; Performing image enhancement preprocessing on the infrared image data set; Processing the preprocessed training set to obtain a classification result, a frame regression result and an instance segmentation mask result graph; Performing back propagation in the convolutional neural network by using a random gradient descent method according to the prediction loss function, and updating parameter values of the convolutional neural network; Selecting a fixed number of infrared image data training sets each time and sending the infrared image data training sets to the network for processing, and repeatedly carrying out iterative updating on the convolutional network parameters until the convolutional network training is completed by the maximum number of iterations; And processing the test set image data to obtain average precision and required time of instance segmentation and a finalinstance segmentation result graph.

The invention discloses a driver behavior identification method based on a multi-scale attention convolutional neural network. The method comprises the following steps: (1) shooting an image data setfor driver behavior identification; (2) carrying out data enhancement on the shot driver behavior data set and simultaneously bringing samples obtained by enhancement into training data; (3) constructing a neural network model which comprises three modules, namely a multi-scale convolution module, an attention module and a classification module; (4) training a multi-scale attention convolutional neural network; using a Pytorch open source tool i for building a network model, and using a random gradient descent method for optimizing network parameters; and (5) testing the multi-column convolutional neural network. According to the invention, the multi-scale model and the attention mechanism are introduced into the driver behavior identification task to extract the behavior fine-grained feature representation with the distinction degree, so that the driver behavior identification accuracy can be further improved.

The invention discloses a neural network feature learning method based on image self coding, which belongs to the technical field of feature learning and image retrieval. Firstly, a segmentation training image set corresponding to a training image set is constructed through a segmentation label of a multi-label image data set, weights of a convolutional neural network and a self coding neural network are then initialized, a stochastic gradient descent method is used for training the self coding neural network, and an implicit variable of a segmentation image corresponding to each training sample is extracted and normalized; and then, the implicit variable serves as a training target corresponding to an original training set image, the convolutional neural network is trained, a feature vector corresponding to each image in a test set image library is extracted, and through calculating Euclidean distances between feature vectors of a query image and each image in the image library and arranging the distances in an sequence from small to large, a similar image retrieval result is obtained. Thus, features extracted from the trained neural network achieve perfect retrieval effects in a multi-label retrieval task.

Determining semantically equivalent text or questions using hybrid representations based on neural network learning. Weighted bag-of-words and convolutional neural networks (CNN) based distributed vector representations of questions or text may be generated to compute the semantic similarity between questions or text. Weighted bag-of-words and CNN based distributed vector representations may be jointly used to compute the semantic similarity. A pair-wise ranking loss function trains neural network. In one embodiment, the parameters of the system are trained by minimizing a pair-wise ranking loss function over a training set using stochastic gradient descent (SGD).

The invention proposes a dangerous driving behavior real-time detection method based on deep learning, and the method comprises the steps: building a front car image collection system, collecting a training sample, carrying out the manual marking, and building a dangerous driving behavior data set; proposing a spatial pyramid pooling convolution depth confidence normalized classification network (SPP-CDBRNet) model according to the characteristics of the dangerous driving behavior data set based on a deep learning method; carrying out the preprocessing of the built data set, carrying out the training of the SPP-CDBRNet model through a random gradient descent method with a momentum and the data set after preprocessing, and obtaining an SPP-CDBRNet which precisely recognize whether there is a dangerous driving behavior (behaviors of using a cellphone and smoking during driving); carrying out the detection of a front car image through the determined SPP-CDBRNet model, and achieving the real-time detection of the dangerous driving behavior. The method can effectively improve the detection precision of dangerous driving behaviors, is good in instantaneity and mobility, and is good in application prospect.

The embodiment of the invention provides an acoustic model training method and device. The method includes the steps of establishing a deep neural network model initial model; dividing voice training data into N non-intersecting data subsets, renewing the deep neural network model initial model for each data subset by means of a stochastic gradient descend algorithm to obtain N deep neural network model sub models, wherein N is an integer larger than or equal to 2; integrating the N deep neural network model sub models to obtain a deep neural network model intermediate model, and judging that the deep neural network model intermediate model is a trained acoustic model when the deep neural network model intermediate model conforms to preset convergence conditions. By means of the acoustic model training method and device, training efficiency of the acoustic model is improved, and performance of voice recognition is not reduced.

The invention discloses a text classification method based on feature information of characters and terms. The method comprises the steps that a neural network model is utilized to perform character and term vector joint pre-training, and initial term vector expression of the terms and initial character vector expression of Chinese characters are obtained; a short text is expressed to be a matrixcomposed of term vectors of all terms in the short text, a convolutional neural network is utilized to perform feature extraction, and term layer features are obtained; the short text is expressed tobe a matrix composed of character vectors of all Chinese characters in the short text, the convolutional neural network is utilized to perform feature extraction, and Chinese character layer featuresare obtained; the term layer features and the Chinese character layer features are connected, and feature vector expression of the short text is obtained; and a full-connection layer is utilized to classify the short text, a stochastic gradient descent method is adopted to perform model training, and a classification model is obtained. Through the method, character expression features and term expression features can be extracted, the problem that the short text has insufficient semantic information is relieved, the semantic information of the short text is fully mined, and classification of the short text is more accurate.

The invention discloses a real-time dense monocular simultaneous localization and mapping (SLAM) method based on an online learning depth prediction network. The method comprises: optimization of a luminosity error of a minimized high gradient point is carried out to obtain a camera attitude of a key frame and the depth of the high gradient point is predicted by using a trigonometric survey methodto obtain a semi-dense map of a current frame; an online training image pair is selected, on-line training and updating of a CNN network model are carried out by using a block-by-block stochastic gradient descent method, and depth prediction is carried out on the current frame of picture by using the trained CNN network model to obtain a dense map; depth scale regression is carried out based on the semi-dense map of the current frame and the predicted dense map to obtain an absolute scale factor of depth information of the current frame; and with an NCC score voting method, all pixel depth prediction values of the current frame are selected based on two kinds of projection results to obtain a predicted depth map, and Gaussian fusion is carried out on the predicted depth map to obtain a final depth map. In addition, the invention also provides a corresponding real-time dense monocular SLAM system based on an online learning depth prediction network.

The invention discloses a deep neural network for the fine recognition of vehicle attributes and a training method thereof. The network comprises a depth residual network, a feature migration layer, aplurality of all-connection layers, a plurality of loss calculation units, and a plurality of parameter updating units. The depth residual network is used for carrying out feature extraction on an input image to obtain a feature image. The feature migration layer comprises a plurality of feature migration units and is used for enabling each of all feature migration units to be adapted to specifictasks according to the features shared by all attribute identifying tasks. The plurality of all-connection layers correspond to the branches of all attribute identifying tasks and are connected withthe feature migration layer so as to obtain feature vectors corresponding to all attribute identifying tasks. The plurality of loss calculation units correspond to the branches of all attribute identifying tasks and are respectively connected with the all-connection layers. The plurality of loss calculation units are used for calculating the loss of a loss function by adopting cross entropies as multiple classifiers. The plurality of parameter updating units correspond to the attribute identifying tasks and are connected with the loss calculation units. The parameter updating units are used for returning the loss based on the random gradient descent optimization algorithm, and updating parameters. According to the invention, various fine vehicle attributes can be identified at the same time by adopting only one neural network.

The invention discloses a motion history image and convolutional neural network-based behavior identification method. The method comprises the following steps of S1, obtaining an input original videoimage, and processing the input original video image through a motion history image-based behavior sequence feature extraction method; and S2, performing behavior identification on a local motion history image by adopting a deep convolutional neural network-based method to obtain a behavior type classifier, and finally outputting a behavior identification result through the behavior type classifier. The motion history image is calculated in an original video sequence, so that the to-be-processed information amount is reduced and key time-space information in behavior identification is extracted; and by taking the motion history image as an input, a deep convolutional neural network is established, then the network is trained by utilizing a stochastic gradient descent method and a Dropout policy, and finally behavior type identification is realized. The method can be effectively applied to online real-time behavior identification.

The invention provides a vehicle type recognition method based on convolutional neural network. Based on a feature extraction module and a vehicle type recognition module, the vehicle type recognition method comprises the following steps: establishing a neural network for vehicle type recognition by designing a convoluting and pooling layer, a full connection layer and a classifier, wherein the convoluting and pooling layer and the full connection layer are used for extracting vehicle type features, and the classifier is used for performing classified vehicle type recognition; training the neural network by using a database containing different vehicle type features; the training way is that labeled data are learned in a supervised manner, and regulating weighting parameter matrixes and offsets by a stochastic gradient descent method; obtaining the well-trained weighting parameter matrixes and offsets in layers, correspondingly assigning the weighting parameter matrixes and the offsets to the layers in the neural network, wherein the neural network has the functions of extracting and recognizing the vehicle type features. By the vehicle type recognition method, the convolutional neural network and the vehicle type recognition are creatively combined; different from the conventional vehicle type recognition method, by the vehicle type recognition method provided by the invention, the accuracy of the vehicle type recognition can be obviously improved.

The invention discloses a deep learning-based text keyword extraction method. The method comprises the following steps of: firstly training a recurrent neural network model, wherein the used training data comprise a large amount of texts and keywords thereof, and the training target is maximizing text-based condition probability of the keywords; converting each text and the keyword thereof into word vectors, inputting the word vectors into the recurrent neural network model and updating network parameters by using a random gradient descent method; and after the model training is finished, converting a section of text, the keyword of which is to be extracted, into a word vector, inputting the word vector into the trained recurrent neural network model so as to generate the keyword of the section of text. According to the method disclosed by the invention, the extraction of text keywords is realized by learning an end-to-end model through data driving; and compared with the traditional statistics and linguistics-based method, the method disclosed by the invention is stronger in adaptability, and can be used for obtaining different models according to different training data so as to extract keywords according to the requirements of specific fields.

The invention provides a human body action recognition method fusing an attention mechanism and a space-time diagram convolutional neural network under a security scene, and the method comprises the steps: firstly carrying out the random division of an obtained human body action analysis data set under the security scene, and dividing the human body action analysis data set into a training set anda verification set; secondly, performing data enhancement processing on video data of the training set and the verification set; carrying out key frame screening on the obtained and enhanced data setby utilizing an attention mechanism; transcoding and labeling the screened key frame video by using a human body posture estimation model framework to prepare for training a human body motion detection and recognition model; and finally, constructing a space-time skeleton map convolutional neural network model, training by using the training set, carrying out network parameter weight optimizationby using random gradient descent, and predicting the accuracy of the neural network model by using the verification set. The method not only can enlarge the original action data volume, but also canenhance the robustness of the model, thereby improving the final action recognition accuracy.

The invention provides a fine-grained human face image fast retrieval method based on deep learning. The method comprises the following steps that a deep convolution neural network model is established, and a loss layer used for calculating a loss function is added to each branch layer of the deep convolution neural network model; parameters of the deep convolution neural network model are initialized; a data set is constructed, and according to a preset proportion, pictures in the data set are randomly divided into a training set, a test set and a verification set; learning parameters of the depth convolution neural network model are set; the deep convolution neural network model is trained, and the parameters of the deep convolution neural network model are updated through a stochastic gradient descent and back propagation algorithm; the trained deep convolution neural network model is tested and is specifically subjected to a coarse-grained test and a fine-grained test, human face retrieval results are obtained, and then the image retrieval efficiency and the image retrieval precision are improved.

An embodiment of the invention relates to a deep learning-based road traffic sign detection and identification method. The method comprises the steps of calculating convolutional features of a plurality of layers for training set data by utilizing a convolutional neural network; training a region-of-interest suggestion network by utilizing the convolutional features; and training a classification layer and a regression layer of an output layer by utilizing a random gradient descent method based on the convolutional features and candidate boxes obtained by the trained region-of-interest suggestion network.