819 results about "Multi-task learning" patented technology

Presented are systems and methods that provide a unified end-to-end detection pipeline for object detection that achieves impressive performance in detecting very small and highly overlapped objects in face and car images. Various embodiments of the present disclosure provide for an accurate and efficient one-stage FCN-based object detector that may be optimized end-to-end during training. Certain embodiments train the object detector on a single scale using jitter-augmentation integrated landmark localization information through joint multi-task learning to improve the performance and accuracy of end-to-end object detection. Various embodiments apply hard negative mining techniques during training to bootstrap detection performance. The presented are systems and methods are highly suitable for situations where region proposal generation methods may fail, and they outperform many existing sliding window fashion FCN detection frameworks when detecting objects at small scales and under heavy occlusion conditions.

The present invention discloses a multi-task learning convolutional neural network (CNN)-based face attribute analysis method. According to the method, based on a convolutional neural network, a multi-task learning method is adopted to carry out age estimation, gender identification and race classification on a face image simultaneously. In a traditional processing method, when face multi-attribute analysis is carried out, a plurality of times of calculation are required, and as a result, time can be wasted, and the generalization ability of a model is decreased. According to the method of the invention, three single-task networks are trained separately; the weight of a network with the lowest convergence speed is adopted to initialize the shared part of a multi-task network, and the independent parts of the multi-task network are initialized randomly; and the multi-task network is trained, so that a multi-task convolutional neural network (CNN) model can be obtained; and the trained multi-task convolutional neural network (CNN) model is adopted to carry out age, gender and race analysis on an inputted face image simultaneously, and therefore, time can be saved, and accuracy is high.

The invention discloses a power load forecasting method based on a long short term memory (LSTM) neuron network. The power load forecasting method comprises inputting power load data and a region feature factor at a historic moment through an input unit; carrying out training and modeling on the power load data and the region feature factor at the historic moment by means of an LSTM network in order to generate a deep neural network load forecasting model which is a single-layer multi-task deep neural network model or a double-layer multi-task deep neural network model used for power supply load forecasting; forecasting the power load in an area needing to be forecasted by means of the deep neural network load forecasting model, and generating a forecasting result of the power load in the area; and outputting the forecasting result of the power load in the area through an output unit. According to the invention, a multi-task learning power load forecasting model is constructed based on the LSTM network in the deep learning field, power consumption loads in multiple areas can be forecasted accurately, and the forecasting effect is improved.

The invention relates to a scene and target identification method and device based on multi-task learning. The method comprises the steps that pictures containing different scenes and targets are collected as image sample data; the image sample data is subjected to manual label marking, and target class labels and scene class labels are obtained; a multi-layer convolutional neural network model is built, and network initialization is conducted; the image sample data and the corresponding target class labels are adopted for pre-training the built model till convergence, and a target identification model is obtained; based on a multi-task learning technology, network branches are added into a specific layer of the target identification model, random initialization is conducted, and a multi-task network is obtained; the image sample data and the corresponding scene class labels and target class labels are adopted for e-training the multi-task network till convergence, and a multi-task learning model is obtained; new image data is input to the multi-task learning model, and classification results of scene and target identification of images are obtained. Accordingly, the single task identification precision is improved.

The embodiment of the invention discloses a face recognition method and a device. The method comprises the steps of extracting the Haar feature of a current to-be-recognized face image, and detecting the human face area of the to-be-recognized face image by adopting an ADaBoost classifier so as to obtain a face region image; performing the multi-scale feature extraction on the face region image by utilizing a convolution neural network model, and obtaining a feature vector of the face region image; inputting the feature vector, a pre-built legal face database and a preset user similarity threshold value into a multi-task learning model pre-constructed according to a Softmax loss function and a Triplet loss function, and judging whether the to-be-recognized face image is a legal user or not according to the output value of the multi-task learning model. The extracted feature is good in robustness and good in generalization ability. Therefore, not only the face recognition rate improved, but also the accuracy of face recognition is improved. The safety of identity authentication is improved.

The invention discloses a multi-task named entity recognition and confrontation training method for medical field. The method includes the following steps of (1) collecting and processing data sets, so that each row is composed of a word and a label; (2) using a convolutional neural network to encode the information at the word character level, obtaining character vectors, and then stitching withword vectors to form input feature vectors; (3) constructing a sharing layer, and using a bidirection long-short-term memory nerve network to conduct modeling on input feature vectors of each word ina sentence to learn the common features of each task; (4) constructing a task layer, and conducting model on the input feature vectors and the output information in (3) through a bidirection long-short-term network to learn private features of each task; (5) using conditional random fields to decode labels of the outputs of (3) and (4); (6) using the information of the sharing layer to train a confrontation network to reduce the private features mixed into the sharing layer. According to the method, multi-task learning is performed on the data sets of multiple disease domains, confrontation training is introduced to make the features of the sharing layer and task layer more independent, and the task of training multiple named entity recognition simultaneously in a specific domain is accomplished quickly and efficiently.

The invention provides a face attribute recognition method of a deep neural network based on cascaded multi-task learning and relates to the computer vision technology. Firstly, a cascaded deep convolutional neural network is designed, then multi-task learning is used for each cascaded sub-network in the cascaded deep convolutional neural network, four tasks of face classification, border regression, face key point detection and face attribute analysis are learned simultaneously, then a dynamic loss weighting mechanism is used in the deep convolutional neural network based on the cascaded multi-task learning to calculate the loss weights of face attributes, finally a face attribute recognition result of a last cascaded sub-network is used as the final face attribute recognition result based on a trained network model. A cascading method is used to jointly train three different sub-networks, end-to-end training is achieved, the result of face attribute recognition is optimized, different from the use of fixed loss weights in a loss function, a difference between the face attributes present is considered in the invention.

The present invention discloses a method and a device for identifying a reticulate pattern face image based on a multi-task convolutional neural network. The method comprises the steps of: collecting reticulate pattern face image and corresponding clear face image pairs, then using the multi-task convolutional neural network to respectively design object functions based on regression and classification, training a face image reticulate pattern removing model, and finally inputting the reticulate pattern face image into the trained reticulate pattern removing model to obtain a face image without reticulate pattern, thereby performing subsequent face image identification tasks. According to the method, a multi-task learning frame is adopted, the task for restoring a reticulate pattern image to a clear image is expressed as two object functions which are assistant with each other, and the convolutional neural network is utilized to learn complicated nonlinear transformation referred therein. The method not only effectively improves convergence rate during model training, but also can greatly improve image restoration effect and generalization ability, thereby greatly improving identification accuracy rate of the reticulate pattern face image.

The invention discloses a human face attribute prediction method and apparatus based on deep study and multi-task study. The face attribute prediction method mainly comprises: collecting a human face, marking a category corresponding to a plurality of attributes, and forming a training data set; detecting a human face and key points of the human face, and aligning the human face via a plurality of key points; encoding the sequential attributes in the category; constructing a deep neural network; and using the training data set to train the deep neural network; deploying a neural network model obtained via training; and finally, using the neural network model to predict the human face attribute in a picture. According to the human face attribute prediction method provided by the invention, through united training of a plurality of attributes, a plurality of attributes can be predicted simultaneously with only one deep network, and a prediction result is improved obviously.

The present invention discloses a multi-task cascade face aligning method based on deep learning. The method comprises the following steps: employing the convolutional neural network to perform face aligning model training, wherein a main line network comprises 8 convolution layers, 4 normalization layers, 4 activation layers, 4 pooling layers and a full connection layer, and the structure comprises each two convolution layers, then one normalization layer, one activation layer and one pooling layer, and so on; and behind the first three normalization layers and the last one full connection layer, the full connection layer is taken as a branch network for prediction to predict face key points and face attributes. The multi-task cascade face aligning method based on the deep learning adds auxiliary information such as genders, smile or not, having eyes or postures or not in the model training process to realize multi-task learning, and there is prediction output at each two layers of the network so as to realize coarse-to-fine cascade face key-point location detection and improve the robustness of face expression, posture, gender and shielding through face aligning.

The invention discloses a power load prediction system based on a long short term memory neural (LSTM) network. The LSTM network comprises an input layer, a LSTM network layer and an output layer. The system comprises an information receiving module which is used for transmitting the inputted power load data of historical moments and regional feature factors to the input layer; a model establishing module which is used for performing training and modeling on the power load data of the historical moments and the regional feature factors through the LSTM network layer so as to generate a deep neural network load prediction model; a power prediction module which is used for predicting the power load of the region by using the deep neural network load prediction model and generating the power load prediction result of the region through a regressor connected with the LSTM network layer; and a result output module which is used for outputting the power load prediction result of the region through the output layer. The multitask learning load prediction model is constructed based on the LSTM network so that the power load of multiple regions can be accurately predicted and the prediction effect can be enhanced.

The invention discloses a method for 3D reconstruction and texture generation from single-view face based on multi-task learning, belonging to the computer vision field. The method comprises the following steps of: selecting a special viewpoint for rendering a three-dimensional human face model; generating depth map and texture map as truth value data from special viewpoint; designing an integrated learning and coding network based on feature sharing of depth information and texture information; designing the branch decoding network to recover the depth map from the shared features, and recovering the depth map, designing the mutual information with shared features as latent variables to maximize the generation of antagonistic network, and restoring the texture unwrapping map, adjusting the proportion of each task loss function and training the model; interpolating the depth map and using texture map to reconstruct the 3D face mesh model with texture details. The invention utilizes multi-task learning to carry out single-view face three-dimensional reconstruction, texture generation and style migration, and has the advantages of high speed, low cost and the like.

The invention discloses a deep neural network multitask hyper-parameter optimization method. The method comprises: firstly, a data training set of each task being subjected to model training to obtaina multi-task learning network model; secondly, predicting all points in an unknown region, screening candidate points from a prediction result, finally evaluating the screened candidate points, adding the candidate points and target function values of the candidate points into the data training set, and establishing a model, predicting, screening and evaluating again; and so on, until the maximumnumber of iterations is reached, finally selecting a candidate point corresponding to the maximum target function value from the data training set, that is, the hyper-parameter combination of each task in the multi-task learning network model. According to the method, the Gaussian model is replaced by the radial basis function neural network model, and the radial basis function neural network model is combined with multi-task learning and is applied to the Bayesian optimization algorithm to realize hyper-parameter optimization, so that the calculation amount of hyper-parameter optimization isgreatly reduced. The invention further discloses an electronic device and a storage medium.

The invention discloses a deep convolutional neural network-based human face occlusion detection method. The method comprises the steps of performing block segmentation on an input image to obtain a target pre-selected region; constructing a first deep convolutional neural network, training the first deep convolutional neural network comprising a first deep convolutional network and a first multilayer perceptron connected with the first deep convolutional neural network to obtain required parameters, extracting features of the target pre-selected region, and performing classification; predicting the position of a human head through a second multilayer perceptron according to the extracted features; filtering the credibility of a classification type which is the human head and the predicted position of the human head through non-maximum suppression to remove an overlapped duplicate detection box; and obtaining a human head block in combination with original image segmentation, constructing a multi-task learning policy-based second deep convolutional neural network, and judging whether the left eye, the right eye, the nose and the mouth of the human head block are occluded or not. According to the method, the occluded human face can be accurately detected and the specific occluded part of the human face can be judged; and the method is mainly used for crime pre-warning of videos of a camera in front of an automatic teller machine.

The invention discloses an image retrieval method based on multi-task hash learning. Firstly, the deep convolutional neural network model is determined. Secondly, the loss function is designed by using multi-task learning mechanism. Then, the training method of convolutional neural network model is determined, in combination with the loss function, and back propagation method is used to optimize the model. Finally, the image is input to the convolutionalal neural network model, and the output of the model is transformed into hash code for image retrieval. The convolutional neural network modelis composed of a convolutional sub-network and a full connection layer. The convolutional subnetwork consists of a first convolutional layer, a maximum pooling layer, a second convolutional layer, anaverage pooling layer, a third volume base layer and a spatial pyramid pooling layer. The full connection layer is composed of a hidden layer, a hash layer and a classification layer. The training method of the model includes two training methods: a combined training method and a separated training method. The method of the invention can effectively retrieve single tag and multi-tag images, and the retrieval performance is better than other deep hashing methods.

The invention discloses a dialog strategy online realization method based on multi-task learning. According to the method, corpus information of a man-machine dialog is acquired in real time, current user state features and user action features are extracted, and construction is performed to obtain training input; then a single accumulated reward value in a dialog strategy learning process is split into a dialog round number reward value and a dialog success reward value to serve as training annotations, and two different value models are optimized at the same time through the multi-task learning technology in an online training process; and finally the two reward values are merged, and a dialog strategy is updated. Through the method, a learning reinforcement framework is adopted, dialog strategy optimization is performed through online learning, it is not needed to manually design rules and strategies according to domains, and the method can adapt to domain information structures with different degrees of complexity and data of different scales; and an original optimal single accumulated reward value task is split, simultaneous optimization is performed by use of multi-task learning, therefore, a better network structure is learned, and the variance in the training process is lowered.

The invention discloses a multimodal brain network feature fusion method based on multi-task learning, and the multimodal brain network feature fusion method based on the multi-task learning includes the steps of preprocessing the obtained functional magnetic resonance imaging (fMRI) images and diffusion tensor imaging (DTI) images, registrating the preprocessed fMRI image to the standard AAL template, carrying out a fiber tracking for preprocessed DTI images, calculating fiber anisotropy (FA) value, and constructing structure connection matrix through the AAL template. Clustering coefficient of each brain area in a function connection matrix and the structure connection matrix is calculated to be regarded as function features and structure features. As two different tasks, the function features and the structure features assess an optimal feature set by solving the problem of multi-task learning optimization. The method uses information with multiple modalities complementing each other to learn simultaneously and to classify, improves the classification accuracy, solves the problems that a single task feature does not consider the correlation between features, and the fact that only one modality feature is used for pattern classification can bring to insufficient amount of information.

The present application disclosed a method and system for detecting facial landmarks of a face image. The method may comprise extracting multiple feature maps from at least one facial region of the face image and / or the whole face image; generating a shared facial feature vector from the extracted multiple feature maps; and predicting facial landmark locations of the face image from the generated shared facial feature vector. With the present method and system, the facial landmark detection can be optimized together with heterogeneous but subtly related task, so that the detection robustness can be improved through multi-task learning.

The invention provides a deep learning image identification method and a deep learning image identification system used for intelligent driving, and a terminal device. The deep learning image identification system comprises a sharing convolutional network, an area segmentation network, and a target identification network. The area segmentation network is used for area classification processing based on a characteristic graph extracted by the sharing convolutional network, and the target identification network is used for target identification positioning processing based on the characteristic graph extracted by the sharing convolutional network. The sharing convolutional network is monitored by using the area segmentation result acquired by the area segmentation network and the target identification result acquired by the target identification network, and the sharing learning of the area segmentation network and the target identification network is completed. An obvious speed advantage on an aspect of multi-task learning is provided, and by comparing with the two independent networks learning individually, the deep learning image identification system has advantages of less consumed time and high efficiency, and in addition, a convolutional layer repetitive operation problem is effectively prevented, and multi-task detection and multi-task identification are completed.

The invention discloses an online comment fine-grained sentiment analysis method based on multi-task learning. The method comprises the steps that a text representation matrix is sequentially input into a text sentiment feature extractor, a coarse-grained sentiment feature extractor and a fine-grained sentiment feature classifier to obtain a fine-grained sentiment classification result; the text sentiment feature extractor selects a single-layer CNN network to extract text sentiment information from the input text representation matrix to obtain an sentiment representation matrix; wherein thecoarse-grained emotion feature extractor extracts coarse-grained emotion features from an input emotion representation matrix by using a plurality of single-layer CNNs (convolutional neural networks)to obtain coarse-grained emotion feature vectors, and the fine-grained emotion feature classifier performs fine-grained emotion classification on the coarse-grained emotion feature vectors by using amulti-layer full-connection neural network. The method has the advantages of accurate classification and short training time, can be used for emotion analysis of multi-level and multi-granularity Internet user comments, and can be used for personalized recommendation, intelligent search or product feedback.

Multi-Task Multi-View Tracking (MTMVT) is used to visually identify and track an object. The MTMVT employs visual cues such as color, edge, and texture as complementary features to intensity in the target appearance representation, and combines a multi-view representation with a robust multi-task learning to solve feature fusion tracking problems. To reduce computational demands, feature matrices are sparsely represented in a single matrix and then decomposed into a pair of matrices to improve robustness to outliers. Views and particles are further combined based on interdependency and commonality single computational task. Probabilities are computed for each particle across all features and the particle with the greatest probability is selected as the target tracking result.

The invention discloses a text classification method based on a generative multi-task learning model, and the method comprises the steps: alternately decoding a multi-task classification model througha training set according to a multi-label classification task and a hierarchical classification task, obtaining a current-moment semantic feature related to a coding moment feature vector through decoding, and carrying out the training of the current-moment semantic feature; optimizing the multi-task classification model according to the training result and the real label set to obtain an optimized multi-task classification model; inputting data in a to-be-detected set into the optimized multi-task classification model, and respectively obtaining classification labels to finish classification. The semantic association between the multi-label classification model and the hierarchical classification model is constructed through a multi-task mechanism in the training process of the model, sothat the semantic association between two sub-task prediction results can be enhanced, and the purpose of improving the sub-task classification performance is achieved.

The invention discloses an identification method of fine-grained attributes of pedestrians under complex scenes. A classification model is adopted for fine-grained attribute identification on sub-components of a detected pedestrian; association analysis is respectively carried out on identified attributes and a pedestrian gender, and attributes with high correlation are selected for multi-task learning; then convolutional neural network (CNN) models constructed by multi-task learning are trained, and results of the convolutional neural network models with a highest identification rate are selected for the multiple attributes to use the same as a final result; and finally, a gender attribute of the pedestrian is judged according to a customized decision function. The method can realize overall-to-local detection of the pedestrians in the complex scenes, realize more accurate detection and identification of the attributes of pedestrian sub-components, and avoid interference of backgrounds and other information, also solves, at the same time, the problem that detection correctness rates of models on small objects are low, and has higher information accuracy.

The invention discloses a three-dimensional face modeling method and a three-dimensional face modeling printing device based on video streaming and face multi-attribute matching. The method comprises the following steps that: establishing a universal three-dimensional face model library; through a pre-trained multi-task learning deep neural network, carrying out face detection and face key point information extraction; and utilizing the pre-trained multi-task learning deep neural network to carry out face attribute analysis prediction, and combining face key point data with face attribute information to carry out coarse registration with the universal three-dimensional face model library to obtain a universal face model which is most similar to real-time acquisition. The printing device comprises an input unit, a feature point labelling unit, a universal three-dimensional face model library unit, a face multi-attribute classification unit, a face key point optimization unit, a texture refining unit and a printing and outputting unit. The method and the printing device have the advantages of high accuracy, simple implementation way, good user friendliness, high automation degree and the like.

The invention provides a sentence backbone analysis method based on a multi-task depth neural network of character segmentation and named entity recognition, and a system. The invention uses three different bi-directional LSTM neural networks with conditional random fields to separate Chinese character segmentation corpus, Chinese named entity recognition corpus and Chinese sentence backbone analysis corpus for character segmentation, Chinese named entity recognition and sentence backbone analysis respectively, and the output vectors of the three networks are transferred to the multi-task parameter sharing layer network respectively. The multi-task parameter sharing layer network uses the fully connected neural network to splice and train the eigenvectors from the three tasks, and transfers the training results back to the input layer of the bi-directional LSTM neural network. After several cycles of iterative training, the result sequence with sentence backbone tagging information isoutput. The invention adopts the method of combining the artificial neural network based on the depth learning and the multi-task learning of the semantic elements in the sentence, which can improve the system accuracy, the reaction speed and the fault tolerance.

The invention discloses a tampered image detection method based on deep learning, and relates to the field of passive evidence obtaining of images. The tampered image detection method includes the steps: constructing a convolution layer based on multi-scale noise constraint to obtain a high-frequency noise residual error in the image; performing tampered image detection by using a double-flow network; utilizing a multi-task learning method to simultaneously realize classification of whether an image area is tampered and detection and segmentation tasks of the tampered area; when the network isoptimized, extracting the output characteristics of the four parts of the region of interest extraction network, the tampering classification branch, the tampering region detection branch and the tampering region segmentation branch, calculating the error of the network for back propagation, and further adjusting the network parameters, so that the network achieves the optimal solution. The tampered image detection method can identify whether the image is tampered or not, and can accurately detect and segment the tampered area in the tampered image, so that the tampered image is suitable foractual application scenes. The tampered image detection method can detect authenticity of the image through a deep learning method, so that the problem of malicious tampering of the image is solved, and the accuracy and generalization ability of tampering detection are improved.

A login access control system is provided. The login access control system includes a camera configured to capture an input image of a subject purported to be a person and attempting to login to a system to access secure data. The login access control system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task. The login access control system also includes a processor configured to apply the deep learning model to the input image to recognize an identity of the subject in the input image regarding being authorized for access to the secure data and a liveness of the subject. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.

The invention discloses a fast multi-label picture retrieval system and a realization method. The method comprises the following steps: deploying an RPN (Region Proposal Network) for extracting region proposals in a convolutional neural network, extracting region proposal information of pictures, and performing ROI pooling calculation on the region proposal information; after pooling, building a multi-label classification loss function through a fully connected layer according to multi-label information to train the convolutional neural network, and building a weighted three-dimensional loss function to train the convolutional neural network; extracting the hash code of each picture from a picture candidate set through the convolutional neural network after multi-task learning, saving the hash codes to a database, and comparing the hash codes with the hash codes in the database, thus completing picture retrieval. The whole network is trained through multi-task learning of classification and hashing, and therefore, the accuracy of retrieval is ensured. Moreover, the similarity is measured using Hamming distance in the process of retrieval, and the efficiency of retrieval is improved greatly.

The invention belongs to the technical field of internet databases, and particularly relates to a community question and answer system and method based on multi-task learning and an electronic device.The system comprises an answer selection model training module which is used for inputting answer input and question input into a bidirectional long-short memory network for encoding, inputting the encoded answer input and question input into a multi-dimensional attention layer, flattening and connecting an output result, and calculating the loss of a prediction result and a real result; a question classification model training module which is used for inputting questions into a bidirectional long and short memory network for encoding, inputting the encoded questions into a two-layer full connection network, and calculating the loss of a prediction result and a real result through a softmax layer; and a joint training module which is used for unifying the answer selection task and the question text classification task under a loss function for joint training to obtain an answer related to the input question. According to the application, the accuracy of the forum community question and answer system can be improved, and the search efficiency of the user is improved.