Intelligent training method and device fusing semantics and image features, equipment and medium

Abstract

The embodiment of the invention discloses an intelligent training method and device fusing semantics and image features, equipment and a medium. The method comprises the steps of performing natural language processing on category hard label information of a training sample to generate corresponding category soft label information, sending the training sample into a convolutional neural network layer and a full connection layer in an image classification model for processing, performing activation through an excitation function to obtain each category prediction probability corresponding to the training sample, and adjusting and updating the parameters of the convolutional neural network layer and the full connection layer according to the category soft label information corresponding to the training sample and the corresponding prediction probability of each category. By adopting the scheme of the invention, a smoother label can be adaptively generated by using natural language information in a category in an image classification task, a soft label integrated with semantics is combined with a prediction probability for parameter updating, and meanwhile, a new mode integrated with the natural language information and the image information is used for model updating; and the robustness of the model can be well improved.

Description

technical field [0001] Embodiments of the present invention relate to the field of deep learning technology, and in particular to an intelligent training method, device, device and medium that integrate semantic and image features. Background technique [0002] Deep learning can achieve good results in tasks such as target object recognition, target detection, and instance segmentation. However, when the sample size is scarce, model training based on conventional deep learning algorithms is prone to overfitting, which is not conducive to the proper performance of the model. [0003] In the related scheme, when the model is trained, it is improved by introducing data enhancement and label smoothing to increase the robustness of the model. However, data enhancement requires manual design and selection of different data enhancement methods, which leads to an increase in training time; common label smoothing treats other categories equally, without considering the differences b...

AI Technical Summary

Problems solved by technology

However, data enhancement requires manual design and selection of different data enhancement methods, which leads to an increase in training time; common label smoothing treats other categories equally, without considering the differences between different categories, and some even need to give object calibration frames, which requires more data sets. high

Images