Image-text fusion classification method and device, equipment, medium and product

Abstract

The invention discloses an image-text fusion classification method and device, equipment, a medium and a product, and the method comprises the steps: carrying out the feature extraction of a commodity image of a commodity object, and obtaining a corresponding image coding vector, wherein the picture coding vector comprises a single-row vector corresponding to a plurality of primitives formed by segmenting the commodity picture; performing feature extraction on the abstract text of the commodity object to obtain a corresponding text coding vector; performing multi-level coding and decoding on an image-text splicing vector formed by splicing the image coding vector and the text coding vector based on a multi-head attention mechanism to obtain an image-text fusion vector; and carrying out classification according to the image-text fusion vector, and judging a classification label of the commodity object according to a classification result. According to the method, feature-level deep semantic interaction can be carried out according to the commodity picture and the abstract text of the commodity object, classification is carried out after fusion of two types of information is deepened, and the classification accuracy can be improved.

Description

technical field [0001] The present application relates to the field of e-commerce information technology, and in particular to a graphic-text fusion classification method and corresponding devices, computer equipment, computer-readable storage media, and computer program products. Background technique [0002] In the application scenario of the e-commerce platform, the classification of product objects based on product information occurs frequently. For example, it is necessary to identify different types of security properties of commodity objects launched by merchant instances, and identify whether they are not for sale. Or sometimes it is necessary to use classification means to perform commodity identification on the commodity information of the commodity object to identify whether the commodity therein belongs to the target item. Such downstream tasks will rely on the deep semantic feature extraction of the product information of the product object, and on this basis, ...

AI Technical Summary

Problems solved by technology

[0004] The reason is that in the various neural network models in the prior art that assist e-commerce platforms to classify commodity objects, the comprehensive vectors they rely on for classification have not truly realized the deep semantics of image information and text information. Therefore, it is always difficult for the classification effect of the corresponding classification model to break through its own inherent threshold, and it is prone to missed recognition, misclassification, difficult training convergence, training Various unfavorable situations such as a long time

Images