Short video scene classification method and system, electronic equipment and storage medium

Abstract

The invention discloses a short video scene classification method and system, electronic equipment and a storage medium, and the method comprises the following steps: a short video image processing step: carrying out the frame extraction of a short video image, obtaining a plurality of video frames, and obtaining an image data set with a plurality of pieces of image information corresponding to the plurality of video frames according to the plurality of video frames; a scene classification and recognition step: performing recognition and classification on multiple pieces of picture information in the image data set through a recognition network after metric learning to obtain the image probability of each piece of picture information; and an image probability processing step: calculating the scene probability of each scene category of the short video according to the image probability of each piece of picture information. According to the invention, on the basis of image-level scene classification, the problem that the difference of feature levels of the same category is too large is solved by using the idea of metric learning, and the final classification effect is ensured.

Description

technical field [0001] The present invention relates to the technical field of video processing, in particular to a short video scene classification method, system, electronic equipment and storage medium combined with metric learning. Background technique [0002] With the rapid development of science and technology, communication technology has developed rapidly, and the current communication technology has developed to the fifth generation of communication---5G. The rapid development of 5G technology has brought a new way of Internet information dissemination --- short video. Short videos have become the most abundant, fastest and most popular way to disseminate information in current social life. The number of monthly active users of Douyin short videos has exceeded 200 million. Various other short video apps are also popular. For example, Kuaishou, Tencent Weishi, and WeChat video accounts also have a very rich number of monthly active users. With such a large numbe...

AI Technical Summary

Problems solved by technology

[0005] Second, different from ordinary scene recognition, in short video scenes, the content of surrounding environment information is relatively scarce, and most of them are characters occupying the main area, such as image 3 , and the difference in information under the same scene category is also relatively obvious. For example, it is the same scene in the live broadcast room, but the difference between each image is relatively large. It is even more difficult to achieve accurate classification when there is not much scene information. like Figure 4

But the disadvantage of this method is: the video-based method can only give the subject scene of this video, and generally ignores other scenes that appear in the short video with a relatively short duration, resulting in a result that does not meet the requirements

However, the disadvantage of this method is: under image-level conditions, the subject in the image is occluded by people, resulting in less information about the surrounding environment, and the difference in images within the category is large, so the recognition accuracy may not be high.

Images