Method, system and electronic device for segmenting news anchor video data set

Abstract

Traditional methods for segmenting video data sets mainly rely on manual annotation, which is a time-consuming and labor-intensive process and is difficult to adapt to the needs of large-scale video data processing. The present invention provides a method, system and electronic device for segmenting news anchor video data sets, including: obtaining news anchor videos submitted by users; quickly screening the news anchor videos to obtain a face database containing news anchors and interviewed audiences; filtering the face database containing news anchors and interviewed audiences, filtering out the face data of interviewed audiences, and obtaining a face database containing only news anchors; based on the face database containing only news anchors, using a deep model to screen the news anchor videos frame by frame to obtain accurate news anchor video segmentation results. This enables high-precision, fully automatic news anchor video segmentation.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method for automatically segmenting a news anchor video data set based on deep learning. Background Art

[0002] With the rapid development of information technology, the news media industry is undergoing unprecedented changes. Standard pronunciation and high-definition video have become standard. The processing and analysis of news anchor video data has become an important part of automated news production and content management. Automatic segmentation of news video datasets is crucial for indexing, retrieval, recommendation, and subsequent in-depth analysis of video content. Traditional video dataset segmentation methods mainly rely on manual annotation, which is time-consuming and labor-intensive, and difficult to adapt to the needs of large-scale video data processing.

[0003] With the development of deep learning in the field of computer vision, the application scenarios of target detection and face recognition have become more and more numerous, and the recognition accuracy has become higher and higher. However, if face detection is used directly to annotate news videos, there are many problems. For example, the background music in the news video is confusing, and the interview audience sometimes occupies the main part of the interview screen. If only face detection is used, meaningless interview screens will be mixed into the segmentation results. Summary of the invention

[0004] Aiming at the problems existing in the prior art of complex news anchor video segmentation scenarios, the present invention proposes a method for automatically segmenting news anchor video data sets based on deep learning, so as to achieve high-precision and fully automatic news anchor video segmentation.

[0005] To achieve the above object, the present invention provides a method for automatically segmenting a news anchor video data set based on deep learning, comprising the following steps:

[0006] Get news anchor videos submitted by users;

[0007] Rapidly screening the news anchor video to obtain a face database containing the news anchor and interview audience;

[0008] Filtering the face database containing the news anchor and the interview audience, filtering out the face data of the interview audience, and obtaining a face database containing only the news anchor;

[0009] According to the face database containing only news anchors, the news anchor video is screened frame by frame using a deep model to obtain an accurate news anchor video segmentation result.

[0010] News anchor videos are often more than 30 minutes long. If every frame is detected, the time efficiency is low, so it is necessary to quickly screen the news anchor videos.

[0011] Specifically, face detection is performed on the video frames every 50 frames. Once a face is detected, a deep learning model is used to extract the face information to obtain the first face information, which is then compared with the historical information in the face database for rapid screening.

[0012] If a match is found, the frame number will be recorded.

[0013] If it is not matched, the number of frames will be recorded after giving a new character name.

[0014] After obtaining the face database of the first rapid screening, it is necessary to conduct a detailed frame-by-frame screening of the news anchor video. At the same time, in order to ensure high efficiency, face detection is performed on the video every 25 frames to obtain the second face information.

[0015] If the face is matched, it will be compared with the historical information in the face database. If it is matched, the next 24 frames will be matched in sequence and the number of frames will be recorded; if it is not matched, the current loop will be jumped out.

[0016] After carefully screening the news anchor's video frame by frame, we can assume that all anchors whose data matches the face database have been recorded in the news anchor's video, and the video can be segmented according to the number of frames in which each face in the face database appears.

[0017] Based on the above-mentioned method for automatically segmenting a news anchor video data set based on deep learning, in order to better realize the present invention, a system for automatically segmenting a news anchor video data set based on deep learning is further proposed, including: a video acquisition module, a fast screening module, a filtering module, and a frame-by-frame screening and segmentation module;

[0018] The video acquisition module includes obtaining the news anchor video submitted by the user;

[0019] The rapid screening module includes, performing rapid screening of the news anchor videos to obtain a face database including the news anchor and interviewing viewers;

[0020] The filtering module includes filtering the face database containing the news anchor and the interview audience according to the rules, filtering out the face data of the interview audience, and obtaining a face database containing only the news anchor;

[0021] The frame-by-frame screening and segmentation module includes, based on the face database containing only news anchors, using a deep model to perform frame-by-frame screening on the news anchor video to obtain accurate news anchor video segmentation results.

[0022] Based on the above-mentioned method of automatically segmenting a news anchor video dataset based on deep learning, in order to better realize the present invention, an electronic device is further proposed, including a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the above-mentioned method of automatically segmenting a news anchor video dataset based on deep learning is implemented.

[0023] Beneficial effects:

[0024] After rapid screening, the database needs to be screened according to certain rules, with the purpose of removing occasional interview audiences and invalid data;

[0025] After obtaining a fast screened face database, it is necessary to conduct a detailed screening of the news anchor videos, with the goal of accurately dividing the news anchor videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Flowchart of a method for automatically splitting news anchor video datasets based on deep learning;

[0027] Figure 2 Schematic diagram for rapid screening of news anchor videos based on deep learning based on automatic split news anchor video dataset method;

[0028] Figure 3 Accurate frame-by-frame screening diagram for a news anchor video based on deep learning-based automatic split news anchor video dataset method;

[0029] Figure 4 Schematic diagram of filtering for a face database based on a deep learning-based automatic splitting news anchor video dataset method. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0032] This embodiment provides a method of automatically splitting the video data set of news anchors based on deep learning, and is illustrated below in conjunction with the embodiments and the accompanying drawings.

[0033] Combination Figure 1The present invention proposes a method for automatically segmenting a news anchor video data set based on deep learning, and the method comprises the following steps:

[0034] Get news anchor videos submitted by users;

[0035] Rapidly screening the news anchor video to obtain a face database containing the news anchor and interview audience;

[0036] The face database containing the news anchor and the interviewed audience is filtered, and the face data of the interviewed audience is filtered out to obtain a face database containing only the news anchor; based on the face database containing only the news anchor, the news anchor video is screened frame by frame using a deep model to obtain an accurate news anchor video segmentation result.

[0037] Specifically, users need to collect the required news anchor videos, which are often news programs of a certain TV station. The number of anchors in such videos is limited and the broadcast quality is high, which is recorded as V = {v1, v2, ...}, where v represents a single news anchor video. Because each news anchor video is usually about 30 minutes long, according to the standard of 25 frames per video, it would be very inefficient to process each video frame by frame, because about 25 × 30 × 60 = 45,000 frames need to be processed, and because there are often many invalid pictures such as interviews and screen switching in the news, it is necessary to use fast screening to remove redundant information from the news video.

[0038] according to Figure 2 After obtaining the news anchor video v, we quickly filter and sample the video once every 50 frames, denoted as f. After obtaining f, we need to use the deep learning model to detect the portrait. If and only if there are at most two detection results in the picture, we think that the current picture frame may contain a standard anchor picture, denoted as f t , otherwise the current frame is discarded. The purpose of this is to quickly detect whether the picture contains a portrait. t After that, the deep learning model is used to extract the 128-dimensional facial feature vector of the person in the current picture, recorded as d. Since there are at most two people in the picture, there are at most two sets of d in the same picture frame. After obtaining d, it is necessary to compare the distance with the existing facial feature vectors in the historical face database. The formula is:

[0039] scores=1 / (1+e^(-((√(Σ(v1_i-v2_i) 2 )-mean) / std))).

[0040] When the score is greater than 0.9, the two facial feature vectors are considered to be the same person, that is, the face is matched, and the frame number in which the current portrait appears is recorded; if there is no match, a name is assigned to the current portrait according to the serial number and recorded in the historical face database.

[0041] It is particularly noted that there are many deep models for detecting human portraits in frames and extracting facial features, such as using face recognition, etc. In the embodiments of the present invention, no restrictions are imposed on the acquisition method.

[0042] according to Figure 4 ,After obtaining the face database after rapid screening, it is recorded as the face database B containing news anchors and interview audiences, it is necessary to filter the face data in the database in order to filter out invalid portraits, which are often occasional interview audiences.

[0043] There are two filtering rules:

[0044] (1) The portrait must appear for at least 10 seconds, i.e. 10×frame;

[0045] (2) The portrait must appear at least multiple times, which means that the interval between the last frame and the first frame of the portrait must be more than half the length of the video, that is,

[0046] The portrait database after screening according to the above rules is recorded as the face database containing only news anchors B ` , which can be considered as a face database that has been screened and does not contain redundant frames.

[0047] according to Figure 3 In the implementation of the embodiment of the present application, the news anchor video is screened frame by frame, the news anchor video v is sampled once at an interval of 25 frames, and the video frames are detected according to the method in the quick screening. When the video frame that meets the conditions is detected, it is necessary to sample and detect the subsequent 24 frames of the current frame, and record the position where the frame appears.

[0048] After obtaining the final face database, the frame positions of all portraits that meet the standards have been recorded, and the news anchor video can be accurately segmented to obtain the final segmentation result.

[0049] This embodiment provides a system for automatically segmenting a news anchor video data set based on deep learning, including: a video acquisition module, a fast screening module, a filtering module, and a frame-by-frame screening and segmentation module;

[0050] The video acquisition module includes acquiring a news anchor video submitted by a user;

[0051] The quick screening module includes: performing quick screening on the news anchor video to obtain a face database containing the news anchor and the interview audience;

[0052] The filtering module includes filtering the face database containing the news anchor and the interview audience according to the rules, filtering out the face data of the interview audience, and obtaining a face database containing only the news anchor;

[0053] The frame-by-frame screening and segmentation module includes, based on the face database containing only news anchors, using a deep model to perform frame-by-frame screening on the news anchor video to obtain accurate news anchor video segmentation results.

[0054] This embodiment provides an electronic device, including a memory and a processor; the memory stores a computer program; when the computer program is executed on the processor, the above-mentioned method of automatically segmenting a news anchor video data set based on deep learning is implemented.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person having ordinary knowledge in the art can still modify the technical solutions recorded in the aforementioned embodiments, or replace part or all of the technical features therein with equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A method for automatically segmenting a news anchor video dataset based on deep learning, characterized in that: include: Get news anchor videos submitted by users; Rapidly screening the news anchor video to obtain a face database containing the news anchor and interview audience; The face database containing the news anchor and the interviewed audience is filtered, and the face data of the interviewed audience is filtered out to obtain a face database containing only the news anchor; based on the face database containing only the news anchor, the news anchor video is screened frame by frame using a deep model to obtain an accurate news anchor video segmentation result.

2. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 1 is characterized in that: In the quick screening, face detection is performed on the news anchor video every 50 frames.

3. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 2 is characterized in that: Based on the detected face, a deep learning model is used to extract the first face information and match it with the information in the historical face database.

4. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 3 is characterized in that: If the same first facial information is matched in the historical face database, the number of frames in which it appears is recorded.

5. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 3 is characterized in that: If the same first facial information is not matched in the historical facial database, a new character name is assigned to the first facial information and the number of frames in which it appears is recorded.

6. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 1, characterized in that: The frame-by-frame screening includes performing face detection on the news anchor video once every 25 frames to obtain second face information; The second facial information is matched with the data in the facial database containing only news anchors until the second facial information does not match the data in the facial database containing only news anchors.

7. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 6, characterized in that: If the second facial information matches the data in the historical facial database, the next 24 frames are matched sequentially and the number of frames is recorded.

8. The method for automatically segmenting a news anchor video data set based on deep learning according to claim 1, characterized in that: The deep model is used to screen the news anchor video frame by frame to obtain accurate news anchor video segmentation results, including: matching the second facial information and the data in the face database containing only the news anchor, recording all matching anchors in the news anchor video, and performing video segmentation according to the number of frames in which each face in the final face database appears.

9. A system for automatically segmenting news anchor video datasets based on deep learning, characterized in that: include: Video acquisition module, quick screening module, filtering module, frame-by-frame screening and segmentation module; The video acquisition module includes acquiring a news anchor video submitted by a user; The quick screening module includes: performing quick screening on the news anchor video to obtain a face database containing the news anchor and the interview audience; The filtering module includes filtering the face database containing the news anchor and the interview audience according to the rules, filtering out the face data of the interview audience, and obtaining a face database containing only the news anchor; The frame-by-frame screening and segmentation module includes, based on the face database containing only news anchors, using a deep model to perform frame-by-frame screening on the news anchor video to obtain accurate news anchor video segmentation results.

10. An electronic device, characterized in that: It comprises a memory and a processor; a computer program is stored on the memory; when the computer program is executed on the processor, the method for automatically segmenting a news anchor video data set based on deep learning as described in any one of claims 1 to 8 is implemented.

Images