Method and apparatus for fusion of multimodal information of live videos
A technology of live video and fusion methods, applied in character and pattern recognition, instruments, computer parts, etc., can solve problems such as not very strong logical correlation, not very strict video processing time requirements, etc.
Active Publication Date: 2018-10-26
BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF10 Cites 4 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
This analysis method is not very strict on the video processing time, and the logical correlation between the analysis results of each modal information is not very strong.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0032] Based on the background technology, it can be seen that the existing automatic segmentation technology of video news generally needs to combine multiple modal information, and obtain the final analysis result of video news through comprehensive analysis of the result sequence of each modal information. However, this method is suitable for the situation where the complete video news has been obtained, and the offline comprehensive analysis is performed to obtain the final split results. However, for the live video, since the live video is distributed frame by frame, the complete video cannot be obtained at one time, so the above method is not applicable to the segmentation of the live video.
[0033] For live video, real-time analysis is required for each frame, and the corresponding results need to be quickly analyzed for subsequent segmentation. Moreover, live video analysis has many dimensions, and it is often necessary to integrate factors of different dimensions in ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The present application provides a method and apparatus for fusion of multimodal information of live videos. As that first and the second modal information is detected respectively in real time for the live videos, the detected first modal information result and the detected second modal information result are written into the first result sequence and the second result sequence respectively, andthen the basic unit fusing with result sequence is constructed according to the first modal information result, one of the basic units comprises a first modal information result and a modal information sequence. Finally, based on the first modal information result of the basic unit, the second modal information result which is provided with frame overlapping with the first modal information resultis written into the modal information sequence of the basic unit of the fusion result sequence to realize the fusion of the first modal information and the second modal information. Therefore, the analysis results of multiple pieces of modal information of live videos are fused in a short time by the method, which lays a key foundation for content analysis of live videos.
Description
technical field [0001] The present application relates to the technical field of multimedia data processing, and in particular to a method and device for fusing multimodal information of live video. Background technique [0002] News videos contain a large amount of latest information, which is of great value to video websites and news applications. Video websites or news applications need to split and launch the entire news that is broadcast every day, so that users can click and watch each piece of news they are interested in. [0003] Due to the large number of TV stations in the country, there are various local stations in addition to the satellite TV stations. If it is necessary to segment all the news, it will take a lot of manpower to segment. At the same time, due to the timeliness of news, the requirements for the segmentation speed of news videos are also very strict, so it brings greater pressure to manual segmentation. News is broadcast in large quantities at a ...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06K9/62G06K9/00
CPCG06V20/49G06V20/40G06F18/25
Inventor 刘楠
Owner BEIJING QIYI CENTURY SCI & TECH CO LTD