Mass video file storage system based on cloud computation, analysis method and system thereof

Abstract

The invention relates to the video monitoring field, and discloses a mass video file storage system based on cloud computation, a mass video file analysis method and a mass video file system. Slicing is performed accurately, so that decoding for video data analysis is complete and effective. According to the mass video file storage system provided by the invention, the method comprises the following steps: a client inquires management information of a video file, and slices the video file according to the management information and generates slice information, wherein a slice comprises at least one data block, each data block comprises at least one video frame sequence, each video frame sequence in each data block is complete; the client uploads the slice information to a main node and submits assignments to the main node; the main node decomposes the assignments and distributes tasks and slice information to task nodes; each task node executes the distributed task, acquires and analyzes the data block; each task node acquires metadata obtained by analysis, concludes and outputs.

Description

technical field [0001] The invention relates to the field of video monitoring, in particular to a massive video file analysis technology. Background technique [0002] At present, Hadoop, a distributed computing model based on cloud computing, is more suitable for text-oriented data processing, and can achieve better results in the field of web search engines. However, the inventors of the present invention have found that the distributed computing model hadoop based on cloud computing has the following problems when performing intelligent analysis on video data: [0003] First of all, in Hadoop's distributed computing, the processing of big data requires dividing large blocks of data into many small pieces of data, and processing each small piece of data independently. If the video file is sliced, Hadoop does not pay attention to the semantics and relevance of the internal data of the video file, but just slices it as ordinary binary data, which is likely to cause inabilit...

AI Technical Summary

Problems solved by technology

Hadoop is more suitable for batch processing. It can achieve better results for input-output-intensive and non-computation-intensive situations, but for computing-intensive problems, Hadoop is not very effective.

This is because hadoop is developed based on java, but the execution efficiency of java programs is not high, and it is slow to start the java virtual machine. For memory management, java adopts a lazy allocation strategy and cannot actively control memory allocation.

Moreover, the intelligent analysis of video data is a computing-intensive operation that requires very high efficiency, and there will be frequent and large amounts of memory operations. Using Hadoop to implement video analysis often results in slow analysis calculations and rapid memory growth but cannot be released quickly. The problem

[0005] Finally, the distributed computing processing objects in hadoop are generally based on the data stored on the distributed file system (Hadoop Distributed File System, referred to as "HDFS"), and many optimizations of its distributed scheduling are done for the storage characteristics of HDFS , and HDFS is too tightly coupled, not suitable for mass video data storage

[0006] To sum up, the existing cloud computing technology is not suitable for intelligent analysis of massive video data

Images