Intelligent analysis method for specific graphics contained in a video

Abstract

The application discloses a kind of intelligent analysis methods containing specific graphics in video.It specifically includes the following steps: frame extraction is carried out on movie video file;All specific graphic sample pictures are subjected to gray processing in multi-thread concurrent manner;Specific graphic after gray processing is subjected to feature point extraction using feature point extraction algorithm;The picture extracted from movie video file is subjected to gray processing to extract feature set;For each picture, the similarity of the features is compared with the sample;Sorting is performed using an algorithm, and each picture retains the value of the highest similarity feature point;Multiple movie clips are trained to calculate the feature point distribution with high accuracy, forming a scoring model;The scoring model interval of the initialization configuration tool is configured;The demand video is analyzed using the configured tool, and the score is given according to the evaluation model and the analyzed feature point distribution.The beneficial effects of the application are: improving analysis efficiency, low resource demand, low investment cost, and improving artificial review efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to an intelligent analysis method for specific patterns in a video. BACKGROUND

[0002] For the scene of whether a movie video file contains a specific pattern (such as a dragon mark), if a mainstream method is used, deep learning is used to train a related training sample set through a large number of sample sets, and there may be overfitting and underfitting problems during training. At the same time, the market is a system-level video analysis and processing scheme, which is relatively heavy to deploy. At the same time, it consumes a lot of server resources, and the video analysis efficiency is also relatively low, and it often takes tens of minutes to analyze a video. SUMMARY

[0003] The present application is to overcome the above-mentioned deficiencies in the prior art, and provides an intelligent analysis method for specific patterns in a video with less resources and high analysis efficiency.

[0004] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme:

[0005] An intelligent analysis method for specific patterns in a video, specifically comprising the following steps:

[0006] (1) Frame the movie video file, save one picture per second according to the configured time period;

[0007] (2) Multi-thread and concurrently process the grayscale of all specific pattern sample pictures collected;

[0008] (3) The specific pattern after grayscale processing uses an innovative and optimized feature point extraction algorithm for feature point extraction;

[0009] (4) The pictures extracted from the movie video file are grayscale processed and the feature set is extracted;

[0010] (5) Use multi-thread and concurrent comparison of each picture with the specific pattern sample to compare the similarity;

[0011] (6) Use an algorithm to sort each picture to retain the value of the highest similarity feature point;

[0012] (7) Train multiple movie clips using the above steps to calculate the feature point distribution with high accuracy, and form a scoring model;

[0013] (8) Initialize the scoring model interval configuration of the configuration tool;

[0014] (9) Use the configured tool to analyze the required video, and score according to the evaluation model and the analyzed feature point distribution.

[0015] The method is mainly aimed at the movie video review scene, and is innovatively optimized. In the algorithm level, only pictures in a certain time period are extracted, and then intermediate analysis results are saved. The intermediate analysis results are used to quickly calculate the picture similarity. A lightweight tool is used. Only a few tens of seconds are needed to analyze a video. The efficiency is several tens of times that of deep learning video analysis. The method is easy to install and deploy. A series of algorithms and processes are used for optimization for this specific scene, which greatly improves the efficiency of video analysis. The movie video recognition specific pattern analysis only takes a few tens of seconds. The resource demand of the server is relatively low. A general-purpose server can be used. The server investment cost is relatively low, but the efficiency of manual review is greatly improved. That is, the present application provides a lightweight tool method, which can quickly frame the movie video file, compare and analyze the framed pictures and specific pattern samples (such as dragon marks), evaluate the possibility of containing specific images in the video, and improve the efficiency of manual video review. At the same time, compared with deep learning comparison and analysis, the method needs to deploy fewer resources, but the performance and efficiency are several tens of times that of deep learning comparison and analysis.

[0016] As preferred, in step (2), specifically, first, the system available thread size is acquired and a thread pool is created. The available threads are acquired from the thread pool, and then different picture processing tasks are allocated to different thread resources for simultaneous processing. The weighted average method is used for gray scale processing.

[0017] As preferred, in step (3), the extracted feature point dataset is stored in a temporary file. Since the feature set extraction is time-consuming, the results are directly extracted from the temporary file after the second start. In order to solve the accuracy of the specific pattern scene similarity matching, the feature point extraction algorithm is innovatively optimized on the basis of the general SIFT algorithm for specific pattern scene recognition. Specifically, the key points are filtered by using the Gaussian kernel function when constructing the scale space. The brightness and contrast saturation of the pictures after gray scale processing are adjusted three times. The feature points are extracted according to the optimization algorithm each time. The more stable extreme points are taken for each feature point. Finally, the extracted feature point set is formed.

[0018] As preferred, the optimization algorithm is specifically: the number of layers in each group is adjusted to 8 layers, the DOG local extreme value is adjusted to 18x4, and the key point direction distribution main direction peak value is adjusted to 90% of the direction as the auxiliary direction of the key point.

[0019] As preferred, the more stable extreme points refer to: after adjusting the brightness and contrast saturation three times, the feature points are extracted and matched, and then the feature points are sorted and filtered, and the feature points with a similarity distance less than 0.7 distance are left.

[0020] As preferred, in step (5), specifically: similarity algorithm is used for each picture and specific graphic features, similarity calculation matching is carried out, and records with a similarity distance less than 0.7 are regarded as effective feature point sets.

[0021] As preferred, in step (6), specifically: each frame picture is matched with specific graphic sample pictures in the specific graphic sample set library, the specific graphic sample set has several specific graphics with different appearances, each frame picture has several similarity results after matching calculation, and the highest similarity result is obtained through sorting.

[0022] As preferred, in step (8), the configuration tool refers to a script file, the script file is started, the model score interval calculated in the foregoing is outputted and entered, and then initialization is successful.

[0023] As preferred, in step (9), the scoring rule is: a specific image possibility of 0% is contained when the feature similarity value is less than 150, a specific image possibility of 60%-99% is contained when the value is between 150 and 300, and a specific image possibility of 99.9% is contained when the value is greater than 300.

[0024] The method has the advantages that: the method is easy to install and deploy, a series of algorithms and processes are used for optimization in the specific scene, the efficiency of video analysis is greatly improved, the resource demand of a server is relatively low, the investment cost of the server is relatively low, and the efficiency of manual review is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a method flowchart of the application;

[0026] Figure 2 is a gray processing diagram of a Long label;

[0027] Figure 3 is a Long label feature point extraction schematic diagram;

[0028] Figure 4 is a Long label matching effect diagram. DETAILED DESCRIPTION

[0029] The application will be further described below in combination with the drawings and specific embodiments.

[0030] As Figure 1 described in the embodiments, an intelligent analysis method for specific graphics in a video specifically comprises the following steps:

[0031] (1) Extract frames from the movie video file and save one image per second according to the configured time period. This configured time period refers to the parameter passed for startup execution. For example, passing 30 120 at startup means starting from the 30th second of the video and cutting to the 120th second.

[0032] (2) Perform grayscale processing on all the collected specific graphic sample images concurrently using multiple threads; specifically: first, obtain the available thread size of the system and create a thread pool, obtain available threads from the thread pool, and then assign different image processing tasks to different thread resources for simultaneous processing. The grayscale processing uses a weighted average method for grayscale conversion. The weighted average method can be directly called using OpenCV. Example of grayscale processing for a specific graphic, taking the dragon logo as an example, such as... Figure 2 As shown.

[0033] (3) Feature points are extracted from specific graphics after grayscale processing using an innovative and optimized feature point extraction algorithm. The extracted feature point dataset is transferred to a temporary file. Since feature set extraction is time-consuming, the results are extracted directly from the temporary file after the second startup. In order to solve the accuracy of similarity matching for specific graphic scenes, the feature point extraction algorithm has been innovatively optimized for specific graphic (dragon mark) scene recognition based on the general SIFT algorithm. Specifically, when extracting key points, Gaussian kernel function is used for filtering when constructing the scale space. The brightness and contrast saturation of the grayscale processed image are adjusted three times. Each time, feature points are extracted according to the optimized algorithm. The more stable extreme point is selected for each feature point. The more stable extreme point refers to the feature point extraction and matching after adjusting the brightness and contrast saturation three times. Then, the feature points are sorted and filtered, leaving feature points with a similarity distance of less than 0.7. This is how the extracted feature point set is finally formed, such as Figure 3 As shown. The optimization algorithm is as follows: the number of layers in each group (usually 3~5) is adjusted to 8 layers per group, the DOG local extremum detection is adjusted from 9×2 to 18×4, and the direction of the key point, which accounts for 80% of the peak value of the main direction, is adjusted to 90% as the auxiliary direction of the key point.

[0034] (4) Perform grayscale processing on the images extracted from the movie video file and extract the feature set;

[0035] (5) Use multi-threaded concurrency to compare the similarity of each image with the features of a specific graphic sample; specifically: use a similarity algorithm to calculate and match the similarity between each image and the features of a specific graphic, and the records with a similarity distance of less than 0.7 are the set of valid feature points. Among them, the similarity algorithm uses the KNN algorithm in OpenCV for matching calculation.

[0036] (6) Use an algorithm to sort the images, and retain the value of the highest similarity feature point for each image. Specifically, each frame of an image will be matched with a specific image sample in a specific image sample library. There are several specific images with different appearances in the specific image sample library. After each frame of an image is matched with the image sample, there are several similarity results. Sort these results and take the result with the highest similarity.

[0037] (7) Use the above steps to train multiple movie clips to calculate the feature point distribution with relatively high accuracy and form a scoring model;

[0038] (8) Initialize the scoring model interval configuration of the configuration tool; the configuration tool refers to a script file. Start this script file, then output the previously calculated model scoring interval for input, and then execute it to initialize successfully.

[0039] The following is an example of a specific configuration range:

[0040] def get_match_score(good_count):

[0041] score = 0

[0042] if good_count < 150: # 0%

[0043] score = 0

[0044] elif good_count >= 150 and good_count < 300: # 60%-99%

[0045] score = round( (good_count - 150) / (300-150) * (0.99-0.6) + 0.6 , 2) *100 # Keep two decimal places

[0046] elif good_count >= 300: # 99.9%

[0047] score = 99.99

[0048] return str(score) + "%"

[0049] (9) using the configured tool to analyze the demand video, and scoring according to the evaluation model and the analyzed feature point distribution; the scoring rule is that the feature similarity value less than 150 is 0% containing specific image possibility, 150~300 is 60%-99% containing specific image possibility, and greater than 300 is 99.9% containing specific image possibility, such as Figure 4

[0050] The method mainly aims at the movie video review scene, is innovated and optimized, only extracts pictures in a certain time period at the algorithm level, then the intermediate analysis result is saved, the picture similarity is quickly calculated using the intermediate analysis result, a lightweight tool is used, and only dozens of seconds are needed to analyze a video, the efficiency is dozens of times of deep learning video analysis. The method is easy to install and deploy, a series of algorithms and process optimization are used for the specific scene, the efficiency of video analysis is greatly improved, the movie video recognition specific pattern analysis only needs dozens of seconds. The resource demand of the server is relatively low, a general server can be used, the server investment cost is relatively low, but the efficiency of manual review is greatly improved. That is, the application provides a lightweight tool method, which can quickly frame the movie video file, compare and analyze the framed pictures and specific pattern samples (such as dragon marks), evaluate the possibility of containing specific images in the video, improve the efficiency of manual video review, and compared with deep learning comparison and analysis, the method needs less resources to be deployed, but the performance and efficiency are dozens of times of deep learning comparison and analysis.​

Claims

1. An intelligent analysis method for videos containing specific graphics, characterized in that, Specifically, the steps include the following: (1) Extract frames from the movie video file and save one image per second according to the configured time period; (2) Perform grayscale processing on all the collected specific graphic sample images concurrently using multiple threads; (3) The specific graphics after grayscale processing are extracted using an innovative and optimized feature point extraction algorithm. The extracted feature point dataset is transferred to a temporary file. Since feature set extraction is time-consuming, the results are extracted directly from the temporary file after the second startup. In order to solve the accuracy of the similarity matching of specific graphic scenes, the feature point extraction algorithm has been innovatively optimized for specific graphic scene recognition based on the general SIFT algorithm. Specifically, when extracting key points, Gaussian kernel function is used for filtering when constructing the scale space. The brightness and contrast saturation of the grayscale processed image are adjusted three times. Each time, feature points are extracted according to the optimized algorithm. The more stable extreme point is taken for each feature point, so as to finally form the extracted feature point set. The optimization algorithm is as follows: the number of layers in each group is adjusted to 8 layers in each group, the local extreme value of DOG is adjusted to 18×4, and the direction of key point is adjusted to the direction of the main direction peak of 90% as the auxiliary direction of the key point. The more stable extreme point refers to: after adjusting the brightness and contrast saturation three times, feature point extraction and matching are performed, and then the feature points are sorted and filtered to leave the feature points with a similarity distance of less than 0.

7. (4) Perform grayscale processing on the images extracted from the movie video file and extract the feature set; (5) Use multi-threaded concurrency to compare the similarity of each image with the features of a specific graphic sample; (6) Sort the images using an algorithm, and retain the value of the feature point with the highest similarity for each image; (7) Use the above steps to train multiple movie clips to calculate the feature point distribution with high accuracy and form a scoring model; (8) Initialize the scoring model range configuration of the configuration tool; (9) Use the configured tools to analyze the required videos and score them based on the evaluation model and the distribution of feature points analyzed.

2. The intelligent analysis method for videos containing specific graphics according to claim 1, characterized in that, In step (2), the specific steps are as follows: First, obtain the available thread size of the system and create a thread pool. Obtain available threads from the thread pool and then assign different image processing tasks to different thread resources for simultaneous processing. The weighted average method is used for grayscale processing.

3. The intelligent analysis method for videos containing specific graphics according to claim 1, characterized in that, In step (5), specifically: for each image, a similarity algorithm is used to perform similarity calculation and matching with specific graphic features. Records with a similarity distance of less than 0.7 are considered as the set of valid feature points.

4. The intelligent analysis method for videos containing specific graphics according to claim 1, characterized in that, in In step (6), specifically: each frame of the image will be matched with a specific graphic sample image in a specific graphic sample set library. There are several specific graphics with different appearances in the specific graphic sample set. After each frame of the image is matched with the specific graphic sample image, there are several similarity results. These results are sorted and the result with the highest similarity is selected.

5. The intelligent analysis method for videos containing specific graphics according to claim 1, characterized in that, In step (8), the configuration tool refers to a script file. Start this script file, then input the previously calculated model score range, and then execute it to initialize successfully.

6. The intelligent analysis method for videos containing specific graphics according to claim 1, characterized in that, In step (9), the scoring rules are as follows: if the feature similarity value is less than 150, it is 0% likely to contain a specific image; if it is between 150 and 300, it is 60%-99% likely to contain a specific image; and if it is greater than 300, it is 99.9% likely to contain a specific image.

Images