Media screen content comparison method and system based on video features and similarity calculation

Abstract

The invention provides a media screen content comparison method and system based on video features and similarity calculation, and the method comprises the steps: obtaining a video image, and carrying out the preprocessing of the video image; carrying out key frame extraction on the preprocessed video image to obtain a key frame sequence; extracting an SURF (Speeded Up Robust Feature) with scale invariance corresponding to each frame from the key frame sequence; constructing a K-D tree according to the SURF features of the video feature points, and performing two-stage feature matching based on the K-D tree and RANSAC; according to a feature matching result, performing statistical index calculation on a matching point number and then outputting a similarity index frame by frame; and according to the similarity indexes, comparing the similarities between the frames of the video images, and formulating corresponding early warning schemes according to different similarity levels. According to the invention, the comprehensive evaluation model is established through the video content comparison technology, real-time monitoring and early warning are realized, the inconsistency between the playing content and the preset content can be found in time, the propagation of error information is avoided, and the green and healthy urban light environment is ensured.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a media screen content comparison method and system based on video features and similarity calculation. Background Art

[0002] As the urbanization process accelerates, urban media screens have multiple functions such as public information dissemination, advertising, and city image and cultural display. In order to ensure the timeliness and accuracy of video content, the management and playback of video content is a technical problem that needs to be solved. Usually, the video content review of urban screens during playback mostly relies on manual work, which consumes a lot of time, manpower, and material resources, and is not enough to meet the needs of efficient processing of video data. When using video content management technology, when faced with diverse video content, the use of keyword search methods requires a high level of experience for staff, and sometimes it is impossible to accurately capture the video content, resulting in inaccurate content comparison results or missing results.

[0003] With the development of urban life and night tourism economy, urban building lighting, LED media screens and other light environment displays have become important carriers of urban cultural propaganda and commercial promotion. In terms of security monitoring, there is an increasing demand for real-time monitoring of the consistency between the playback content and the preset content. Traditional monitoring methods often rely on manual inspections, which are not only inefficient, but also difficult to ensure the accuracy and comprehensiveness of real-time monitoring.

[0004] In the feature-based video content detection technical solution, firstly, the continuous video frames are segmented into different video scenes according to the structure of the video data, key frames are selected in each video scene to represent the features of the video, and the video content is detected by comparing the similarity of the key frames. Compared with the present invention, this technical solution focuses more on the segmentation of video content, the extraction of key frames, and the representation and retrieval of video features.

[0005] In the object similarity comparison and retrieval technical solution based on video analysis, typical reference points of the object are obtained from the image group containing the object, and the similarity of the object is calculated based on the typical reference points. Compared with the present invention, this technology focuses on the extraction of typical reference points of video objects, the calculation of similarity, and the object retrieval based on similarity.

[0006] In the video information extraction method based on video features, the method extracts video feature data from the video program stored on the server side, and compares and analyzes the video feature data of the currently playing video program collected by the terminal to provide corresponding playback information. For example, the patent document with application number CN201310354549.6 discloses a method and system for providing television information based on video features. The playback terminal of the patent document is a television terminal, and the playback terminals are inconsistent and there are large differences in playback scenarios such as lighting conditions and viewing distance. Summary of the invention

[0007] In view of the defects in the prior art, an object of the present invention is to provide a media screen content comparison method and system based on video features and similarity calculations.

[0008] According to the present invention, a media screen content comparison method based on video features and similarity calculation is provided, comprising:

[0009] Step S1: Acquire a video image and pre-process the video image;

[0010] Step S2: extracting key frames from the preprocessed video image to obtain a key frame sequence;

[0011] Step S3: extracting scale-invariant SURF features corresponding to each frame from the key frame sequence;

[0012] Step S4: constructing a KD tree according to the SURF features of the video feature points, and performing two-stage feature matching based on the KD tree and RANSAC;

[0013] Step S5: Based on the result of the feature matching, statistical indicators are calculated for the matching points and similarity indicators are output frame by frame;

[0014] Step S6: According to the similarity index, the similarity between each frame of the video image is compared, and corresponding early warning plans are formulated according to different similarity levels.

[0015] Preferably, the preprocessing includes enhancing image contrast, highlighting image details and edges, and removing image noise on the basis of retaining image edges and details.

[0016] Preferably, the step S2 comprises:

[0017] Step S2.1: obtaining a candidate key frame sequence according to the color feature differences between the original video frames;

[0018] Step S2.2: obtaining a key frame sequence by comparing the structural similarity between adjacent frames in the key frame candidate sequence using a structural similarity method;

[0019] Step S2.3: Determine whether the number of current key frames meets the preset minimum number of key frames. If so, determine that the current key frames are valid; if not, directly extract the corresponding number of key frames from the original video in an equal-length manner.

[0020] Preferably, the step S4 comprises:

[0021] In the process of searching for the nearest neighbor, given a feature point to be matched, starting from the root node of the constructed KD tree, the search direction is determined by comparing the coordinate values ​​of the query point and the partition point. During the search process, the nearest neighbor point currently encountered is recorded and continuously updated. When the search reaches a leaf node, the nodes on the path are backtracked to check whether there are closer neighbors and calculate the Euclidean distance between the feature points.

[0022] After using KNN for rough matching, RANSAC is used again for precise matching.

[0023] Preferably, the step S5 comprises: performing calculations based on matching point related information, and performing weighted summation based on Euclidean calculation on all calculated indices, thereby obtaining a similarity index of the current frame;

[0024] The matching point related information includes the matching point ratio, matching point direction consistency, and matching pair confidence;

[0025] The matching point ratio refers to the ratio of the matching points obtained after RANSAC exact matching to the total number of feature points in the standard video key frame. The higher the ratio, the more similar the two frames of video are in terms of feature points.

[0026] The matching point direction consistency refers to the degree of consistency of directions between matching point pairs. For each matching point pair, the difference in the direction angle or direction vector of the matching point is calculated. The smaller the difference, the higher the consistency of the local texture or structure of the two frames of video.

[0027] The matching pair confidence refers to the probability or confidence level that each matching point pair is correctly matched. The confidence of the matching pair is estimated using the inlier ratio in the RANSAC algorithm. The higher the confidence value, the more accurate the matching result.

[0028] Preferably, the similarity levels include a crimson level, a red level, a yellow level and a green level;

[0029] According to the brightness and contrast of the video frames, the maximum threshold and the minimum threshold are established. When the content difference between the two frames of video exceeds the set maximum threshold, it is judged as a dark red level; if the difference is between the minimum threshold and the maximum threshold, it is judged as a red level; when the difference is close to but slightly lower than the minimum threshold, it is judged as a yellow level; if the difference is lower than the set minimum threshold, it is judged as a green level.

[0030] According to the present invention, a media screen content comparison system based on video features and similarity calculation is provided, comprising:

[0031] Module M1: acquiring a video image and preprocessing the video image;

[0032] Module M2: extract key frames from the preprocessed video image to obtain a key frame sequence;

[0033] Module M3: extracting scale-invariant SURF features corresponding to each frame from the key frame sequence;

[0034] Module M4: construct a KD tree based on the SURF features of the video feature points, and perform two-stage feature matching based on the KD tree and RANSAC;

[0035] Module M5: Based on the result of the feature matching, the matching points are statistically calculated and the similarity index is output frame by frame;

[0036] Module M6: According to the similarity index, the similarity between each frame of the video image is compared, and corresponding early warning plans are formulated according to different similarity levels.

[0037] Preferably, the preprocessing includes enhancing image contrast, highlighting image details and edges, and removing image noise on the basis of retaining image edges and details.

[0038] Preferably, the module M2 comprises:

[0039] Module M2.1: Obtaining a candidate sequence of key frames based on the color feature differences between the original video frames;

[0040] Module M2.2: obtaining a key frame sequence by comparing the structural similarity between adjacent frames of the key frame candidate sequence through a structural similarity system;

[0041] Module M2.3: Determine whether the number of current key frames meets the preset minimum number of key frames. If so, determine that the current key frames are valid; if not, directly extract the corresponding number of key frames from the original video in an equal-length manner.

[0042] Preferably, the module M4 comprises:

[0043] In the process of searching for the nearest neighbor, given a feature point to be matched, starting from the root node of the constructed KD tree, the search direction is determined by comparing the coordinate values ​​of the query point and the partition point. During the search process, the nearest neighbor point currently encountered is recorded and continuously updated. When the search reaches a leaf node, the nodes on the path are backtracked to check whether there are closer neighbors and calculate the Euclidean distance between the feature points.

[0044] After using KNN for rough matching, RANSAC is used again for precise matching.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The present invention can use a content monitoring and early warning system to predict the display of the light environment, establish a comprehensive evaluation model through video content comparison technology, realize real-time monitoring and early warning, and can promptly discover the inconsistency between the played content and the preset content, avoid the spread of erroneous information, and ensure the green and healthy urban light environment.

[0047] 2. The present invention is highly flexible and scalable, and can adapt to the display needs of urban light environments of different sizes and types, providing more efficient support for urban cultural publicity and commercial promotion.

[0048] 3. The present invention uses machine learning, visual recognition, comparison of real-time images with preset content, and continuous learning and judgment to accurately compare and detect anomalies in the content played on the city media screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0050] Figure 1 It is a schematic diagram of the working method of the present invention;

[0051] Figure 2 It is a schematic diagram of the key frame extraction process based on video temporal and spatial redundant information and structural characteristics in the present invention;

[0052] Figure 3 This is a schematic diagram of extracting SURF features with scale invariance in the present invention. DETAILED DESCRIPTION

[0053] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0054] The present invention uses a series of feature extraction algorithms based on the field of video analysis to extract key feature information of the video itself from the visual level and feature level, such as brightness distribution, color histogram, shape contour, texture pattern, object trajectory, etc. After these feature information are integrated into the spatial relationship and time series information between features, they constitute the fingerprint features of the video content to represent this video; then, the extracted video features or fingerprint information are quickly similarity calculated to efficiently find out whether the content is consistent, similar or different with the video to be compared. Based on this, for the content of the urban media large screen, without human intervention, it can be quickly identified whether the content is played in compliance with regulations. If there is infringing content or playback failure, early warning measures or processing measures can be taken in time to ensure the correctness of the content played on the large screen.

[0055] The present invention uses visual recognition technology to monitor in real time the large-screen playback content affected by ambient light and resolution, captures and identifies abnormal images, uses a machine learning model to extract color, texture, structure and other features of the real-time image, and calculates similarity with the features of the preset content. Through continuous learning and judgment, in order to adapt to the ever-changing playback content and environment, and iteratively update the model, the accuracy and robustness of anomaly detection are improved. In terms of difference comparison and early warning, an early warning mechanism is set by setting a reasonable similarity threshold. For example, when the similarity between the real-time image and the preset content is lower than the threshold, a corresponding early warning is triggered, wherein the early warning information may include similarity scores and screenshots, locations and time of the abnormal content, etc., to facilitate subsequent management personnel or processing personnel to quickly locate and repair.

[0056] Embodiment 1

[0057] According to the media screen content comparison method based on video features and similarity calculation provided by the present invention, candidate frames can be quickly obtained based on the underlying features of the video; video key frame sequences are screened out based on structural features, and weights are assigned to each frame; Surf features of each frame are calculated, and the weights are combined to construct a video fingerprint vector. Figure 1 As shown, including:

[0058] Step S1: Acquire a video image and preprocess the video image. The preprocessing includes enhancing image contrast, highlighting image details and edges, and removing image noise on the basis of retaining image edges and details. Specifically, the acquired video image is subjected to adaptive histogram equalization to enhance the image contrast, and image sharpening technology is used to highlight the image details and edges, thereby improving the image resolution and clarity. In view of the problem of heavy graininess in large-screen playback in cities, morphological filtering techniques such as corrosion, expansion, opening and closing operations are used to remove image noise on the basis of retaining image edges and details. In order to overcome the influence of ambient light changes on subsequent image feature extraction, an adaptive threshold is set for the overall brightness and contrast of the processed image.

[0059] Step S2: Extract key frames from the preprocessed video image to obtain a key frame sequence. Key frame extraction based on video temporal and spatial redundant information and structural features, such as Figure 2 As shown, step S2 includes:

[0060] Step S2.1: Obtain a candidate key frame sequence based on the color feature differences between the original video frames. Use the RGB color space to calculate the color histogram of each frame, and calculate the difference in color histogram features between video frames to quickly form a candidate key frame sequence.

[0061] Step S2.2: Obtain a key frame sequence according to the structural feature differences between the frames of the key frame candidate sequence. That is, the structural similarity between adjacent frames of the candidate key frames is judged, thereby reducing the excessive redundancy of key frames. Specifically, the structural similarity Structure Similiary is a method for image quality evaluation. In the present invention, the covariance comparison part of the structure in the original SSIM method is taken out separately for the selection of key frames.

[0062] Using covariance as a measure of structural similarity, σ xy is the correlation coefficient of image blocks x and y, that is, the covariance of x and y, and the calculation formula is:

[0063]

[0064] The previous frame of the candidate key frame sequence is taken as the original image, and the adjacent next frame is taken as the test image to calculate the structural similarity component between the two. The corresponding image blocks x and y at the same position in the two images are regarded as two major input signals, and their structural similarity components s(x, y) are calculated:

[0065]

[0066] C=((KL) 2 / 2),K<<1,L∈(0,255],σx , σ y are the variances of x and y respectively;

[0067] If the component value s(x, y) is small, it means that the content information between the two is not very different, and they do not need to be retained as key frames at the same time. Only one of the frames can be extracted as the optimized key frame.

[0068] Step S2.3: Determine the validity of the key frames in the key frame sequence by judging the number of key frames. Whether the number of key frames meets the preset minimum number of key frames. For example, for some videos with relatively smooth content, the key frame may be 0. In this case, a few more video frames may be directly extracted at time intervals as key frames.

[0069] By analyzing the spatial redundancy and temporal redundancy in the video data, without considering the shot segmentation, the key frames are directly extracted from a video using the frame difference method, thereby forming a key frame sequence of the video. The provisions for key frames in the video played on the city's large screen include: the grayscale value is within a certain range, which allows viewers to subjectively perceive the video content; the time sequence of the key frames is consistent with the time sequence of the frames in the original video; there can be appropriate redundancy to ensure that there is a key frame every other time. First, the candidate key frames are extracted based on color features, and then the extracted key frames are optimized based on structural features, that is, the structural similarity between adjacent frames of the candidate key frames is judged, thereby reducing the excessive redundancy of the key frames. The structural similarity method is a judgment method derived from image quality evaluation. It is an indicator for measuring the similarity of two images. The closer the value is to 1, the closer the quality of the two images is. The structural similarity theory believes that natural image signals are highly structured, that is, there is a strong correlation between pixels, especially the closest pixels in the spatial domain. This correlation contains important information about the structure of objects in the visual scene. After the candidate keyframes are extracted based on color features and the keyframes are optimized based on structural features, it is determined whether the number of keyframes meets the requirements. If the number of keyframes in a video is 0, the corresponding number of keyframes will be directly extracted from the original video in an equal-length manner. In other words, the number of keyframes cannot be 0.

[0070] Step S3: extracting scale-invariant SURF features corresponding to each frame from the key frame sequence. The extraction method includes using a Speeded Up Robust Features extraction algorithm. In order to speed up the extraction speed, the sampling interval is preferably set to 2 during filtering to reduce the amount of calculation.

[0071] In order to meet the scale invariance (i.e., not affected by image scaling, rotation, and translation) and robustness (not sensitive to noise and able to resist changes in brightness and contrast to a certain extent) of the video fingerprint, and to ensure the feasibility of large-scale detection, a scale-invariant SURF feature detection is used to minimize the computational complexity of the video fingerprint algorithm. The scale factor is added when calculating the feature points. While fully improving the reliability of the video fingerprint algorithm, the computational complexity is effectively reduced, which can speed up the running time of the program. The extracted SURF features are as follows: Figure 3 shown.

[0072] Step S4: Construct a KD tree based on the SURF features of the video feature points, and perform two-stage feature matching based on the KD tree and RANSAC. Construct a KD tree based on the SURF features of the video feature points. In the process of searching for the nearest neighbor, a feature point to be matched is given, starting from the root node of the KD tree, and the search direction is determined by comparing the coordinate values ​​of the query point and the partition point. During the search process, the algorithm records the nearest neighbor point currently encountered and continuously updates the point. When the search reaches a leaf node, the algorithm backtracks the nodes on the path, checks whether there are closer neighbors, and calculates the Euclidean distance between the feature points. After using KNN for rough matching, RANSAC is used again for precise matching. A certain number of samples are randomly selected from the obtained matching data set as hypothetical in-place points and the parameters of the homography matrix are estimated. The model is used to test all other data points and update the matching data set and the optimal model, which can gradually eliminate the interference of out-of-place points to improve the accuracy of matching. Since the RANSAC method has a strong anti-interference ability to outliers and noise, it can maintain stable matching performance in complex environments.

[0073] Feature point matching is a problem of similarity retrieval between high-dimensional vectors through distance functions. In order to quickly and accurately find the neighbors of the query point, the present invention adopts a nearest neighbor search algorithm based on the KD tree, utilizes the clustered clustering form of the data set itself, designs an effective data index, and improves the search efficiency through the structural information contained in the data set itself, thereby effectively coping with the feasibility of large-scale video detection. The KD tree is a balanced binary tree structure used to organize points in multidimensional space. It divides the space into two parts by recursively selecting the median on each dimension as the dividing point, and constructs the left subtree and the right subtree respectively. This division method ensures that the height of the tree is low, which is conducive to fast search.

[0074] Step S5: Based on the result of the feature matching, the matching points are statistically calculated and the similarity index is output frame by frame. The video similarity comparison method based on matching point calculation and distance calculation is adopted. On the basis of video fingerprint extraction, the SURF features of the key frames of the standard video and the key frames of the comparison video are extracted. KNN is first used for rough matching, and RANSAC is used for precise matching. Then, the matching points are statistically calculated, and finally the similarity index is given frame by frame.

[0075] Specifically, the video similarity comparison scheme is obtained by calculating the ratio of matching points, the consistency of matching point directions, the confidence of matching pairs, etc. Among them, the ratio of matching points refers to the ratio of the matching points obtained after RANSAC precise matching to the total number of feature points in the standard video key frame, which can reflect the similarity between the key frame of the compared video and the key frame of the standard video in terms of feature points. The higher the ratio value, the more similar the two frames of video are in terms of feature points. The consistency of matching point directions refers to the consistency of directions between matching point pairs. For each matching point pair, the difference in its direction angle (or direction vector) is calculated, and the distribution of these differences is statistically analyzed. The consistency of matching point directions can reflect the similarity of the local texture or structure of the two frames of video. If the direction difference is small and the distribution is concentrated, it means that the two frames of video are relatively consistent in local texture or structure. The confidence of matching pairs refers to the probability or confidence level that each matching point pair is correctly matched. The confidence of matching pairs is estimated by using the ratio of inliers in the RANSAC algorithm, which can reflect the reliability and accuracy of the matching results. The higher the value, the more accurate the matching results. Finally, all indices calculated based on the number of matching points are weighted summed based on the Euclidean calculation to obtain the similarity index of the frame.

[0076] Step S6: According to the similarity index, the similarity between each frame of the video image is compared, and corresponding early warning plans are formulated according to different similarity levels. A video comparison and early warning model based on multiple measurement points is adopted. On the basis of video content comparison, the weight of key frames is combined to establish a comprehensive evaluation model. The model divides the similarity between video frames into four levels according to the degree of content comparison: dark red (highly different), red (different), yellow (suspicious) and green (same).

[0077] According to the brightness and contrast of the video frame, a custom two-level threshold is established, namely the maximum threshold and the minimum threshold. When the content difference between the two frames of video exceeds the set maximum threshold, it is judged as a dark red level, indicating that the two are significantly different in content. If the difference is between the minimum threshold and the maximum threshold, it is judged as a red level, indicating that the two are different in content to a certain extent. When the difference is close to but slightly lower than the minimum threshold, it is classified as a yellow level, indicating that there is similarity but still doubt. If the difference is lower than the set minimum threshold, it is judged as a green level, indicating that the two are exactly the same in content. There is a corresponding early warning response mechanism for different levels, that is, the judgment level and the time, location and other detailed information of the difference content are transmitted to the subsequent workflow. Specifically, when the video playback system detects that the video content is different from the original video, according to the degree of difference (such as dark red, red, yellow, green levels), a series of preset response measures or processes are automatically triggered to transmit information about the content difference to subsequent content security monitoring and other workflows, including the difference location, occurrence time, and difference type (color distortion, content loss, picture jitter, etc.), so that relevant personnel can quickly locate the problem, analyze the cause and take corresponding corrective measures.

[0078] The present invention aims to improve the accuracy and efficiency of content comparison, enhance the robustness of the algorithm, improve the real-time performance of the comparison, etc., and adapt to complex environments such as different lighting or viewing angles, to ensure that content and anomaly detection can be accurately and real-timely identified and compared under various conditions, reduce false alarm rates and missed alarm rates, and ensure that managers can respond and handle correctly and promptly. First, overcome the interference of complex factors such as changes in ambient light and resolution differences, use visual recognition technology to capture the playback screen of the city media large screen in real time, and accurately extract key features such as color, texture, and structure of the real-time screen. Then, by analyzing the extracted real-time screen features and constructing a "video fingerprint", the similarity is calculated with the features of the preset content. In terms of difference comparison and early warning, the present invention sets a reasonable similarity threshold. When the similarity between the real-time screen and the preset content is lower than the threshold, the system will automatically trigger the early warning mechanism. The early warning information may include similarity scores or levels, as well as screenshots, locations, and time of abnormal content.

[0079] The present invention is mainly used in the field of content management of urban media large screens, especially for those scenarios that require accurate content playback and real-time monitoring. The accuracy and compliance of the content played on urban media large screens are crucial. However, in actual applications, due to human operational errors, untimely content updates, equipment failures, illegal content intrusions, etc., the content played on the large screen may be deviated or abnormal. The present invention can timely detect and warn of these abnormal contents by comparing the differences between real-time images and preset content, thereby ensuring the accuracy and compliance of the content played on the large screen. For example, when public information such as government announcements is released, the accuracy of the information must be ensured, and when commercial or cultural activities such as brand promotion and art exhibitions are displayed, the integrity of the content played and the absence of infringing content must be ensured.

[0080] Embodiment 2

[0081] The present invention also provides a media screen content comparison system based on video features and similarity calculations. The media screen content comparison system based on video features and similarity calculations can be implemented by executing the process steps of the media screen content comparison method based on video features and similarity calculations, that is, those skilled in the art can understand the media screen content comparison method based on video features and similarity calculations as a preferred implementation of the media screen content comparison system based on video features and similarity calculations.

[0082] According to the present invention, a media screen content comparison based on video features and similarity calculation is provided, including:

[0083] Module M1: Acquire a video image and preprocess the video image; the preprocessing includes enhancing image contrast, highlighting image details and edges, and removing image noise on the basis of retaining image edges and details.

[0084] Module M2: extract key frames from the preprocessed video image to obtain a key frame sequence; the module M2 includes: module M2.1: obtain a key frame candidate sequence based on the color feature differences between the original video frames; module M2.2: obtain a key frame sequence by comparing the structural similarity between adjacent frames in the key frame candidate sequence; module M2.3: determine whether the number of current key frames meets the preset minimum number of key frames, if so, determine that the current key frames are valid; if not, directly extract a corresponding number of key frames from the original video in an equal duration manner.

[0085] Module M3: extracting scale-invariant SURF features corresponding to each frame from the key frame sequence;

[0086] Module M4: construct a KD tree based on the SURF features of the video feature points, and perform two-stage feature matching based on the KD tree and RANSAC; the module M4 includes: in the process of searching for the nearest neighbor, given a feature point to be matched, starting from the root node of the constructed KD tree, determining the search direction by comparing the coordinate values ​​of the query point and the partition point, during the search process, recording the nearest neighbor point currently encountered, and continuously updating the neighbor point, when the search reaches a leaf node, backtracking the nodes on the path, checking whether there are closer neighbors and calculating the Euclidean distance between the feature points; after using KNN for coarse matching, using RANSAC again for precise matching.

[0087] Module M5: Through the result of the feature matching, the matching points are statistically calculated and the similarity index is output frame by frame; the module M5 includes: calculating through the matching point related information, and performing weighted summation based on Euclidean calculation on all the calculated indicators, so as to obtain the similarity index of the current frame; the matching point related information includes the matching point ratio, the matching point direction consistency, and the matching pair confidence; the matching point ratio refers to the ratio of the matching points obtained after RANSAC precise matching to the total number of feature points in the standard video key frame, and the higher the ratio, the more similar the two frames of video are in terms of feature points; the matching point direction consistency refers to the degree of consistency of the direction between the matching point pairs, and the difference in the direction angle or direction vector of the matching point is calculated for each matching point pair. The smaller the difference, the higher the consistency of the local texture or structure of the two frames of video; the matching pair confidence refers to the probability or confidence level that each matching point pair is correctly matched, and the confidence of the matching pair is estimated using the inlier ratio in the RANSAC algorithm. The higher the confidence value, the more accurate the matching result.

[0088] Module M6: According to the similarity index, the similarity between each frame of the video image is compared, and corresponding early warning plans are formulated according to different similarity levels. The similarity levels include crimson level, red level, yellow level and green level; according to the brightness and contrast of the video frame, the maximum threshold and the minimum threshold are established. When the content difference between the two frames of video exceeds the set maximum threshold, it is judged as crimson level; if the difference is between the minimum threshold and the maximum threshold, it is judged as red level; when the difference is close to but slightly lower than the minimum threshold, it is judged as yellow level; if the difference is lower than the set minimum threshold, it is judged as green level.

[0089] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0090] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A media screen content comparison method based on video features and similarity calculation, characterized in that: include: Step S1: Acquire a video image and pre-process the video image; Step S2: extracting key frames from the preprocessed video image to obtain a key frame sequence; Step S3: extracting scale-invariant SURF features corresponding to each frame from the key frame sequence; Step S4: constructing a KD tree according to the SURF features of the video feature points, and performing two-stage feature matching based on the KD tree and RANSAC; Step S5: Based on the result of the feature matching, statistical indicators are calculated for the matching points and similarity indicators are output frame by frame; Step S6: According to the similarity index, the similarity between each frame of the video image is compared, and corresponding early warning plans are formulated according to different similarity levels.

2. The media screen content comparison method based on video features and similarity calculation according to claim 1 is characterized in that: The preprocessing includes enhancing image contrast, highlighting image details and edges, and removing image noise on the basis of retaining image edges and details.

3. The media screen content comparison method based on video features and similarity calculation according to claim 1 is characterized in that: The step S2 comprises: Step S2.1: obtaining a candidate key frame sequence according to the color feature differences between the original video frames; Step S2.2: obtaining a key frame sequence by comparing the structural similarity between adjacent frames in the key frame candidate sequence using a structural similarity method; Step S2.3: Determine whether the number of current key frames meets the preset minimum number of key frames. If so, determine that the current key frames are valid; if not, directly extract the corresponding number of key frames from the original video in an equal-length manner.

4. The media screen content comparison method based on video features and similarity calculation according to claim 1, characterized in that: The step S4 comprises: In the process of searching for the nearest neighbor, given a feature point to be matched, starting from the root node of the constructed KD tree, the search direction is determined by comparing the coordinate values ​​of the query point and the partition point. During the search process, the nearest neighbor point currently encountered is recorded and continuously updated. When the search reaches a leaf node, the nodes on the path are backtracked to check whether there are closer neighbors and calculate the Euclidean distance between the feature points. After using KNN for rough matching, RANSAC is used again for precise matching.

5. The media screen content comparison method based on video features and similarity calculation according to claim 1, characterized in that: The step S5 comprises: performing calculations based on the matching point related information, and performing weighted summation based on the Euclidean calculation on all the calculated indices, thereby obtaining a similarity index of the current frame; The matching point related information includes the matching point ratio, matching point direction consistency, and matching pair confidence; The matching point ratio refers to the ratio of the matching points obtained after RANSAC exact matching to the total number of feature points in the standard video key frame. The higher the ratio, the more similar the two frames of video are in terms of feature points. The matching point direction consistency refers to the degree of consistency of directions between matching point pairs. For each matching point pair, the difference in the direction angle or direction vector of the matching point is calculated. The smaller the difference, the higher the consistency of the local texture or structure of the two frames of video. The matching pair confidence refers to the probability or confidence level that each matching point pair is correctly matched. The confidence of the matching pair is estimated using the inlier ratio in the RANSAC algorithm. The higher the confidence value, the more accurate the matching result.

6. The media screen content comparison method based on video features and similarity calculation according to claim 1, characterized in that: The similarity levels include a crimson level, a red level, a yellow level, and a green level; According to the brightness and contrast of the video frames, the maximum threshold and the minimum threshold are established. When the content difference between the two frames of video exceeds the set maximum threshold, it is judged as a dark red level; if the difference is between the minimum threshold and the maximum threshold, it is judged as a red level; when the difference is close to but slightly lower than the minimum threshold, it is judged as a yellow level; if the difference is lower than the set minimum threshold, it is judged as a green level.

7. A media screen content comparison system based on video features and similarity calculation, characterized in that: include: Module M1: acquiring a video image and preprocessing the video image; Module M2: extract key frames from the preprocessed video image to obtain a key frame sequence; Module M3: extracting scale-invariant SURF features corresponding to each frame from the key frame sequence; Module M4: construct a KD tree according to the SURF features of the video feature points, and perform two-stage feature matching based on the KD tree and RANSAC; Module M5: Based on the result of the feature matching, the matching points are statistically calculated and the similarity index is output frame by frame; Module M6: According to the similarity index, the similarity between each frame of the video image is compared, and corresponding early warning plans are formulated according to different similarity levels.

8. The media screen content comparison system based on video features and similarity calculation according to claim 7, characterized in that: The preprocessing includes enhancing image contrast, highlighting image details and edges, and removing image noise on the basis of retaining image edges and details.

9. The media screen content comparison system based on video features and similarity calculation according to claim 7, characterized in that: The module M2 comprises: Module M2.1: Obtaining a candidate sequence of key frames based on the color feature differences between the original video frames; Module M2.2: obtaining a key frame sequence by comparing the structural similarity between adjacent frames in the key frame candidate sequence through a structural similarity system; Module M2.3: Determine whether the number of current key frames meets the preset minimum number of key frames. If so, determine that the current key frames are valid; if not, directly extract the corresponding number of key frames from the original video in an equal-length manner.

10. The media screen content comparison system based on video features and similarity calculation according to claim 7, characterized in that: The module M4 comprises: In the process of searching for the nearest neighbor, given a feature point to be matched, starting from the root node of the constructed KD tree, the search direction is determined by comparing the coordinate values ​​of the query point and the partition point. During the search process, the nearest neighbor point currently encountered is recorded and continuously updated. When the search reaches a leaf node, the nodes on the path are backtracked to check whether there are closer neighbors and calculate the Euclidean distance between the feature points. After using KNN for rough matching, RANSAC is used again for precise matching.

Images