A method, system, device and medium for video text sensitive information inspection based on a stay frame

Abstract

The present application relates to a kind of video text sensitive information inspection method, system, equipment and medium based on stay frame, wherein, method includes: from video data continuously extract frame sequence, and the detection and description of key point are carried out to each frame in frame sequence, to generate feature vector;The similarity of feature vector between adjacent frames is compared, and the matching feature point pair of adjacent frame is obtained, and based on the dynamic behavior between the matching feature point pair of adjacent frame is identified and extracted stay frame;Text recognition is carried out to stay frame using Paddle OCR deep learning model, and the text content embedded in video is obtained;Similarity calculation is carried out to text content to exclude duplicate text, and sensitive information matching and marking are carried out to non-duplicate text. Therefore, the present application solves the deficiency of traditional information security means in dealing with high-resolution camera equipment illegal shooting sensitive information leakage by deeply understanding video data and accurately extracting text information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video information recognition, and in particular to a video text sensitive information checking method, system, device and medium based on a stay frame. BACKGROUND

[0002] With the continuous progress of mobile communication technology, the camera function of devices such as smartphones has been significantly improved. High-resolution camera technology not only greatly enriches people's daily life, bringing convenience to socializing, entertainment and work, but also poses new challenges to the field of information security. Since these devices can easily capture high-quality images and videos, criminals may use this technical feature to illegally photograph paper sensitive documents, and then generate high-definition video files. Once these videos containing sensitive information are spread through the Internet and other dissemination channels, they will pose a serious threat to personal privacy, business secrets and even national security.

[0003] Therefore, the field of information security is facing a new problem of how to effectively detect and prevent the leakage of sensitive information. Traditional information security methods, such as encryption technology and access control, can protect the security of data to some extent, but it is difficult to fundamentally prevent sensitive information from being leaked through illegal photography. This is because these traditional methods mainly focus on the security of data storage and transmission, and are not effective in protecting the security of data when it is photographed into a video. Therefore, the field of information security urgently needs a new technical method that can effectively detect and prevent the leakage of sensitive information through illegal photography. SUMMARY

[0004] (I) Technical problem to be solved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present application provides a video text sensitive information checking method, system, device and medium based on a stay frame, which solves the technical problem that traditional information security methods cannot effectively prevent sensitive information from being leaked through illegal photography by high-resolution camera devices.

[0006] (II) Technical solution

[0007] In order to achieve the above-mentioned purpose, the main technical solution adopted by the present application includes:

[0008] In a first aspect, the present application provides a video text sensitive information checking method based on a stay frame, comprising:

[0009] extracting a frame sequence continuously from the obtained video data, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for representing the visual features of the frame;

[0010] The matching feature point pairs of adjacent frames are obtained by comparing the similarities of the feature vectors of the feature points between the adjacent frames, and the staying frame is identified and extracted based on the dynamic behaviors between the matching feature point pairs of the adjacent frames;

[0011] The pre-trained Paddle OCR deep learning model is used to perform text recognition on the staying frame to obtain the text content embedded in the video;

[0012] The similarity of the recognized text content is calculated to exclude duplicate texts, and the processed non-duplicate texts are matched with single sensitive words, sensitive word groups or sensitive text segments to detect and mark the existing sensitive information.

[0013] Optionally, a frame sequence is continuously extracted from the obtained video data, and key points are detected and described for each frame in the frame sequence to generate a feature vector for representing the visual features of the frame, including:

[0014] A frame sequence is continuously extracted from the obtained video data, and frames are selected from the video as processing objects at a set time interval to form the frame sequence;

[0015] For each frame image in the frame sequence, a Gaussian blur function is applied to the image to perform multi-scale spatial representation, forming a Gaussian pyramid;

[0016] The constructed Gaussian pyramid is subjected to a difference operation to generate a Gaussian difference pyramid;

[0017] Based on the Gaussian difference pyramid, local extreme points are searched in a three-dimensional scale space to preliminarily determine the key points;

[0018] The positions and scales of the preliminarily determined key points are optimized using Taylor expansion, and points with contrast that does not meet a preset contrast condition or points with edge response that does not meet an edge response condition are removed;

[0019] The gradient direction and amplitude of the neighborhood of the optimized key points are calculated, and one or more dominant directions are assigned to the key points according to the gradient direction and amplitude to achieve rotation invariance of the feature vector;

[0020] For the optimized key points, a fixed-size window is constructed around the key points to generate a feature vector of a specific dimension for representing the visual features of the frame.

[0021] Optionally, the matching feature point pairs of adjacent frames are obtained by comparing the similarities of the feature vectors of the feature points between the adjacent frames, including:

[0022] All feature points and corresponding feature vectors of a plurality of consecutive frames in the frame sequence are obtained;

[0023] Using Euclidean distance as the matching criterion, the Euclidean distance between any selected feature point in the current frame and all feature points in adjacent frames is calculated.

[0024] Sort all the calculated distances and select the feature point with the closest distance as the matching point for any selected feature point;

[0025] After multiple iterations of matching, until all feature points in the current video frame have found matching points or it is determined that there are no matching points, a series of matching point pairs between adjacent frames are obtained.

[0026] Optionally, identifying and extracting dwell frames based on dynamic behavior between matching feature point pairs of adjacent frames includes:

[0027] Initialize a new sequence of similar frames to analyze the number of matching points in consecutive adjacent frames;

[0028] W adjacent video frames k W k+1 The number of matching points is added to the new similar frame sequence as the starting element of the sequence;

[0029] Calculate the next adjacent video frame W k+2 The standard deviation of the number of matching points between a new sequence of similar frames and existing frames is E = D. ε (W0,W k+1 Simultaneously calculate the next adjacent video frame W. k+2 The difference D between the number of matching points of existing frames in the new similar frame sequence and the number of matching points of existing frames. a (W k+1 W k+2 And the average number of matching points D within the new similar frame sequence. a (W0,W k+1 The absolute value of the difference between ΔD and ΔD;

[0030] If ΔD≤N×E, where N is the preset magnification factor, then the next adjacent video frame W... k+2 Add it to a new sequence of similar frames;

[0031] If ΔD>N×E, then select the intermediate frame from the new similar frame sequence as the dwell frame;

[0032] After extracting the dwell frames, the similar frame sequence is reset, and a new sequence construction and dwell frame identification process begins from the next frame until dwell frames are identified and extracted from all adjacent frames.

[0033] All extracted pause frames are combined to form a pause frame set of the video.

[0034] Optionally, the pre-trained Paddle OCR deep learning model is used to perform text recognition on the stay frame to obtain the text content embedded in the video, including:

[0035] According to the text recognition requirements, a Paddle OCR deep learning model is selected and constructed;

[0036] An image dataset containing text samples of various fonts, sizes, directions and backgrounds is collected, and expert experience is introduced to label the positions of the text regions of the text samples and the corresponding text content;

[0037] The Paddle OCR deep learning model is trained based on the labeled image dataset;

[0038] The stay frame is input into the trained Paddle OCR model to locate and segment the text regions of the stay frame, and the located text regions are recognized to output the text content of each text region. The text content of each text region includes the video title, subtitles, bullet screen, watermark, advertising text and video text.

[0039] Optionally, the similarity calculation is performed on the recognized text content to exclude duplicate texts, including:

[0040] The recognized text content is subjected to word segmentation processing to cut continuous text into independent word units;

[0041] The TF-IDF algorithm is used to calculate the TF-IDF weight of each word in the current text content based on the frequency of the word in the current text and the scarcity of the word in the preset corpus, and the top N words with the highest TF-IDF weight are selected as key feature words;

[0042] The selected key feature words are subjected to hash processing to generate a fixed-length binary hash value for each key feature word;

[0043] According to the hash value and the TF-IDF weight of each key feature word, a corresponding weight vector is generated, wherein in the weight vector, the bit with a hash value of 1 is assigned a positive TF-IDF weight, and the bit with a hash value of 0 is assigned a negative TF-IDF weight;

[0044] The weight vectors of all key feature words are accumulated column by column to obtain an accumulated vector representing the overall features of the text;

[0045] According to the positive and negative values of each bit of the accumulated vector, a fixed-length Simhash value of the text is generated, wherein a positive number corresponds to 1 and a negative number corresponds to 0;

[0046] Calculate the Hamming distance between two text Simhash values, and determine whether the Hamming distance is less than a set threshold value;

[0047] When the Hamming distance between the Simhash values of two texts is less than or equal to the set threshold value, determine that the two texts are similar texts;

[0048] By means of a sliding window, traverse the text content of all the stay frames to obtain repeated texts similar to the identified text;

[0049] In the identified text content, eliminate the repeated texts similar to the identified text, and retain the non-repeated texts as the final result.

[0050] Optionally, the processed non-repeated texts are matched with single sensitive words, sensitive word groups, or sensitive text segments to detect and mark the existing sensitive information, including:

[0051] According to the collected historical sensitive data, generate a sensitive data set containing single sensitive words, sensitive word groups, and sensitive text segments by separation, disassembly, or analysis of variants;

[0052] Construct a basic regular expression of sensitive words, sensitive word groups, and sensitive text segments, and introduce meta characters representing any character and the number of character occurrences and character classes representing character replacement on the basic regular expression to construct a final regular expression that can accommodate certain spelling errors or variants;

[0053] Scan the processed non-repeated texts using the final regular expression to compare each word, word group, or text segment in the processed non-repeated texts with the entries in the sensitive database one by one;

[0054] In the matching process, once the text content is found to match any entry in the sensitive database, it is determined that sensitive information is detected;

[0055] Record all detected sensitive information and its location in the text, and for each detected sensitive information, mark it in the non-repeated text using at least one of highlighting, replacing with a placeholder, and adding a label.

[0056] In a second aspect, an embodiment of the present application provides a video text sensitive information checking system based on stay frames, comprising:

[0057] A feature vector generation module is configured to continuously extract a frame sequence from the obtained video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for representing the visual features of the frame;

[0058] The staying frame extraction module is configured to obtain a matching feature point pair of adjacent frames by comparing the similarity of feature vectors of feature points between the adjacent frames, and to identify and extract a staying frame based on the dynamic behavior between the matching feature point pair of adjacent frames.

[0059] The text recognition module is configured to recognize text in the staying frame by using a pre-trained Paddle OCR deep learning model, and to obtain text content embedded in the video.

[0060] The redundancy elimination and matching module is configured to eliminate duplicate text by calculating the similarity of the recognized text content, and to match a single sensitive word, a sensitive word group or a sensitive sentence segment in the processed non-duplicate text, so as to detect and mark the sensitive information.

[0061] In a third aspect, an embodiment of the present application provides a video text sensitive information checking device based on a staying frame, which comprises at least one database and a memory in communication connection with the at least one database, wherein the memory stores instructions executable by the at least one database, and the instructions are executed by the at least one database to enable the at least one database to execute the video text sensitive information checking method based on a staying frame as described above.

[0062] In a fourth aspect, an embodiment of the present application provides a computer readable medium having computer executable instructions stored thereon, and the executable instructions are executed by a processor to implement the video text sensitive information checking method based on a staying frame as described above.

[0063] (III) Advantages

[0064] The present application has the following advantages:

[0065] Firstly, the present application can continuously extract a frame sequence from the obtained video data, and detect and describe key points, which means that even in the case of poor video shooting conditions or blurred text information, it helps to more accurately capture the dynamic changes in the video and accurately generate feature vectors representing the visual features of the frames. This step provides a solid data foundation for subsequent sensitive information detection.

[0066] Secondly, the staying frame is intelligently identified and extracted by comparing the similarity of the feature vectors of adjacent frames. This innovative point effectively utilizes the characteristic that the lens stays for a long time in video shooting. From this dynamic behavior-based staying frame identification method, the behavior pattern of the video shooter can be deeply understood and considered, so that the present application performs well in processing complex and variable video data.

[0067] Furthermore, using the pre-trained Paddle OCR deep learning model to perform text recognition on the stay frame can quickly and accurately obtain the text content in the video, and the method can cope with text information of various fonts, sizes and colors, which is faster and more accurate than the traditional method.

[0068] Finally, by calculating the similarity, the repeated text is effectively eliminated, the interference of redundant information is reduced, the detection of sensitive information is more accurate and efficient. Meanwhile, the non-repeated text processed is matched with sensitive words, word groups or sentence segments in detail, so that the potential sensitive information can be found and marked in time, and strong technical support is provided for information security.

[0069] Therefore, the video text sensitive information checking method based on the stay frame can solve the deficiency of the traditional information security means in dealing with the illegal shooting of sensitive information leakage by high-resolution camera equipment, greatly improve the efficiency and accuracy of video sensitive information detection, and provide strong technical support for the information security field. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 The flowchart of the method provided by the embodiment of the present application is shown in the figure;

[0071] Figure 2 The specific flowchart of step S1 of the method provided by the embodiment of the present application is shown in the figure;

[0072] Figure 3 The first part of the specific flowchart of step S2 of the method provided by the embodiment of the present application is shown in the figure;

[0073] Figure 4 The adjacent video frame matching point diagram of the method provided by the embodiment of the present application is shown in the figure;

[0074] Figure 5 The second part of the specific flowchart of step S2 of the method provided by the embodiment of the present application is shown in the figure;

[0075] Figure 6 The video frame matching point quantity diagram of the method provided by the embodiment of the present application is shown in the figure;

[0076] Figure 7 The stay frame identification diagram of the method provided by the embodiment of the present application is shown in the figure;

[0077] Figure 8 The specific flowchart of step S3 of the method provided by the embodiment of the present application is shown in the figure;

[0078] Figure 9 The first part of the specific flowchart of step S4 of the method provided by the embodiment of the present application is shown in the figure

[0079] Figure 10 The simhash construction process of the method provided by the embodiment of the application;

[0080] Figure 11 The second part of the specific process of step S4 of the method provided by the embodiment of the application;

[0081] Figure 12 The overall process of the method provided by the embodiment of the application. DETAILED DESCRIPTION

[0082] In order to better explain the application, so as to be understood, the application is described in detail by specific embodiments in combination with the accompanying drawings.

[0083] As Figure 1 shown, the video text sensitive information checking method based on a stay frame proposed by the embodiment of the application comprises: continuously extracting a frame sequence from acquired video data, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for representing visual features of the frame; obtaining matching feature point pairs of adjacent frames by comparing the similarity of the feature vectors of the feature points between the adjacent frames, and identifying and extracting a stay frame based on the dynamic behavior between the matching feature point pairs of the adjacent frames; performing text recognition on the stay frame by using a pre-trained Paddle OCR deep learning model to obtain text content embedded in the video; performing similarity calculation on the recognized text content to exclude repeated texts, and performing single sensitive word, sensitive word group or sensitive sentence matching on the processed non-repeated texts to detect and mark the existing sensitive information.

[0084] Firstly, the application can continuously extract a frame sequence from acquired video data, and detect and describe key points, which means that even in the case of poor video shooting conditions or relatively blurred text information, it helps to more accurately capture the dynamic changes in the video and accurately generate feature vectors representing visual features of the frame. This step provides a solid data foundation for subsequent sensitive information detection.

[0085] Secondly, by comparing the similarity of the feature vectors of adjacent frames, the stay frame is intelligently identified and extracted, which effectively utilizes the characteristic that the lens stays for a long time in video shooting. From this stay frame identification method based on dynamic behavior, the behavior pattern of the video shooter can be deeply understood and considered, so that the application performs well when processing complex and variable video data.

[0086] Furthermore, using the pre-trained Paddle OCR deep learning model to perform text recognition on the stay frames can quickly and accurately obtain the text content in the video, and the method can cope with text information of various fonts, sizes and colors, which is faster and more accurate than the traditional method.

[0087] Finally, by calculating the similarity, the repeated text is effectively removed, the interference of redundant information is reduced, the detection of sensitive information is more accurate and efficient. Meanwhile, the non-repeated text after processing is matched with sensitive words, word groups or sentences in detail, so that the potential sensitive information can be found and marked in time, and strong technical support is provided for information security.

[0088] Therefore, the video text sensitive information checking method based on stay frames solves the deficiency of traditional information security means in dealing with high-resolution camera equipment illegal shooting of sensitive information leakage by deeply understanding video data and accurately extracting text information, greatly improves the efficiency and accuracy of video sensitive information detection, and provides strong technical support for the field of information security.

[0089] In order to better understand the above technical solutions, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer, more thorough understanding of the present application and to convey the scope of the present application to those skilled in the art.

[0090] Specifically, the embodiment of the present application provides a video text sensitive information checking method based on stay frames, comprising:

[0091] S1, a frame sequence is continuously extracted from the obtained video data, and key point detection and description are performed on each frame in the frame sequence to generate a feature vector for representing the visual features of the frame.

[0092] Further, as shown in Figure 2 , step S1 comprises:

[0093] S11, a frame sequence is continuously extracted from the obtained video data, and frames are selected as processing objects from the video according to a set time interval to form the frame sequence.

[0094] S12, for each frame image in the frame sequence, a multi-scale spatial representation of the image is formed by applying a Gaussian blur function to the image to form a Gaussian pyramid.

[0095] S13, a Gaussian difference pyramid is generated by performing a difference operation on the constructed Gaussian pyramid.

[0096] S14, based on the Gaussian difference pyramid, the key points are preliminarily determined by searching local extreme points in a three-dimensional scale space.

[0097] S15, the position and scale of the preliminarily determined key points are optimized by using Taylor expansion, and the points with contrast not meeting a preset contrast condition or the points with edge response not meeting an edge response condition are removed.

[0098] S16, the gradient direction and amplitude of the neighborhood of the optimized key points are calculated, and one or more dominant directions are assigned to the key points according to the gradient direction and amplitude, so as to realize the rotation invariance of the feature vector.

[0099] S17, for the optimized key points, a fixed-size window is constructed with the key points as the center, so as to generate a feature vector of a specific dimension for representing the visual features of the frame.

[0100] In a specific embodiment, the SIFT feature extraction algorithm processing flow of the video frame is as follows: first, frame sequences F1, F2,..., Fn are extracted from the video at time intervals. n as input. Then, the key points are detected by constructing the Gaussian pyramid and the Gaussian difference pyramid. The Gaussian blur function is The blurred image L(x, y, σ) = G(x, y, σ) * I(x, y), and the difference is D(x, y, σ) = L(x, y, kσ) - L(x, y, σ). The key points are preliminarily determined by searching local extreme points in a 3D space. Then, the key point position is optimized by using Taylor expansion, and the points with low contrast or strong edge response are removed. Subsequently, the gradient direction and amplitude of the neighborhood of the key points are calculated The direction information is assigned to the key points to realize the rotation invariance. Finally, a 16*16 window is constructed to generate a 128-dimensional feature vector as a feature descriptor.

[0101] S2, the matching feature point pairs of the adjacent frames are obtained by comparing the similarity of the feature vectors of the feature points between the adjacent frames, and the stay frame is recognized and extracted based on the dynamic behavior between the matching feature point pairs of the adjacent frames.

[0102] Further, with reference to Figure 3 , the matching feature point pairs of the adjacent frames obtained by comparing the similarity of the feature vectors of the feature points between the adjacent frames in step S2 include:

[0103] S21, all feature points and corresponding feature vectors of a plurality of continuous frames in the frame sequence are obtained.

[0104] S22, the Euclidean distance is used as a matching criterion, and the Euclidean distance between any selected feature point in the current frame and all feature points in the adjacent frame is calculated.

[0105] S23, sort all the calculated distances, and select the nearest one as the matching point of any selected feature point.

[0106] S24, after multiple iterations of matching, until all feature points in the current video frame are found matching points or determine no matching points, get a series of matching point pairs between adjacent frames.

[0107] When the feature vector of the video frame is generated, in order to find the matching point of a certain feature point in the adjacent frame, the feature point matching algorithm is needed. In this embodiment, the Euclidean distance is used for feature point matching, and the specific steps are as follows:

[0108] First, get a plurality of continuous frames in the video frame sequence. For each frame, use a feature extraction algorithm (such as SIFT, SURF, etc.) to extract all the feature points and generate the corresponding feature vectors. These feature vectors are the basis for subsequent feature point matching.

[0109] After obtaining all the feature points and corresponding feature vectors, the matching of the feature points is started. Here, the Euclidean distance is used as the matching criterion. Specifically, select any feature point in the current frame, and then calculate the Euclidean distance between the feature point and all feature points in the adjacent frame. The formula for calculating the Euclidean distance is: Where p and q are two feature vectors to be compared, and n is the dimension of the feature vector.

[0110] Next, after calculating the Euclidean distance between the current feature point and all feature points in the adjacent frame, a distance list is obtained. Then, sort this distance list and select the nearest feature point as the matching point of the current selected feature point. The reason for this is that in the feature space, the two feature points with the closest distance are more likely to correspond to different perspectives or states of the same object or scene, so their matching degree is higher.

[0111] In order to find the matching points of all feature points in the current video frame, new feature points are selected for calculation and matching until all feature points in the current video frame are found matching points or determined no matching points. Finally, referring to Figure 4 , a series of matching point pairs between adjacent frames are obtained, which can be used for subsequent video analysis, target tracking and other tasks.

[0112] Furthermore, as shown in Figure 5 , the dynamic behavior recognition and extraction of the matching feature point pairs based on adjacent frames in step S2 include:

[0113] S25, initialize a new similar frame sequence for analyzing the number of matching points of consecutive adjacent frames.

[0114] S26, combine two adjacent video frames W k W k+1 The number of matching points is added to the new similar frame sequence as the starting element of the sequence.

[0115] S27. Calculate the next adjacent video frame W. k+2 The standard deviation of the number of matching points between a new sequence of similar frames and existing frames is E = D. ε (W0,W k+1 Simultaneously calculate the next adjacent video frame W. k+2 The difference D between the number of matching points of existing frames in the new similar frame sequence and the number of matching points of existing frames. a (W k+1 W k+2 And the average number of matching points D within the new similar frame sequence. a (W0,W k+1 The absolute value of the difference between ΔD and ΔD.

[0116] S28. If ΔD≤N×E, where N is the preset magnification factor, then the next adjacent video frame W... k+2 Add it to the new similar frame sequence; if ΔD>N×E, then select the middle frame from the new similar frame sequence as the dwell frame.

[0117] S29. After extracting the dwell frames, reset the similar frame sequence and start a new sequence construction and dwell frame recognition process from the next frame until dwell frames are identified and extracted from all adjacent frames; collect all extracted dwell frames to form a dwell frame set of the video.

[0118] In one specific embodiment, in order for the video reader to see the text information in the video clearly, the camera needs to stay on a page of text for a relatively long time. This time variation will be reflected in the number of adjacent frames. That is, the number of frame matching points when shooting a page of text is relatively stable, while when the video viewer flips through the paper text, it will cause the number of frame matching points to fluctuate greatly. Figure 6 This is an example graph showing the number of matching points per video frame in a video file. According to the graph, there are some relatively stable matching points among the video frames. These matching points contain a large amount of repetitive information, and these highly repetitive texts can be removed to identify stationary frames. To identify stationary frames, the following formula needs to be constructed first:

[0119] (1) The difference in the number of matching points D a , indicating the number of matching points W k With W m difference.

[0120] D a (W kW m ) = W m -W k ;

[0121] (2) The average number of matching points within similar frame sequences, Dv, represents the number of the first matching point in the sequence, W. k The last one is W m The average number of matching points.

[0122]

[0123] (3) Standard deviation of the number of matching points within similar frame sequences D ε , indicating that the number of the first matching points in the sequence is W k The last one is W m The standard deviation of the number of matching points.

[0124]

[0125] Next, as Figure 7 As shown, the specific steps for dwell frame recognition are as follows: First, create a new similar frame sequence QF. m (m≥1); secondly, the number of matching points W between two adjacent frames. k W k+1 Add to similar frame sequence QF m Next, calculate W for the next frame. k+2 Similar frame sequences QF m Standard deviation E = D ε (W0,W k+1 The average number of matching points D within the new similar frame sequence. a (W0,W k+1 The absolute value of the difference between ) is ΔD = |D a (W k+1 W k+2 )-D v (W0, W) k+1 )|.

[0126] Therefore, if ΔD≤N×E, where N is the preset magnification factor, then W k+2 Add it to the sequence of similar frames and return to this step: "Calculate the next frame W". k+2 Similar frame sequences QF m Standard deviation E = D ε (W0,W k+1 The average number of matching points D within the new similar frame sequence. a (W0,W k+1 The absolute value of the difference between ) is ΔD = |D a (W k+1 W k+2)-D v (W0, W k+1 ) if AD < N x E, the middle frame in the similar frame sequence QF m is taken as the stay frame, and m+1 is returned to the starting step.

[0127] S3, a pre-trained Paddle OCR deep learning model is used to perform text recognition on the stay frame to obtain the text content embedded in the video.

[0128] Further, as shown in Figure 8 , step S3 includes:

[0129] S31, according to the text recognition requirement, a Paddle OCR deep learning model is selected and constructed.

[0130] It should be understood that text recognition is a process of recognizing and extracting text contained in a target area. There are currently a large number of deep learning models that can accurately recognize English text. However, since Chinese contains more words and characters than English and the composition of Chinese characters is more complex, these models have lower accuracy when recognizing Chinese data. The Paddle OCR deep learning model is a text recognition model based on deep learning developed by Baidu. Compared with other deep learning models, it supports more Chinese recognition scenarios and has higher recognition accuracy in Chinese scenarios.

[0131] S32, an image dataset containing text samples of various fonts, sizes, directions, and backgrounds is collected, and expert experience is introduced to label the positions of the text regions of the text samples and the corresponding text content.

[0132] S33, the Paddle OCR deep learning model is trained based on the labeled image dataset.

[0133] S34, the stay frame is input into the trained Paddle OCR model to locate and segment the text regions of the stay frame, and the located text regions are recognized to output the text content of each text region. The text content of each text region includes video title, subtitle, bullet screen, watermark, advertising text, and video text.

[0134] S4, similarity calculation is performed on the recognized text content to exclude duplicate text, and single sensitive word, sensitive word group, or sensitive sentence matching is performed on the processed non-duplicate text to detect and mark the existing sensitive information.

[0135] In the dwell frame extraction process, although the dwell frame extraction algorithm can remove redundant video frames, there are still repeated video frames remaining. Therefore, in the text post-processing stage, these repeated texts must be further excluded. In order to identify the repeated texts, text similarity calculation must be performed.

[0136] Further, as shown in Figure 9 , the step S4 of excluding repeated texts by performing similarity calculation on the identified text content includes:

[0137] S41, performing word segmentation processing on the identified text content to divide the continuous text into independent word units.

[0138] S42, using the TF-IDF algorithm to calculate the TF-IDF weight of each word in the current text content based on the frequency of the word in the current text and the scarcity of the word in the preset corpus, and selecting the top N words with the highest TF-IDF weight as key feature words.

[0139] S43, performing hash processing on the selected key feature words to generate a fixed-length binary hash value for each key feature word.

[0140] S44, generating a corresponding weight vector according to the hash value and TF-IDF weight of each key feature word, wherein in the weight vector, the bit with hash value 1 is assigned a positive TF-IDF weight, and the bit with hash value 0 is assigned a negative TF-IDF weight.

[0141] S45, accumulating the weight vectors of all key feature words column by column to obtain an accumulated vector representing the overall features of the text.

[0142] S46, generating a fixed-length Simhash value of the text according to the positive and negative values of each bit of the accumulated vector, wherein positive numbers correspond to 1 and negative numbers correspond to 0.

[0143] S47, calculating the Hamming distance between the Simhash values of two texts and determining whether the Hamming distance is less than a set threshold.

[0144] S48, when the Hamming distance between the Simhash values of two texts is less than or equal to the set threshold, determining that the two texts are similar texts.

[0145] S49, traversing all the text content of the dwell frames in a sliding window manner to obtain repeated texts similar to the identified text; in the identified text content, the repeated texts similar to the identified text are excluded, and the non-repeated texts are retained as the final result.

[0146] In yet another specific embodiment, referring to Figure 10 , the steps of identifying and excluding repeated text content are as follows:

[0147] (1) Tokenization and TF-IDF Calculation: To convert the text content in video frames into numerical features, the first step is to perform tokenization on the text of the video frames. The tokenized text will be used for subsequent calculations. Specifically, the TF-IDF (Term Frequency-Inverse Document Frequency) method is used to weight each word, calculating its importance. The TF-IDF algorithm measures the importance of a word in a document, combining the frequency of the word in the current text (TF) and the scarcity of the word in the entire corpus (IDF). The calculation method is:

[0148]

[0149] where TF(t, d) represents the number of times word t appears in document d, DF(t) is the number of documents containing word t, and N is the total number of documents.

[0150] For each video frame, the top 20 words with the highest TF-IDF weights are selected, which are considered as the key features of the frame text. In this way, each video frame text can be converted into a set of 20 words, and the higher the weight of a word, the more important it is in the text.

[0151] (2) Hash Conversion: The 20 important words selected in step 1 are subjected to hash processing. Each word is converted into a 64-bit binary value through a regular hash function, which represents the "fingerprint" of the word. This hash value is calculated based on the text information of the word, so different text content will generate different hash values. Through this process, the text of each video frame is converted into a set of length 20, where each element is a 64-bit binary number representing the hash value of the word in the text.

[0152] (3) Calculate Weight Vector: Next, a weight vector is generated based on the hash value of each word. For each bit in the hash value of a word, if the bit is 1, assign the TF-IDF weight of the word to a positive value, and if the bit is 0, assign a negative value. For example, assuming a word's hash value is 010111 and its TF-IDF weight is 5, the weight vector generated according to the hash value is:

[0153] [-5,5,-5,5,5,5];

[0154] Here, the 1 bits in the hash value correspond to positive weights, and the 0 bits correspond to negative weights. This process converts the 20 words of each video frame into a set of weight vectors with a length of 64.

[0155] (4) Column-wise vector accumulation: For each of the 20 hash words in a video frame, accumulate its corresponding weight vector column-wise. Suppose there are three weight vectors:

[0156] [-5, 5, -5, 5, 5, 5], [-3, -3, -3, 3, -3, 3], [1, -1, -1, 1, 1, 1];

[0157] Add these weight vectors column-wise to get a new accumulated vector:

[0158] [-7, 1, -9, 9, 3, 9];

[0159] This accumulated vector is the "feature vector" of the current video frame text, which summarizes the overall features of the video frame text.

[0160] (5) Generate Simhash value: After obtaining the accumulated vector, it needs to be converted into a 64-bit binary Simhash value. For each position value, if it is negative, the bit is 0; if it is positive, the bit is 1.

[0161] For example, the conversion result of the accumulated vector [-7, 1, -9, 9, 3, 9] is:

[0162] Simhash = 010111;

[0163] This 64-bit binary value is the Simhash value of the video frame, which can represent the features of the frame text.

[0164] (6) Calculate similarity To judge the similarity of two video frame texts, we need to calculate the Hamming distance between their Simhash values. Hamming distance is the number of different bits between two binary strings of the same length.

[0165]

[0166] where, is an indicator function, which is 1 when the two corresponding bits are different, and 0 otherwise. If the Hamming distance between two Simhash values is less than or equal to 3, it is considered that the text content of the two video frames is similar. If the Hamming distance is greater than 3, it is considered that they are not similar.

[0167] (7) Similarity processing. When the text content of two video frames is similar, the identified text content needs to be removed from the similar frame sequence. This can avoid repeated calculation and improve processing efficiency. After removal, return to step 1 to continue processing. If the text of the two video frames is not similar, the sliding window is moved one position, and the text content of the next frame is continuously checked. When the sliding window traverses all the text content of the stop frame, the process ends.

[0168] and, as shown in FIG. 4, the step S4 of matching the processed non-redundant text with single sensitive words, sensitive word groups or sensitive text segments to detect and mark the existing sensitive information includes: Figure 11

[0169] A41, according to the collected historical sensitive data, a sensitive data set containing single sensitive words, sensitive word groups and sensitive text segments is generated by separating, disassembling or analyzing variants.

[0170] A42, a basic regular expression of sensitive words, sensitive word groups and sensitive text segments is constructed, and meta characters representing any character and the number of character occurrences and character classes representing character replacement are introduced on the basic regular expression to construct a final regular expression that can accommodate spelling errors or variants.

[0171] In a specific embodiment, according to the collected historical sensitive data, a sensitive data set containing single sensitive words, sensitive word groups and sensitive text segments is generated by separating, disassembling or analyzing variants, wherein for each sensitive word or word group, its possible variant forms are analyzed. Variants can include but are not limited to spelling errors, synonym replacement, word order transposition, addition or deletion of non-key characters, etc. Then, according to the accurate form of the sensitive word or word group, a basic regular expression is constructed. This expression can accurately match the target text. On the basis of the basic regular expression, by adding appropriate meta characters (such as. representing any character, * representing that the previous character can appear zero or multiple times,? representing that the previous character can appear zero or one time, etc.), a regular expression that can accommodate certain spelling errors or variants is constructed. Character classes (such as [a-z] representing any lowercase letter) are used to match possible character replacements. In addition, fuzzy matching techniques such as approximate matching functions in regular expressions can also be considered to capture more extensive variants.

[0172] A43, the processed non-redundant text is scanned using the final regular expression to compare each word, word group or text segment in the processed non-redundant text with the entries in the sensitive database one by one.

[0173] It should be emphasized that string matching algorithms (such as KMP algorithm, BM algorithm, etc.) can also be used to scan the processed non-redundant text in the embodiments of the present application.

[0174] ​A44、In the matching process, once the text content is found to match any entry in the sensitive database, it is determined that sensitive information is detected.

[0175] A45、All detected sensitive information and its location in the text are recorded, and for each detected sensitive information, at least one of highlighting, replacing with a placeholder, and adding a label is used to mark in the non-repeated text. This needs to ensure that the marking method can clearly indicate the existence of sensitive information, and will not damage the overall structure and meaning of the original text.

[0176] In another aspect, the embodiments of the present application provide a video text sensitive information checking system based on stay frames, comprising:

[0177] A feature vector generation module is configured to continuously extract a frame sequence from the acquired video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for representing visual features of the frame.

[0178] A stay frame extraction module is configured to obtain matching feature point pairs of adjacent frames by comparing the similarity of feature vectors of feature points between adjacent frames, and identify and extract stay frames based on dynamic behaviors between the matching feature point pairs of adjacent frames.

[0179] A text recognition module is configured to perform text recognition on the stay frames using a pre-trained Paddle OCR deep learning model to obtain text content embedded in the video.

[0180] A duplicate elimination and matching module is configured to eliminate duplicate text by performing similarity calculation on the recognized text content, and match single sensitive words, sensitive word groups, or sensitive text segments in the processed non-duplicate text to detect and mark the existing sensitive information.

[0181] Furthermore, the embodiments of the present application provide a video text sensitive information checking device based on stay frames, comprising: at least one database; and a memory in communication connection with the at least one database; wherein the memory stores instructions executable by the at least one database, and the instructions are executed by the at least one database to enable the at least one database to perform the video text sensitive information checking method based on stay frames as described above.

[0182] The embodiments of the present application introduce an innovative video text sensitive information checking device, which operates based on the technology of stay frames. The core components of the device include at least one database and a memory in communication connection with the databases.

[0183] The memory here not only serves for data storage, but more importantly, it stores instructions that can be executed by the database. When these instructions are executed by the database, the method of checking sensitive information in video text based on stay frames can be implemented. The implementation of this method can effectively identify sensitive text information in the video, further improving the efficiency of information security and auditing.

[0184] In short, through the close cooperation of the database and the memory, this device uses a special instruction set to achieve the function of efficiently and accurately checking sensitive information in the text content of the stay frames in the video.

[0185] Meanwhile, the embodiment of the present application provides a computer readable medium having computer executable instructions stored thereon, and the executable instructions are executed by the processor to implement the method of checking sensitive information in video text based on stay frames as described above.

[0186] The embodiment of the present application also provides a computer readable medium having specific computer executable instructions stored thereon. When the instructions are executed by the processor, the method of checking sensitive information in video text based on stay frames as described above can be implemented. Through this medium, users can conveniently deploy the method to various computing devices, thereby realizing fast and accurate checking of sensitive information in video text and improving the efficiency and reliability of information security protection. This computer readable medium provides a convenient and efficient solution for checking sensitive information in video text.

[0187] In summary, the present application provides a method, system, device and medium for checking sensitive information in video text based on stay frames, which are described in detail with reference to Figure 12 Firstly, the extraction of video frames is needed. This process involves extracting image data frame by frame from the original video file, laying the foundation for subsequent processing steps. After the video frames are extracted, the next step is to process these frame images using an algorithm called Scale-Invariant Feature Transform (SIFT). The SIFT algorithm can extract feature points with scale invariance from video images. These feature points have strong robustness and can remain stable under different scales and rotations. Through data analysis of these feature points, stay frames in the video can be effectively identified, thereby reducing the impact of redundant video frames on subsequent text information extraction.

[0188] After the identification of the dwell frames is completed, the next step is to use the Paddle OCR deep learning model to recognize the text in the video frames. Paddle OCR is a deep learning-based optical character recognition model that can efficiently extract text information from images. By inputting the video frames into the Paddle OCR model, accurate recognition of the text information contained in the video can be achieved.

[0189] Finally, further post-processing steps are performed on the recognized text. This process includes removing redundant text information to ensure that the final output text content is concise and targeted. In addition, sensitive word matching is also needed to identify possible sensitive information. After the matching is completed, the output text content is highlighted for sensitive words to facilitate users to quickly identify and handle these sensitive information. Through this series of steps, the video file sensitive information detection process can effectively identify and handle the sensitive information in the video, ensuring the safety and compliance of the content.

[0190] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0191] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowcharts and / or block diagrams.

[0192] In addition, it should be noted that in the description of the present application, the description of the terms "one embodiment", "some embodiments", "embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0193] While the preferred embodiments of the application have been described, those skilled in the art will recognize that many modifications and variations of this preferred embodiment are possible without departing from the spirit or scope of the present application. Therefore, it is intended that the scope of the application be defined by the following claims and their equivalents.

[0194] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for video text sensitive information inspection based on stay frames, characterized in that, The method comprises the following steps: extracting a frame sequence from the obtained video data in succession, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for representing visual features of the frame; obtaining matching feature point pairs between adjacent frames by comparing the similarity of the feature vectors of the feature points between the adjacent frames, and identifying and extracting a stay frame based on the dynamic behavior between the matching feature point pairs of the adjacent frames; performing text recognition on the stay frame by using a pre-trained Paddle OCR deep learning model to obtain text content embedded in the video; performing similarity calculation on the recognized text content to exclude duplicate texts, and performing single sensitive word, sensitive word group or sensitive sentence matching on the processed non-duplicate texts to detect and mark the existing sensitive information; The method comprises the following steps: initializing a new similar frame sequence for analyzing the number of matching points of continuous adjacent frames; adding the number of matching points of two adjacent video frames W k ,W k+1 into the new similar frame sequence as the starting element of the sequence; calculating the standard deviation E=D k+2 (W0,W ε ) of the number of matching points of the next adjacent video frame W k+1 and the existing frames in the new similar frame sequence, and calculating the absolute value ΔD of the difference between the difference D k+2 (W a ,W k+1 ) of the number of matching points of the next adjacent video frame W k+2 and the existing frames in the new similar frame sequence and the average value D a (W0,W k+1 ) of the number of matching points in the new similar frame sequence; if ΔD≤N×E, wherein N is a preset magnification, then the next adjacent video frame W k+2 is added to the new similar frame sequence; if ΔD>N×E, then the middle frame is selected from the new similar frame sequence as a stay frame; after the stay frame is extracted, the similar frame sequence is reset, and the new sequence construction and stay frame identification process is started from the next frame, until the stay frame is extracted from all adjacent frames; and all extracted stay frames are collected to form a stay frame set of the video.

2. The stay frame based video text sensitive information checking method of claim 1, wherein, extracting a frame sequence from the obtained video data in succession, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for representing visual features of the frame comprises: extracting a frame sequence from the obtained video data, and selecting frames from the video as processing objects at a set time interval to form the frame sequence; performing multi-scale spatial representation on the image by applying a Gaussian blur function to the image to form a Gaussian pyramid for each frame image in the frame sequence; performing difference operation on the constructed Gaussian pyramid to generate a Gaussian difference pyramid; preliminarily determining key points by searching for local extreme points in a three-dimensional scale space based on the Gaussian difference pyramid; optimizing the positions and scales of the preliminarily determined key points by using Taylor expansion, and eliminating points with contrast that does not meet a preset contrast condition or points with edge response that does not meet an edge response condition; calculating the gradient direction and amplitude of the neighborhood of the optimized key points, and assigning one or more dominant directions to the key points according to the gradient direction and amplitude to realize rotation invariance of the feature vector; for the optimized key points, constructing a fixed-size window centered on the key points to generate a feature vector of a specific dimension for representing visual features of the frame.

3. The stay frame based video text sensitive information checking method of claim 1, wherein, obtaining matching feature point pairs between adjacent frames by comparing the similarity of the feature vectors of the feature points between the adjacent frames comprises: obtaining all feature points and corresponding feature vectors of a plurality of continuous frames in the frame sequence; calculating the Euclidean distance between any selected feature point in the current frame and all feature points in the adjacent frame by taking the Euclidean distance as a matching criterion; sorting all the calculated distances, and selecting a feature point with the closest distance as the matching point of the selected feature point; after multiple iterations of matching, until all feature points in the current video frame find matching points or it is determined that there is no matching point, a series of matching point pairs between adjacent frames are obtained.

4. The stay frame-based video text sensitive information checking method of claim 1, wherein, performing text recognition on the stay frame by using a pre-trained Paddle OCR deep learning model to obtain text content embedded in the video comprises: selecting and constructing a Paddle OCR deep learning model according to text recognition requirements; collecting an image dataset containing text samples of various fonts, sizes, directions and backgrounds, and introducing expert experience to label the positions of the text regions of the text samples and the corresponding text content; Train the Paddle OCR deep learning model based on the annotated image dataset; Input the stay frame into the trained Paddle OCR model to locate and segment the text region of the stay frame, and recognize the located text region to output the text content of each text region.

5. The stay frame based video text sensitive information checking method of claim 1, wherein, Exclude duplicate texts by calculating the similarity of the recognized text content, including: Carry out word segmentation processing on the recognized text content to cut continuous text into independent word units; Calculate the TF-IDF weight of each word in the current text content based on the frequency of the word in the current text and the scarcity of the word in the preset corpus using the TF-IDF algorithm, and select the top N words with the highest TF-IDF weight as key feature words; Hash the selected key feature words to generate a fixed-length binary hash value for each key feature word; Generate a corresponding weight vector based on the hash value and TF-IDF weight of each key feature word, wherein the bit with a hash value of 1 is assigned a positive TF-IDF weight, and the bit with a hash value of 0 is assigned a negative TF-IDF weight; Accumulate the weight vectors of all key feature words column by column to obtain an accumulated vector representing the overall features of the text; Generate a fixed-length Simhash value for the text based on the positive and negative values of each bit in the accumulated vector, wherein positive numbers correspond to 1 and negative numbers correspond to 0; Calculate the Hamming distance between the Simhash values of two texts and determine whether the Hamming distance is less than a set threshold; When the Hamming distance between the Simhash values of two texts is less than or equal to the set threshold, the two texts are determined to be similar texts; Iterate through the text content of all stay frames in a sliding window manner to obtain duplicate texts similar to the recognized texts; Remove the duplicate texts similar to the recognized texts from the recognized text content and retain the non-duplicate texts as the final result.

6. The stay frame based video text sensitive information inspection method of claim 1, wherein, Match the processed non-duplicate texts with single sensitive words, sensitive word groups, or sensitive text segments to detect and mark the presence of sensitive information, including: Based on the collected historical sensitive data, generate a sensitive dataset containing single sensitive words, sensitive word groups, and sensitive text segments by separating, disassembling, or analyzing variants; Construct a basic regular expression for sensitive words, sensitive word groups, and sensitive text segments, and introduce meta characters representing any character and the number of character occurrences and character classes representing character replacement to build a final regular expression that can accommodate certain spelling errors or variants; Scan the processed non-duplicate texts using the final regular expression to compare each word, word group, or text segment in the processed non-duplicate texts with the entries in the sensitive database one by one; During the matching process, if the text content matches any entry in the sensitive database, it is determined that sensitive information has been detected; Record all detected sensitive information and their positions in the text, and for each detected sensitive information, use at least one of the following marking methods: highlighting, replacing with a placeholder, and adding a label to mark in the non-duplicate text.

7. A stay frame-based video text sensitive information inspection system characterized by comprising: ​ The feature vector generation module is configured to continuously extract a frame sequence from the acquired video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for representing visual features of the frame. The stay frame extraction module is configured to obtain matching feature point pairs of adjacent frames by comparing similarities of feature vectors of feature points between the adjacent frames, and identify and extract a stay frame based on dynamic behaviors between the matching feature point pairs of the adjacent frames. The text recognition module is configured to perform text recognition on the stay frame by using a pre-trained Paddle OCR deep learning model to obtain text content embedded in the video. The duplicate elimination and matching module is configured to perform similarity calculation on the recognized text content to eliminate duplicate texts, and perform single sensitive word, sensitive word group or sensitive sentence segment matching on the processed non-duplicate texts to detect and mark sensitive information present. The process of identifying and extracting stationary frames based on dynamic behavior between matching feature point pairs in adjacent frames includes: initializing a new sequence of similar frames to analyze the number of matching points in consecutive adjacent frames; and connecting two adjacent video frames W... k W k+1 The number of matching points is added to the new similar frame sequence as the starting element of the sequence; the next adjacent video frame W is calculated. k+2 The standard deviation of the number of matching points between a new sequence of similar frames and existing frames is E = D. ε (W0,W k+1 Simultaneously calculate the next adjacent video frame W. k+2 The difference D between the number of matching points of existing frames in the new similar frame sequence and the number of matching points of existing frames. a (W k+1 W k+2 And the average number of matching points D within the new similar frame sequence. a (W0,W k+1 The absolute value of the difference between the two frames is ΔD; if ΔD ≤ N × E, where N is the preset magnification factor, then the next adjacent video frame W will be... k+2 Add it to a new similar frame sequence; if ΔD > N×E, select the middle frame from the new similar frame sequence as the dwell frame; after extracting the dwell frame, reset the similar frame sequence and start a new sequence construction and dwell frame recognition process from the next frame until the dwell frame is identified and extracted from all adjacent frames; collect all the extracted dwell frames to form the dwell frame set of the video.

8. A stay frame-based video text sensitive information inspection apparatus characterized by comprising: The system comprises: at least one database; and a memory in communication connection with the at least one database; wherein the memory stores instructions executable by the at least one database, and the instructions are executed by the at least one database to enable the at least one database to perform the stay frame-based video text sensitive information checking method according to any one of claims 1-6.

9. A computer readable medium having stored thereon computer- executable instructions, characterized in that, The executable instructions are executed by the processor to implement the stay frame-based video text sensitive information checking method according to any one of claims 1-6.

Images