Video text sensitive information checking method, system and equipment based on stay frame and medium

Abstract

The invention relates to a video text sensitive information checking method, system and device based on stay frames and a medium, and the method comprises the steps: continuously extracting a frame sequence from video data, and carrying out the detection and description of key points of each frame in the frame sequence, so as to generate a feature vector; matching feature point pairs of the adjacent frames are obtained by comparing the similarity of the feature vectors between the adjacent frames, and stay frames are identified and extracted based on dynamic behaviors between the matching feature point pairs of the adjacent frames; performing text recognition on the stay frame by adopting a Paddle OCR deep learning model to obtain text content embedded in the video; the method comprises the following steps: performing similarity calculation on text contents to eliminate repeated texts, and performing sensitive information matching and marking on non-repeated texts; therefore, by deeply understanding the video data and accurately extracting the text information, the defect of the traditional information security means in the aspect of dealing with the leakage of illegal shooting sensitive information of the high-resolution camera equipment is overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of video information recognition, and in particular to a method, system, device and medium for checking sensitive information in video text based on a stay frame. Background Art

[0002] With the continuous advancement of mobile communication technology, the camera function of devices such as smartphones has been significantly improved. High-resolution camera technology has not only greatly enriched people's daily lives and brought convenience to social interaction, entertainment and work, but also posed new challenges to the field of information security. Since these devices can easily capture high-quality images and videos, criminals may take advantage of this technical feature to illegally shoot sensitive paper documents and generate high-definition video files. Once these videos containing sensitive information are disseminated through the Internet and other channels, they will pose a serious threat to personal privacy, commercial secrets and even national security.

[0003] Therefore, the field of information security is facing a new problem of how to effectively detect and prevent the leakage of sensitive information. Although traditional information security methods, such as encryption technology and access control, can protect data security to a certain extent, it is difficult to fundamentally prevent sensitive information from being leaked through illegal filming. This is because these traditional methods mainly focus on the security of data storage and transmission, but are unable to protect the security of data when it is filmed into video. Therefore, the field of information security urgently needs a new technical method that can effectively detect and prevent the leakage of sensitive information through illegal filming. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for checking sensitive information in video text based on dwell frames, which solves the technical problem that traditional information security measures cannot effectively prevent the leakage of sensitive information through illegal shooting of high-resolution camera equipment.

[0006] (II) Technical solution

[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, an embodiment of the present invention provides a method for checking sensitive information in a video text based on a stay frame, comprising:

[0009] Continuously extracting frame sequences from the acquired video data, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for characterizing the visual features of the frame;

[0010] By comparing the similarity of feature vectors of feature points between adjacent frames, matching feature point pairs of adjacent frames are obtained, and the stay frames are identified and extracted based on the dynamic behavior between the matching feature point pairs of adjacent frames;

[0011] Use the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame to obtain the text content embedded in the video;

[0012] By calculating the similarity of the identified text content to exclude duplicate text, and matching the processed non-duplicate text with single sensitive words, sensitive phrases or sensitive paragraphs, the system can detect and mark the existing sensitive information.

[0013] Optionally, continuously extracting a frame sequence from the acquired video data, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for characterizing the visual features of the frame includes:

[0014] Continuously extracting frame sequences from the acquired video data, selecting frames from the video as processing objects at set time intervals to form a frame sequence;

[0015] For each frame image in the frame sequence, a Gaussian blur function is applied to perform multi-scale spatial representation of the image to form a Gaussian pyramid;

[0016] Perform a difference operation on the constructed Gaussian pyramid to generate a Gaussian difference pyramid;

[0017] Based on the Gaussian difference pyramid, the key points are preliminarily determined by searching for local extreme points in the three-dimensional scale space;

[0018] The positions and scales of the key points initially determined are optimized using Taylor expansion, and points whose contrast does not meet the preset contrast conditions or whose edge responses do not meet the edge response conditions are eliminated;

[0019] Calculate the gradient direction and magnitude of the optimized key point neighborhood, and assign one or more dominant directions to the key point according to the gradient direction and magnitude to achieve rotation invariance of the feature vector;

[0020] For the optimized key points, a fixed-size window is constructed with the key points as the center to generate a feature vector of a specific dimension for characterizing the visual features of the frame.

[0021] Optionally, by comparing the similarity of feature vectors of feature points between adjacent frames, the matching feature point pairs of adjacent frames are obtained, including:

[0022] Obtain all feature points and corresponding feature vectors of multiple consecutive frames in a frame sequence;

[0023] Using Euclidean distance as the matching criterion, the Euclidean distance between any selected feature point in the current frame and all feature points in adjacent frames is calculated;

[0024] Sort all calculated distances and select the feature point with the closest distance as the matching point of any selected feature point;

[0025] After multiple iterations of matching, until all feature points in the current video frame have found matching points or it is determined that there are no matching points, a series of matching point pairs between adjacent frames are obtained;

[0026] Optionally, identifying dynamic behaviors between matching feature point pairs of adjacent frames and extracting stay frames includes:

[0027] Initialize a new similar frame sequence to analyze the number of matching points in consecutive adjacent frames;

[0028] The two adjacent video frames W k ,W k+1 The number of matching points is added to the new similar frame sequence as the starting element of the sequence;

[0029] Calculate the next adjacent video frame W k+2 The standard deviation of the number of matching points in the new similar frame sequence is E = D ε (W 0 ,W k+1 ), and calculate the next adjacent video frame W k+2 The difference D between the number of matching points of the existing frames in the new similar frame sequence a (W k+1 ,W k+2 ) and the average number of matching points in the new similar frame sequence D a (W 0 ,W k+1 ) is the absolute value ΔD of the difference between

[0030] If ΔD≤N×E, where N is the preset magnification factor, the next adjacent video frame W k+2 Add to a new sequence of similar frames;

[0031] If ΔD>N×E, then select the middle frame from the new similar frame sequence as the dwell frame;

[0032] After the stay frame is extracted, the similar frame sequence is reset, and a new sequence construction and stay frame identification process is started from the next frame until the stay frame is identified and extracted from all adjacent frames;

[0033] All the extracted stay frames are collected together to form a stay frame set of the video.

[0034] Optionally, use the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame to obtain the text content embedded in the video, including:

[0035] Select and build the Paddle OCR deep learning model based on text recognition requirements;

[0036] Collect image datasets containing text samples of various fonts, sizes, directions, and backgrounds, and introduce expert experience to annotate the location of the text area and the corresponding text content of the text samples;

[0037] Train the Paddle OCR deep learning model based on the annotated image dataset;

[0038] The stay frame is input into the trained Paddle OCR model to locate and segment the text area of ​​the stay frame, identify the located text area, and output the text content of each text area. The text content of each text area includes video title, subtitle, bullet comment, watermark, advertising text and video text.

[0039] Optionally, similarity calculation is performed on the identified text content to exclude duplicate texts including:

[0040] Perform word segmentation on the recognized text content, dividing the continuous text into independent vocabulary units;

[0041] Using the TF-IDF algorithm, the TF-IDF weight of each word in the current text content is calculated based on the frequency of the word in the current text and the scarcity of the word in the preset corpus, and the top N words with the highest TF-IDF weight are selected as key feature words;

[0042] Performing hash processing on the selected key feature words so that each key feature word generates a binary hash value of a fixed length;

[0043] According to the hash value and TF-IDF weight of each key feature word, a corresponding weight vector is generated, where the bits with hash value 1 in the weight vector are assigned positive TF-IDF weights, and the bits with hash value 0 are assigned negative TF-IDF weights;

[0044] The weight vectors of all key feature words are accumulated column by column to obtain the accumulated vector representing the overall features of the text;

[0045] Generate a Simhash value of a fixed-length text based on the positive or negative value of each bit of the accumulated vector, where positive numbers correspond to 1 and negative numbers correspond to 0;

[0046] Calculate the Hamming distance between the Simhash values ​​of two texts and determine whether the Hamming distance is less than the set threshold;

[0047] When the Hamming distance of the Simhash values ​​of two texts is less than or equal to the set threshold, the two texts are judged to be similar texts;

[0048] Traverse the text content of all the stay frames by sliding the window to obtain the repeated text similar to the recognized text;

[0049] In the recognized text content, duplicate texts similar to the recognized texts are removed, and non-duplicate texts are retained as the final result.

[0050] Optionally, the processed non-repetitive text is matched with a single sensitive word, a sensitive phrase, or a sensitive paragraph to detect and mark the existing sensitive information including:

[0051] Based on the collected historical sensitive data, generate sensitive data sets containing single sensitive words, sensitive phrases and sensitive paragraphs by separating, disassembling or analyzing variants;

[0052] Construct basic regular expressions for sensitive words, sensitive phrases, and sensitive paragraphs, and introduce metacharacters representing arbitrary characters and the number of times characters appear, as well as character classes representing character replacements, to construct a final regular expression that can accommodate certain spelling errors or variations.

[0053] Scanning the processed non-repeated text using the final regular expression to compare each word, phrase or paragraph in the processed non-repeated text with the entries in the sensitive database one by one;

[0054] During the matching process, once the text content is found to match any entry in the sensitive database, it is determined that sensitive information has been detected;

[0055] All detected sensitive information and its location in the text are recorded, and for each detected sensitive information, at least one of highlighting, replacing with a placeholder, and adding a label is used to mark it in the non-repeating text.

[0056] In a second aspect, an embodiment of the present invention provides a video text sensitive information inspection system based on a stay frame, comprising:

[0057] A feature vector generation module is used to continuously extract a frame sequence from the acquired video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for representing the visual features of the frame;

[0058] The stay frame extraction module is used to obtain matching feature point pairs of adjacent frames by comparing the similarity of feature vectors of feature points between adjacent frames, and to identify and extract the stay frame based on the dynamic behavior between the matching feature point pairs of adjacent frames;

[0059] The text recognition module uses the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame and obtain the text content embedded in the video;

[0060] The duplicate elimination and matching module is used to eliminate duplicate text by calculating the similarity of the identified text content, and to match the processed non-duplicate text with single sensitive words, sensitive phrases or sensitive paragraphs to detect and mark the existing sensitive information.

[0061] In a third aspect, an embodiment of the present invention provides a video text sensitive information inspection device based on a dwell frame, comprising: at least one database; and a memory communicatively connected to the at least one database; wherein the memory stores instructions executable by the at least one database, and the instructions are executed by the at least one database so that the at least one database can execute the video text sensitive information inspection method based on a dwell frame as described above.

[0062] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer executable instructions stored thereon, and when the executable instructions are executed by a processor, the method for checking sensitive information in video text based on stay frames as described above is implemented.

[0063] (III) Beneficial effects

[0064] The beneficial effects of the present invention are:

[0065] First, the present invention can continuously extract frame sequences from the acquired video data and detect and describe key points, which means that even when the video shooting conditions are poor or the text information is relatively vague, it helps to more accurately capture the dynamic changes in the video and accurately generate feature vectors that characterize the visual features of the frame. This step provides a solid data foundation for subsequent sensitive information detection.

[0066] Secondly, by comparing the similarity of feature vectors of adjacent frames, the dwell frames are intelligently identified and extracted. This innovation effectively utilizes the characteristic that the lens stays for a long time in video shooting. From this dwell frame recognition method based on dynamic behavior, we can deeply understand and consider the behavior patterns of the video shooter, so that the present invention performs well in processing complex and changeable video data.

[0067] Furthermore, using the pre-trained Paddle OCR deep learning model to perform text recognition on the pause frame can quickly and accurately obtain the text content in the video, and enable this method to handle text information in various fonts, sizes and colors. Compared with traditional methods, this is not only faster but also more accurate.

[0068] Finally, similarity calculation effectively removes duplicate texts, reduces the interference of redundant information, and makes the detection of sensitive information more accurate and efficient. At the same time, the processed non-duplicate texts are carefully matched with sensitive words, phrases or paragraphs, which can timely discover and mark potential sensitive information, providing strong technical support for information security.

[0069] Therefore, the video text sensitive information inspection method based on dwell frames of the present invention solves the shortcomings of traditional information security measures in dealing with the leakage of sensitive information illegally filmed by high-resolution camera equipment by deeply understanding the video data and accurately extracting text information, greatly improves the efficiency and accuracy of video sensitive information detection, and provides strong technical support for the field of information security. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a flow chart of a method provided by an embodiment of the present invention;

[0071] Figure 2 A schematic diagram of a specific flow chart of step S1 of the method provided in an embodiment of the present invention;

[0072] Figure 3 A schematic diagram of a specific flow chart of the first part of step S2 of the method provided in an embodiment of the present invention;

[0073] Figure 4 A schematic diagram of adjacent video frame matching points of the method provided by an embodiment of the present invention;

[0074] Figure 5 A schematic diagram of a specific flow chart of the second part of step S2 of the method provided in an embodiment of the present invention;

[0075] Figure 6 A schematic diagram of the number of matching points in a video frame of the method provided by an embodiment of the present invention;

[0076] Figure 7 A schematic diagram of a stay frame recognition method provided by an embodiment of the present invention;

[0077] Figure 8 A schematic diagram of a specific flow chart of step S3 of the method provided in an embodiment of the present invention;

[0078] Fig. 9 A schematic diagram of a specific flow chart of the first part of step S4 of the method provided in an embodiment of the present invention

[0079] Fig.10 The simhash construction process of the method provided in the embodiment of the present invention;

[0080] Fig.11 A schematic diagram of a specific flow chart of the second part of step S4 of the method provided in an embodiment of the present invention;

[0081] Fig.12 The present invention provides a schematic diagram of the overall process of the method. DETAILED DESCRIPTION

[0082] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0083] like Figure 1 As shown, a method for checking sensitive information in video text based on dwell frames proposed in an embodiment of the present invention includes: continuously extracting a frame sequence from the acquired video data, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for characterizing the visual features of the frame; obtaining matching feature point pairs of adjacent frames by comparing the similarity of the feature vectors of each feature point between adjacent frames, and identifying and extracting dwell frames based on the dynamic behavior between the matching feature point pairs of adjacent frames; using a pre-trained Paddle OCR deep learning model to perform text recognition on the dwell frames to obtain text content embedded in the video; performing similarity calculation on the recognized text content to exclude repeated text, and matching the processed non-duplicate text with single sensitive words, sensitive phrases or sensitive paragraphs to detect and mark the existing sensitive information.

[0084] First, the present invention can continuously extract frame sequences from the acquired video data and detect and describe key points, which means that even when the video shooting conditions are poor or the text information is relatively vague, it helps to more accurately capture the dynamic changes in the video and accurately generate feature vectors that characterize the visual features of the frame. This step provides a solid data foundation for subsequent sensitive information detection.

[0085] Secondly, by comparing the similarity of feature vectors of adjacent frames, the dwell frames are intelligently identified and extracted. This innovation effectively utilizes the characteristic that the lens stays for a long time in video shooting. From this dwell frame recognition method based on dynamic behavior, we can deeply understand and consider the behavior patterns of the video shooter, so that the present invention performs well in processing complex and changeable video data.

[0086] Furthermore, using the pre-trained Paddle OCR deep learning model to perform text recognition on the pause frame can quickly and accurately obtain the text content in the video, and enable this method to handle text information in various fonts, sizes and colors. Compared with traditional methods, this is not only faster but also more accurate.

[0087] Finally, similarity calculation effectively removes duplicate texts, reduces the interference of redundant information, and makes the detection of sensitive information more accurate and efficient. At the same time, the processed non-duplicate texts are carefully matched with sensitive words, phrases or paragraphs, which can timely discover and mark potential sensitive information, providing strong technical support for information security.

[0088] Therefore, the video text sensitive information inspection method based on dwell frames of the present invention solves the shortcomings of traditional information security measures in dealing with the leakage of sensitive information illegally filmed by high-resolution camera equipment by deeply understanding the video data and accurately extracting text information, greatly improves the efficiency and accuracy of video sensitive information detection, and provides strong technical support for the field of information security.

[0089] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0090] Specifically, an embodiment of the present invention provides a method for checking sensitive information in video text based on a stay frame, comprising:

[0091] S1. Continuously extract frame sequences from the acquired video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for characterizing the visual features of the frame.

[0092] Furthermore, if Figure 2 As shown, step S1 includes:

[0093] S11, continuously extracting a frame sequence from the acquired video data, selecting frames from the video as processing objects at set time intervals, and forming a frame sequence.

[0094] S12. For each frame image in the frame sequence, a Gaussian blur function is applied to perform multi-scale spatial representation on the image to form a Gaussian pyramid.

[0095] S13. Perform a difference operation on the constructed Gaussian pyramid to generate a Gaussian difference pyramid.

[0096] S14. Based on the Gaussian difference pyramid, the key points are preliminarily determined by searching for local extreme points in the three-dimensional scale space.

[0097] S15. Optimize the positions and scales of the key points initially determined using Taylor expansion, and remove points whose contrast does not meet the preset contrast condition or whose edge response does not meet the edge response condition.

[0098] S16. Calculate the gradient direction and magnitude of the optimized key point neighborhood, and assign one or more dominant directions to the key point according to the gradient direction and magnitude to achieve rotation invariance of the feature vector.

[0099] S17. For the optimized key point, a fixed-size window is constructed with the key point as the center to generate a feature vector of a specific dimension for characterizing the visual features of the frame.

[0100] In a specific embodiment, the SIFT feature extraction algorithm processing flow of the video frame is as follows: First, extract the frame sequence F from the video at time intervals. 1 , F 2 , ..., F n As input. Then, the key points are detected by constructing Gaussian pyramid and Gaussian difference pyramid. The Gaussian blur function is The blurred image L(x, y, σ) = G(x, y, σ) * I(x, y), the difference is D(x, y, σ) = L(x, y, kσ) - L(x, y, σ), and the key points are initially determined by finding local extreme points in 3D space. Then, the Taylor expansion is used to optimize the key point position And remove points with low contrast or strong edge response. Then, calculate the gradient direction and amplitude of the key point neighborhood Direction information is assigned to the key points to achieve rotation invariance. Finally, a 128-dimensional feature vector is generated as a feature descriptor by constructing a 16*16 window.

[0101] S2. By comparing the similarity of feature vectors of feature points between adjacent frames, matching feature point pairs of adjacent frames are obtained, and based on the dynamic behavior between the matching feature point pairs of adjacent frames, the stay frame is identified and extracted.

[0102] Further, refer to Figure 3 In step S2, by comparing the similarity of the feature vectors of each feature point between adjacent frames, the matching feature point pairs of adjacent frames are obtained, including:

[0103] S21. Obtain all feature points and corresponding feature vectors of multiple consecutive frames in a frame sequence.

[0104] S22, using the Euclidean distance as a matching criterion, and calculating the Euclidean distance between any selected feature point in the current frame and all feature points in adjacent frames.

[0105] S23, sorting all the calculated distances, and selecting a feature point with the closest distance as a matching point for any selected feature point.

[0106] S24, after multiple iterations of matching, until all feature points in the current video frame have found matching points or it is determined that there are no matching points, a series of matching point pairs between adjacent frames are obtained.

[0107] After the feature vector of a video frame is generated, a feature point matching algorithm is needed to find the matching point of a feature point in an adjacent frame. In this embodiment, Euclidean distance is used for feature point matching, and the specific steps are as follows:

[0108] First, multiple consecutive frames in the video frame sequence are obtained. For each frame, all feature points are extracted using feature extraction algorithms (such as SIFT, SURF, etc.) and corresponding feature vectors are generated. These feature vectors are the basis for subsequent feature point matching.

[0109] After obtaining all feature points and corresponding feature vectors, the matching of feature points begins. Here, Euclidean distance is used as the matching criterion. Specifically, any feature point in the current frame is selected, and then the Euclidean distance between the feature point and all feature points in the adjacent frames is calculated. The calculation formula of Euclidean distance is: Where p and q are two eigenvectors to be compared, and n is the dimension of the eigenvector.

[0110] Next, after calculating the Euclidean distance between the current feature point and all feature points in the adjacent frames, a distance list is obtained. Then, the distance list is sorted and the feature point with the closest distance is selected as the matching point of the currently selected feature point. The reason for this is that in the feature space, the closer the distance between two feature points is, the more likely it is that they correspond to different perspectives or states of the same object or scene, so the matching degree between them is higher.

[0111] In order to find the matching points of all feature points in the current video frame, new feature points are continuously selected for calculation and matching until all feature points in the current video frame have found matching points or it is determined that there are no matching points. Figure 4 , and obtain a series of matching point pairs between adjacent frames. These matching point pairs can be used for subsequent video analysis, target tracking and other tasks.

[0112] And, further, if Figure 5 As shown, the dynamic behavior recognition based on the matching feature point pairs between adjacent frames and the extraction of the stay frame in step S2 includes:

[0113] S25, initializing a new similar frame sequence for analyzing the number of matching points of consecutive adjacent frames.

[0114] S26, two adjacent video frames W k ,W k+1 The number of matching points is added to the new similar frame sequence as the starting element of the sequence.

[0115] S27, calculate the next adjacent video frame W k+2 The standard deviation of the number of matching points in the new similar frame sequence is E = D ε (W 0 ,W k+1 ), and calculate the next adjacent video frame W k+2 The difference D between the number of matching points of the existing frames in the new similar frame sequence a (W k+1 ,W k+2 ) and the average number of matching points in the new similar frame sequence D a (W 0 ,W k+1 ) is the absolute value ΔD of the difference between them.

[0116] S28, if ΔD≤N×E, where N is the preset magnification factor, the next adjacent video frame W k+2 Add it to the new similar frame sequence; if ΔD>N×E, select the middle frame from the new similar frame sequence as the dwell frame.

[0117] S29, after the stay frame is extracted, the similar frame sequence is reset, and a new sequence construction and stay frame identification process is started from the next frame, until the stay frame is identified and extracted from all adjacent frames; all the extracted stay frames are collected to form a stay frame set of the video.

[0118] In a specific embodiment, in order for the video reader to clearly see the text information in the video, the camera needs to stay on a page of text for a longer time, and this time change will be reflected in the number of adjacent frames, that is, the number of frame matching points for shooting a page of text is relatively stable, and when the video shooter flips the paper text, it will cause the number of frame matching points to fluctuate greatly. Figure 6 This is an example of the number of video frame matching points of a video file frame. From this figure, we can see that there are some relatively stable matching points in the number of matching points of the video frame. These matching points contain a lot of repeated information, and these highly repeated texts can be removed to achieve the recognition of the stay frame. To recognize the stay frame, you need to first construct the following formula:

[0119] (1) The difference in the number of matching points D a , indicating the number of matching points W kWith W m difference.

[0120] D a (W k , W m )=W m -W k ;

[0121] (2) The average number of matching points in a similar frame sequence Dv, indicating that the number of the first matching point in the sequence is W k With the last one being W m The average number of matching points.

[0122]

[0123] (3) Standard deviation D of the number of matching points in similar frame sequences ε , indicating that the number of the first matching points in the sequence is W k With the last one being W m The standard deviation of the number of matching points.

[0124]

[0125] Next, if Figure 7 As shown, the specific steps of stay frame recognition are as follows: First, create a new similar frame sequence QF m (m≥1); Secondly, the number of matching points of two adjacent frames W k ,W k+1 Join to similar frame sequence QF m Next, calculate the next frame W k+2 With similar frame sequence QF m The standard deviation of E = D ε (W 0 ,W k+1 ) and the average number of matching points in the new similar frame sequence D a (W 0 ,W k+1 The absolute value of the difference between ΔD and D a (W k+1 , W k+2 )-D v (W 0 , W k+1 )|.

[0126] Therefore, if ΔD≤N×E, where N is the preset magnification, then W k+2 Add to the similar frame sequence and return to this step: "Calculate the next frame W k+2 With similar frame sequence QF m The standard deviation of E = D ε (W 0,W k+1 ) and the average number of matching points in the new similar frame sequence D a (W 0 ,W k+1 The absolute value of the difference between ΔD and D a (W k+1 , W k+2 )-D v (W 0 , W k+1 )|”; if ΔD≤N×E, then in the similar frame sequence QF m Take the middle frame as the dwell frame and return to the starting step by taking m+1.

[0127] S3. Use the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame to obtain the text content embedded in the video.

[0128] Furthermore, if Figure 8 As shown, step S3 includes:

[0129] S31. Select and build the Paddle OCR deep learning model based on text recognition requirements.

[0130] It is important to understand that text recognition is the process of identifying and extracting the text contained in the target area. Currently, there are a large number of deep learning models that can accurately recognize English text. However, since Chinese contains more characters and words than English and the composition of Chinese characters is more complex, these models have lower accuracy when recognizing Chinese data. The Paddle OCR deep learning model is a text recognition model based on deep learning developed by Baidu. Compared with other deep learning models, it supports more Chinese recognition scenarios and has a higher recognition accuracy in Chinese scenarios.

[0131] S32. Collect an image dataset containing text samples of various fonts, sizes, directions and backgrounds, and introduce expert experience to annotate the positions of text areas of the text samples and the corresponding text contents.

[0132] S33. Train the Paddle OCR deep learning model based on the annotated image dataset.

[0133] S34, input the stay frame into the trained Paddle OCR model to locate and segment the text area of ​​the stay frame, identify the located text area, and output the text content of each text area. The text content of each text area includes video title, subtitle, bullet screen, watermark, advertising text and video text.

[0134] S4. Calculate the similarity of the identified text content to exclude duplicate texts, and match the processed non-duplicate texts with single sensitive words, sensitive phrases, or sensitive paragraphs to detect and mark the existing sensitive information.

[0135] During the stay frame extraction process, although the stay frame extraction algorithm can remove redundant video frames, there are still duplicate video frames remaining. Therefore, in the text post-processing stage, these duplicate texts must be further excluded. In order to identify duplicate texts, text similarity calculation must be performed.

[0136] Furthermore, if Fig. 9 As shown, in step S4, similarity calculation is performed on the identified text content to exclude duplicate texts, including:

[0137] S41, performing word segmentation processing on the recognized text content, dividing the continuous text into independent vocabulary units.

[0138] S42. Using the TF-IDF algorithm, the TF-IDF weight of each word in the current text content is calculated based on the frequency of the word in the current text and the scarcity of the word in the preset corpus, and the top N words with the highest TF-IDF weights are selected as key feature words.

[0139] S43, performing hash processing on the selected key feature words so that each key feature word generates a binary hash value of a fixed length.

[0140] S44. Generate a corresponding weight vector according to the hash value and TF-IDF weight of each key feature word, wherein in the weight vector, the bit with a hash value of 1 is assigned a positive TF-IDF weight, and the bit with a hash value of 0 is assigned a negative TF-IDF weight.

[0141] S45. The weight vectors of all key feature words are accumulated column by column to obtain an accumulated vector representing the overall features of the text.

[0142] S46. Generate a Simhash value of a text of a fixed length according to the positive or negative value of each bit of the accumulated vector, where a positive number corresponds to 1 and a negative number corresponds to 0.

[0143] S47, calculating the Hamming distance between the Simhash values ​​of the two texts, and determining whether the Hamming distance is less than a set threshold.

[0144] S48. When the Hamming distance of the Simhash values ​​of two texts is less than or equal to a set threshold, the two texts are determined to be similar texts.

[0145] S49, traversing the text contents of all the stay frames by means of a sliding window to obtain repeated texts similar to the recognized texts; in the recognized text contents, removing repeated texts similar to the recognized texts, and retaining non-repeated texts as the final result.

[0146] In yet another specific embodiment, reference Fig.10 , the steps to identify and exclude duplicate text content are as follows:

[0147] (1) Word segmentation and TF-IDF calculation: In order to convert the text content in the video frame into digital features, the text of the video frame needs to be segmented first. The text after word segmentation will be used for subsequent calculations. Specifically, the TF-IDF (term frequency-inverse document frequency) method is used to weight each word and calculate the importance of each word. The TF-IDF algorithm measures the importance of a word in a document. It combines the frequency of the word in the current text (TF) and the scarcity of the word in the entire corpus (IDF). The calculation method is:

[0148]

[0149] Among them, TF(t, d) represents the number of times word t appears in document d, DF(t) is the number of documents containing word t, and N is the total number of documents.

[0150] For each video frame, the top 20 words with the highest TF-IDF weights are selected, and these words are considered to be the key features of the text of the frame. In this way, each video frame text can be converted into a set of 20 words, and the higher the weight of these words, the greater their importance in the text.

[0151] (2) Hash conversion: Hash the 20 important words selected in step 1. Each word generates a 64-bit binary value through a conventional hash function, and this binary value represents the "fingerprint" of the word. This hash value is calculated based on the text information of the word, so different text content will generate different hash values. Through this process, the text of each video frame is converted into a set of length 20, in which each element is a 64-bit binary number, representing the hash value of the word in the text.

[0152] (3) Calculate the weight vector: Next, generate a weight vector based on the hash value of each word. For each bit in the hash value of each word, if the bit is 1, the TF-IDF weight of the word is assigned a positive value, and if the bit is 0, it is assigned a negative value. For example, assuming that the hash value of a word is 010111 and the TF-IDF weight of the word is 5, the weight vector generated based on the hash value is:

[0153] [-5,5,-5,5,5,5];

[0154] Here, a 1 bit in the hash value corresponds to a positive weight, and a 0 bit corresponds to a negative weight. This process converts the 20 words in each video frame into a set of weight vectors of length 64.

[0155] (4) Column vector accumulation: For each of the 20 hash words in each video frame, its corresponding weight vector is accumulated column by column. Assume there are three sets of weight vectors:

[0156] [-5,5,-5,5,5,5],[-3,-3,-3,3,-3,3],[1,-1,-1,1,1,1];

[0157] Add these weight vectors column by column to get a new accumulated vector:

[0158] [-7,1,-9,9,3,9];

[0159] This accumulated vector is the "feature vector" of the text in the current video frame, which summarizes the overall characteristics of the text in the video frame.

[0160] (5) Generate Simhash value: After obtaining the accumulated vector, it needs to be converted into a 64-bit binary Simhash value. The value of each position is judged: if the value is negative, the bit is 0; if the value is positive, the bit is 1.

[0161] For example, the conversion result of the accumulated vector [-7,1,-9,9,3,9] is:

[0162] simhash=010111;

[0163] This 64-bit binary value is the Simhash value of the video frame, which can represent the characteristics of the text in the frame.

[0164] (6) Calculating Similarity To determine the similarity between the texts of two video frames, we need to calculate the Hamming distance between their Simhash values. The Hamming distance is the number of bits in different positions between two binary strings of the same length.

[0165]

[0166] in, is an indicator function. When two corresponding bits are different, the result is 1, otherwise it is 0. If the Hamming distance of two Simhash values ​​is less than or equal to 3, the text contents of the two video frames are considered to be similar. If the Hamming distance is greater than 3, they are considered to be dissimilar.

[0167] (7) Similarity processing When the text content of two video frames is similar, it is necessary to remove the recognized text content from the similar frame sequence. This can avoid repeated calculations and improve processing efficiency. After removal, return to step 1 to continue processing. If the text of the two video frames is not similar, the sliding window moves back one position and continues to check the text content of the next frame. When the sliding window traverses the text content of all the frames it stays on, the process ends.

[0168] And, if Fig.11 As shown, in step S4, matching a single sensitive word, a sensitive phrase or a sensitive paragraph on the processed non-repetitive text to detect and mark the existing sensitive information includes:

[0169] A41. Based on the collected historical sensitive data, generate sensitive data sets containing single sensitive words, sensitive phrases and sensitive paragraphs by separating, disassembling or analyzing variants.

[0170] A42. Construct basic regular expressions for sensitive words, sensitive phrases, and sensitive paragraphs, and introduce metacharacters that represent arbitrary characters and the number of times characters appear, as well as character classes that represent character replacements, on the basis of the basic regular expressions to construct a final regular expression that can accommodate spelling errors or variations.

[0171] In a specific embodiment, based on the collected historical sensitive data, a sensitive data set containing a single sensitive word, a sensitive phrase, and a sensitive paragraph is generated by separating, disassembling, or analyzing variants, wherein for each sensitive word or phrase, its possible variant forms are analyzed. Variants may include, but are not limited to: spelling errors, synonym replacements, word order changes, adding or deleting non-key characters, etc. Then, a basic regular expression is constructed based on the exact form of the sensitive word or phrase. Ensure that this expression can accurately match the target text. On the basis of the basic regular expression, a regular expression that can accommodate certain spelling errors or variants is constructed by adding appropriate metacharacters (such as. represents any character, * represents that the preceding character can appear zero or more times,? represents that the preceding character can appear zero or once, etc.). Use character classes (such as [az] represents any lowercase letter) to match possible character replacements. In addition, it is also possible to consider introducing fuzzy matching techniques, such as the approximate matching function in regular expressions, to capture a wider range of variants.

[0172] A43. Use the final regular expression to scan the processed non-repetitive text to compare each word, phrase or paragraph in the processed non-repetitive text with the entries in the sensitive database one by one.

[0173] It should be emphasized that in the embodiment of the present invention, a string matching algorithm (such as KMP algorithm, BM algorithm, etc.) can also be used to scan the processed non-repetitive text.

[0174] A44. During the matching process, once the text content is found to match any entry in the sensitive database, it is determined that sensitive information has been detected.

[0175] A45. Record all detected sensitive information and its location in the text, and mark each detected sensitive information in non-repeated text by highlighting, replacing with placeholders, or adding labels. This requires ensuring that the marking method can clearly point out the existence of sensitive information while not destroying the overall structure and meaning of the original text.

[0176] On the other hand, an embodiment of the present invention provides a video text sensitive information inspection system based on a stay frame, comprising:

[0177] The feature vector generation module is used to continuously extract frame sequences from the acquired video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for characterizing the visual features of the frame.

[0178] The stay frame extraction module is used to obtain matching feature point pairs of adjacent frames by comparing the similarity of feature vectors of feature points between adjacent frames, and to identify and extract stay frames based on the dynamic behavior between the matching feature point pairs of adjacent frames.

[0179] The text recognition module is used to use the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame and obtain the text content embedded in the video.

[0180] The duplicate elimination and matching module is used to eliminate duplicate text by calculating the similarity of the identified text content, and to match the processed non-duplicate text with single sensitive words, sensitive phrases or sensitive paragraphs to detect and mark the existing sensitive information.

[0181] Furthermore, an embodiment of the present invention provides a video text sensitive information inspection device based on stay frames, comprising: at least one database; and a memory communicatively connected to the at least one database; wherein the memory stores instructions executable by the at least one database, and the instructions are executed by the at least one database so that the at least one database can execute the video text sensitive information inspection method based on stay frames as described above.

[0182] The embodiment of the present invention introduces an innovative video text sensitive information inspection device, which operates based on the technology of staying in frames. The core components of the device include at least one database and a memory that is connected to the database in communication.

[0183] The memory here is not only used for data storage, but more importantly, it stores instructions that can be executed by the database. When these instructions are executed by the database, the video text sensitive information inspection method based on the stay frame can be implemented. The execution of this method can effectively identify sensitive text information in the video, further improving the efficiency of information security and auditing.

[0184] Simply put, this device, through the close cooperation of database and memory and the use of a special instruction set, enables efficient and accurate sensitive information inspection of the text content of the frames in the video.

[0185] At the same time, an embodiment of the present invention provides a computer-readable medium on which computer executable instructions are stored. When the executable instructions are executed by a processor, the above-mentioned video text sensitive information inspection method based on the stay frame is implemented.

[0186] The embodiment of the present invention also provides a computer-readable medium on which specific computer executable instructions are stored. When these instructions are executed by a processor, the above-mentioned video text sensitive information inspection method based on the stay frame can be implemented. Through this medium, users can easily deploy the method on various computing devices, thereby realizing rapid and accurate inspection of sensitive information in video texts, and improving the efficiency and reliability of information security protection. This computer-readable medium provides a convenient and efficient solution for video text sensitive information inspection.

[0187] In summary, the present invention provides a method, system, device and medium for checking sensitive information in video text based on a stay frame. Fig.12 First, it is necessary to extract the video frames. This process involves extracting image data frame by frame from the original video file, laying the foundation for subsequent processing steps. After extracting the video frames, the next step is to process these frame images using an algorithm called Scale-Invariant Feature Transform (SIFT). The SIFT algorithm can extract scale-invariant feature points from video images. These feature points are highly robust and can remain stable under different scales and rotation conditions. Through data analysis of these feature points, the dwell frames in the video can be effectively identified, thereby reducing the impact of redundant video frames on subsequent text information extraction.

[0188] After the dwell frame recognition is completed, the next step is to use the Paddle OCR deep learning model to recognize the text in the video frame. Paddle OCR is a deep learning-based optical character recognition model that can efficiently extract text information from images. By inputting the video frame into the Paddle OCR model, accurate recognition of the text information contained in the video can be achieved.

[0189] Finally, the identified text is further post-processed. This process includes removing redundant text information to ensure that the final output text content is concise and targeted. In addition, the text needs to be matched for sensitive words to identify possible sensitive information. After the matching is completed, the text content is output and the sensitive words are highlighted to facilitate users to quickly identify and process this sensitive information. Through this series of steps, the video file sensitive information detection process can effectively identify and process sensitive information in the video, ensuring the security and compliance of the content.

[0190] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0191] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0192] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0193] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0194] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A method for checking sensitive information in video text based on a stay frame, characterized in that: include: Continuously extracting frame sequences from the acquired video data, and detecting and describing key points of each frame in the frame sequence to generate a feature vector for characterizing the visual features of the frame; By comparing the similarity of feature vectors of feature points between adjacent frames, matching feature point pairs of adjacent frames are obtained, and the stay frames are identified and extracted based on the dynamic behavior between the matching feature point pairs of adjacent frames; Use the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame to obtain the text content embedded in the video; By calculating the similarity of the identified text content to exclude duplicate text, and matching the processed non-duplicate text with single sensitive words, sensitive phrases or sensitive paragraphs, the system can detect and mark the existing sensitive information.

2. The method for checking sensitive information in video text based on stay frames as claimed in claim 1, characterized in that: Frame sequences are continuously extracted from the acquired video data, and key points are detected and described for each frame in the frame sequence to generate feature vectors for characterizing the visual features of the frame, including: Continuously extracting frame sequences from the acquired video data, selecting frames from the video as processing objects at set time intervals to form a frame sequence; For each frame image in the frame sequence, a Gaussian blur function is applied to perform multi-scale spatial representation of the image to form a Gaussian pyramid; Perform a difference operation on the constructed Gaussian pyramid to generate a Gaussian difference pyramid; Based on the Gaussian difference pyramid, the key points are preliminarily determined by searching for local extreme points in the three-dimensional scale space; The positions and scales of the key points initially determined are optimized using Taylor expansion, and points whose contrast does not meet the preset contrast conditions or whose edge responses do not meet the edge response conditions are eliminated; Calculate the gradient direction and magnitude of the optimized key point neighborhood, and assign one or more dominant directions to the key point according to the gradient direction and magnitude to achieve rotation invariance of the feature vector; For the optimized key points, a fixed-size window is constructed with the key points as the center to generate a feature vector of a specific dimension for characterizing the visual features of the frame.

3. The method for checking sensitive information in video text based on stay frames as claimed in claim 1, characterized in that: By comparing the similarity of the feature vectors of each feature point between adjacent frames, the matching feature point pairs of adjacent frames are obtained, including: Obtain all feature points and corresponding feature vectors of multiple consecutive frames in a frame sequence; Using Euclidean distance as the matching criterion, the Euclidean distance between any selected feature point in the current frame and all feature points in adjacent frames is calculated; Sort all calculated distances and select the feature point with the closest distance as the matching point of any selected feature point; After multiple iterations of matching, until all feature points in the current video frame have found matching points or it is determined that there are no matching points, a series of matching point pairs between adjacent frames are obtained.

4. The method for checking sensitive information in video text based on stay frames as claimed in claim 1, characterized in that: Dynamic behavior recognition based on matching feature point pairs between adjacent frames and extraction of stay frames include: Initialize a new similar frame sequence to analyze the number of matching points in consecutive adjacent frames; The two adjacent video frames W k ,W k+1 The number of matching points is added to the new similar frame sequence as the starting element of the sequence; Calculate the next adjacent video frame W k+2 The standard deviation of the number of matching points in the new similar frame sequence is E = D ε (W0,W k+1 ), and calculate the next adjacent video frame W k+2 The difference D between the number of matching points of the existing frames in the new similar frame sequence a (W k+1 ,W k+2 ) and the average number of matching points in the new similar frame sequence D a (W0,W k+1 ) is the absolute value ΔD of the difference between If ΔD≤N×E, where N is the preset magnification factor, the next adjacent video frame W k+2 Add to a new sequence of similar frames; If ΔD>N×E, then select the middle frame from the new similar frame sequence as the dwell frame; After the stay frame is extracted, the similar frame sequence is reset, and a new sequence construction and stay frame identification process is started from the next frame until the stay frame is identified and extracted from all adjacent frames; All the extracted stay frames are collected together to form a stay frame set of the video.

5. The method for checking sensitive information in video text based on stay frames as claimed in claim 1, characterized in that: The pre-trained Paddle OCR deep learning model is used to perform text recognition on the stop frame, and the embedded text content in the video is obtained, including: Select and build the Paddle OCR deep learning model based on text recognition requirements; Collect image datasets containing text samples of various fonts, sizes, directions, and backgrounds, and introduce expert experience to annotate the location of the text area and the corresponding text content of the text samples; Train the Paddle OCR deep learning model based on the annotated image dataset; The dwell frame is input into the trained Paddle OCR model to locate and segment the text area in the dwell frame, identify the located text area, and output the text content of each text area.

6. The method for checking sensitive information in video text based on stay frames as claimed in claim 1, characterized in that: By calculating the similarity of the identified text content, duplicate text is excluded, including: Perform word segmentation on the recognized text content, dividing the continuous text into independent vocabulary units; Using the TF-IDF algorithm, the TF-IDF weight of each word in the current text content is calculated based on the frequency of the word in the current text and the scarcity of the word in the preset corpus, and the top N words with the highest TF-IDF weight are selected as key feature words; Performing hash processing on the selected key feature words so that each key feature word generates a binary hash value of a fixed length; According to the hash value and TF-IDF weight of each key feature word, a corresponding weight vector is generated, where the bits with hash value 1 in the weight vector are assigned positive TF-IDF weights, and the bits with hash value 0 are assigned negative TF-IDF weights; The weight vectors of all key feature words are accumulated column by column to obtain the accumulated vector representing the overall features of the text; Generate a Simhash value of a fixed-length text based on the positive or negative value of each bit of the accumulated vector, where positive numbers correspond to 1 and negative numbers correspond to 0; Calculate the Hamming distance between the Simhash values ​​of two texts and determine whether the Hamming distance is less than the set threshold; When the Hamming distance of the Simhash values ​​of two texts is less than or equal to the set threshold, the two texts are judged to be similar texts; Traverse the text content of all the stay frames by sliding the window to obtain repeated text similar to the recognized text; In the recognized text content, duplicate texts similar to the recognized texts are removed, and non-duplicate texts are retained as the final result.

7. The method for checking sensitive information in video text based on stay frames as claimed in claim 1, characterized in that: The processed non-repeated text is matched with a single sensitive word, sensitive phrase or sensitive paragraph to detect and mark the existing sensitive information including: Based on the collected historical sensitive data, generate sensitive data sets containing single sensitive words, sensitive phrases and sensitive paragraphs by separating, disassembling or analyzing variants; Construct basic regular expressions for sensitive words, sensitive phrases, and sensitive paragraphs, and introduce metacharacters representing arbitrary characters and the number of times characters appear, as well as character classes representing character replacements, to construct a final regular expression that can accommodate certain spelling errors or variations. Scanning the processed non-repeated text using the final regular expression to compare each word, phrase or paragraph in the processed non-repeated text with the entries in the sensitive database one by one; During the matching process, once the text content is found to match any entry in the sensitive database, it is determined that sensitive information has been detected; All detected sensitive information and its location in the text are recorded, and for each detected sensitive information, at least one of highlighting, replacing with a placeholder, and adding a label is used to mark it in the non-repeating text.

8. A video text sensitive information inspection system based on a stay frame, characterized in that: include: A feature vector generation module is used to continuously extract a frame sequence from the acquired video data, and detect and describe key points of each frame in the frame sequence to generate a feature vector for representing the visual features of the frame; The stay frame extraction module is used to obtain matching feature point pairs of adjacent frames by comparing the similarity of feature vectors of feature points between adjacent frames, and to identify and extract the stay frame based on the dynamic behavior between the matching feature point pairs of adjacent frames; The text recognition module uses the pre-trained Paddle OCR deep learning model to perform text recognition on the stop frame and obtain the text content embedded in the video; The duplicate elimination and matching module is used to eliminate duplicate text by calculating the similarity of the identified text content, and to match the processed non-duplicate text with single sensitive words, sensitive phrases or sensitive paragraphs to detect and mark the existing sensitive information.

9. A video text sensitive information inspection device based on a stay frame, characterized in that: include: at least one database; and a memory in communication with at least one database; The memory stores instructions that can be executed by at least one database, and the instructions are executed by at least one database so that the at least one database can execute the video text sensitive information inspection method based on the stay frame as described in any one of claims 1-7.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by the processor, the method for checking sensitive information in video text based on the stay frame as described in any one of claims 1 to 7 is implemented.

Images