Cross-platform video content identification method based on fingerprint multiplication and expansion

Abstract

The invention relates to a cross-platform video content identification method based on fingerprint multiplication and expansion, and belongs to the technical field of video content identification. The method comprises the following steps: acquiring a video sample file from a large platform; an MP4 video sample file is restored into a YUV original video file, and the limitation of a compression format is broken through; for a plurality of different video platforms, residual coding parameters of videos are extracted, key frames are positioned through an FFmpeg tool, segmentation parameters of the videos are counted, and after the maximum value and the minimum value of the coding parameters and the segmentation parameters are determined, possible values of all the parameters are permutated and combined to generate a parameter library. After a video sample file of one platform is restored into a YUV file, recoding and re-segmenting the video according to all parameter combinations of the parameter library, so that video fingerprints of a plurality of platforms are multiplied and expanded from the video of one platform, and a cross-platform fingerprint library is obtained; and reducing a PDU sequence extracted from the traffic into an ADU sequence as a length sequence feature by using a Transform model, and matching the length sequence feature with fingerprints in a cross-platform fingerprint database to realize accurate identification of cross-platform video contents. According to the method, video identification difficulty caused by coding, fragmentation and protocol differences among platforms can be effectively solved, the accuracy, robustness and cross-platform adaptability of video content identification are improved, and the method is suitable for scenes of video content supervision, copyright protection, flow analysis and the like.

Description

Technical Field

[0001] This invention relates to the field of video content recognition technology, and in particular to a cross-platform video content recognition method based on fingerprint multiplication expansion. Background Technology

[0002] With the rapid evolution of mobile internet technology, video content has become an important carrier of information dissemination and digital consumption. High-definition, multi-screen interaction, strong interactivity, social sharing, and low-latency transmission are becoming core demands of users for video experience. Currently, online video traffic dominates overall internet traffic. As of December 2024, my country had 1.04 billion online video users, with a user penetration rate of 93.8%.

[0003] The surge in video traffic has brought significant revenue to content platforms and network operators, while also placing higher demands on their video service management and optimization capabilities. In video service quality assessment systems, video fingerprints, as key features for identifying video content, directly impact the refined operation of Quality of Service (QoS) with their accuracy and scalability. With the improvement of privacy and copyright protection technologies across platforms, most platforms are unable to directly obtain MPD files during video access, resulting in the inability to directly read video segment length sequence fingerprints.

[0004] Faced with the rapid iteration of video applications, frequent updates to transmission protocols, and deep encryption of content fragmentation mechanisms, the robustness of traditional fingerprint features declines sharply, causing recognition models to quickly fail. Particularly in cross-platform scenarios, the significant differences in encoding parameters, fragmentation strategies, and transmission protocols employed by different video platforms make it difficult to directly transfer fingerprint features from one platform to others, further exacerbating the problems of insufficient generalization ability and increased maintenance costs of recognition models. For small and medium-sized platforms with scarce samples and diverse protocols, building and maintaining a dedicated fingerprint database presents enormous challenges, making network monitoring difficult.

[0005] Generating highly distinctive and cross-platform adaptable video traffic fingerprints in encrypted environments to address the challenges of cross-platform video propagation has become a core technical challenge for expanding video traffic classification scenarios and solving current network operation pain points. By constructing a cross-platform compatible fingerprint generation and reconstruction mechanism, we can not only enhance regulatory coverage of small and medium-sized video platforms and improve overall network traffic identification and control capabilities, but also provide unified technical support for traffic governance in multi-protocol and multi-architecture environments. This enables accurate and efficient video service perception and security management in complex and ever-changing network environments. Summary of the Invention

[0006] The purpose of this application is to provide a cross-platform video content recognition method based on fingerprint multiplication expansion, so as to achieve cross-platform video content recognition.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows: A cross-platform video content recognition method based on fingerprint multiplication and expansion, comprising the following steps:

[0008] Step 1: Obtain samples in batches from a large platform as the original files for fingerprint multiplication and expansion;

[0009] Step 2 involves reconstructing the MP4 video sample file from Step 1 into a raw YUV file. This overcomes the limitations of compression formats, enabling in-depth analysis and flexible processing of video data. It also facilitates research on video encoding conversion and adaptability across different platforms.

[0010] Step 3: Based on Step 2, for multiple video platforms, statistically analyze the platform's encoding and segmentation parameters, and arrange and combine the possible values ​​of each parameter to obtain a parameter library for all parameter combinations. The target video YUV file is first re-encoded, retaining only a portion of the original frames as keyframes. Other frames are supplemented with a certain number of reference frames according to parameter settings. The frame count and frame quality are set according to parameters such as bitrate, resolution, and frame rate. During the re-segmentation stage, using this YUV file and all possible encoding and segmentation parameters in the parameter library, the video is segmented into corresponding segment sequences to extract fingerprint sequences of video length. Each encoding and segmentation parameter corresponds to one sequence, and each video yields several fingerprint sequences under different encoding and segmentation parameter conditions. Fingerprint sequences under different encoding and segmentation parameters are extracted from all videos in the video sample library from Step 1, resulting in a multiplied and expanded fingerprint library that can cover multiple video platforms.

[0011] Step 4: Accurately reconstruct the length sequence features of the video ADU from the video traffic, and match the cross-platform video fingerprint based on the fingerprint database obtained in Step 3.

[0012] Step 1 includes:

[0013] This method analyzes and downloads video segments from large platforms, using the open-source tool yt-dlp to automatically adapt to different segmentation methods. By parsing playlists returned by the platform's webpage or API, the open-source tool Aria2 is used for video segment downloading, significantly improving download speed.

[0014] Step 2 utilizes the FFmpeg library to implement the process of restoring MP4 files to YUV files.

[0015] Before officially starting the YUV file restoration, it is necessary to check the encoding information of the MP4 file to confirm the resolution, frame rate, encoding format, and pixel format of the original video, providing a basis for subsequent restoration parameters. After determining the encoding information, specify the MP4 file to be restored, and use FFmpeg to automatically detect the source file format and separate the audio and video streams; at the same time, specify the video encoder as the original video, bypassing the encoder and directly outputting the decoded original pixel data; then, select an appropriate pixel format based on the encoding information determined in the previous step to ensure output compatibility. Frame rate synchronization needs to be disabled during output; otherwise, FFmpeg may discard or duplicate some frames due to timestamp issues, resulting in the input and output frame counts not being exactly the same. Through the above operations, the YUV original file data stored frame by frame can be output.

[0016] If you need to extract segments from a long video to output a file of a specific time or size, you can use FFmpeg's frame and t parameters to perform secondary operations on the already output YUV file, or determine the corresponding frame number and time when outputting the YUV file, and simultaneously output and extract the YUV file.

[0017] Step 3 requires extracting encoding and segmentation parameters that can be used on small and medium-sized platforms, and then using these parameters to extract cross-platform video fingerprints.

[0018] The system searches for residual video encoding methods in the platform's video files, extracts available encoding parameters, and statistically analyzes parameters that differ across platforms. It also calculates the maximum and minimum values ​​of these parameters to determine their value range. These statistically derived encoding parameters are then combined with the following segmentation parameters to form a parameter library.

[0019] The video segmentation method extracts keyframe intervals. FFmpeg is used to extract keyframe positions from the platform's video. Each frame's `pict_type` field is set to "I" to indicate a keyframe, used to determine if the video uses constant interval keyframes and extract the corresponding keyframe intervals. During the segmentation process, segmentation parameters are generally not directly stored in the video file; keyframe intervals are extracted to infer the platform's segment duration. For the streaming media protocol used by the platform and the file format of the segmented fragments, browser plugins and browser developer tools are used to search for streaming media-related information in network interactions for judgment. By statistically analyzing all possible segment durations and streaming media protocols for each video platform, all possible segmentation parameters are obtained. All encoding parameters from claim 7 are combined with the possible values ​​of all segmentation parameters described here, with each parameter combination representing a possible parameter combination for a platform, forming a parameter library.

[0020] The restored YUV file is re-encoded and segmented using the FFmpeg tool according to the parameter library in claim 8. Each YUV generates a set of video slices under each parameter, and the video fingerprint corresponding to the set is extracted to construct a cross-platform video fingerprint library.

[0021] Step 4 includes:

[0022] Step 4-1: In the experimental environment, collect and decrypt video traffic, extract the Protocol Data Unit (PDU) of each video segment at the TLS layer, and combine the PDUs corresponding to all video segments of a video to form the PDU sequence of that video; and extract the actual length of the content-length field in the HTTP header of each video segment after decryption, which is the Application Data Unit (ADU), and combine the ADUs corresponding to all video segments of a video to form the ADU sequence of that video; the PDU sequence and ADU sequence of the same video constitute a sample;

[0023] Step 4-2: Collect video PDU sequences and ADU sequences from different platforms. It is recommended to have more than 1000 videos from each platform. Use a Transformer model, taking the PDU sequences as encoder input and the ADU sequences as decoder input, to train the model to reconstruct video ADU sequences from video PDU sequences. Because it uses data from multiple platforms for training, the model has the ability to reconstruct ADU sequences from PDU sequences across different platforms.

[0024] Compared to existing technologies, this invention has the following advantages: After downloading video samples from large platforms, this scheme restores the MP4 video files to YUV files, re-encodes the videos, and then segments them to obtain multiplied video fingerprints. This invention designs a fingerprint multiplication algorithm to address the fingerprint differences of the same video content on different platforms, and constructs a video fingerprint database. Other schemes generally collect video files from each platform independently to extract video fingerprints for each platform. This scheme uses only video files from one video platform, directly generating video fingerprints for multiple platforms by combining encoding and segmentation parameters and re-encoding and segmenting the video files. Each platform only needs to analyze its unified encoding and segmentation parameters instead of collecting videos individually, avoiding the need to repeatedly collect the same videos on different platforms, greatly improving the efficiency of cross-platform video fingerprint acquisition. Simultaneously, by combining all possible values ​​of the encoding and segmentation parameters, it is possible to combine encoding and segmentation forms for some unknown video platforms, and the generated video fingerprints may reveal the same video from an unknown platform. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the background art, the accompanying drawings used in the embodiments of the present invention or the background art will be described below.

[0026] Figure 1 This is a schematic diagram of a cross-platform video content recognition technology provided in an embodiment of the present invention.

[0027] Figure 2 This is a diagram showing the relationship between YUV, MP4, and ADU. Detailed Implementation

[0028] To enhance understanding of the present invention, the embodiments will be described in detail below with reference to the accompanying drawings.

[0029] Example:

[0030] In one embodiment of this application, a cross-platform video content recognition method is provided, the method comprising:

[0031] Obtain video samples from large platforms;

[0032] Reconstructing YUV raw files based on video segmentation;

[0033] To achieve video fingerprint multiplication for small and medium-sized platforms and build a fingerprint database;

[0034] Video content is identified by the length sequence features of the video ADU.

[0035] As can be seen from the above, compared with the prior art, the solution provided in this embodiment can restore video fingerprints across multiple video platforms. For videos in the video sample library, video fingerprints for different platforms can be directly generated through fingerprint multiplication, thereby reducing the workload of cross-platform video fingerprint extraction and further enabling the identification of videos from unknown platforms.

[0036] The following is a general overview of the video content recognition method provided in the embodiments of this application.

[0037] The surge in video traffic has brought significant revenue to content platforms and network operators, but it has also placed higher demands on their video service management and optimization capabilities. In video service quality assessment systems, video fingerprints, as a key feature for identifying video content, directly impact the refined operation of service quality due to their accuracy and scalability. With the improvement of privacy and copyright protection technologies across platforms, most platforms are unable to directly obtain MPD files during video access, resulting in the inability to directly read video segment length sequence fingerprints.

[0038] Faced with the rapid iteration of video applications, frequent updates to transmission protocols, and deep encryption of content fragmentation mechanisms, the robustness of traditional fingerprint features declines sharply, causing recognition models to quickly fail. Particularly in cross-platform scenarios, the significant differences in encoding parameters, fragmentation strategies, and transmission protocols employed by different video platforms make it difficult to directly transfer fingerprint features from one platform to others, further exacerbating the problems of insufficient generalization ability and increased maintenance costs of recognition models. For small and medium-sized platforms with scarce samples and diverse protocols, building and maintaining a dedicated fingerprint database presents enormous challenges, making network monitoring difficult.

[0039] Generating highly distinctive and cross-platform adaptable video traffic fingerprints in encrypted environments to address the ever-emerging new applications and protocols has become a core technical challenge for improving video traffic classification performance and accuracy, and solving current network operation pain points. In light of this, the inventors analyzed the protocol stacks and data segmentation methods used by various large, medium, and small platforms. By constructing a cross-platform compatible fingerprint generation and reconstruction mechanism, they efficiently reconstruct video content from different platforms, thereby achieving accurate and efficient video service perception and security management in complex and ever-changing network environments.

[0040] S101: Obtain video samples from large platforms.

[0041] A length sequence is a sequence of data related to the transmission of data extracted from traffic. Different protocol layers have different Protocol Data Units (PDUs). In the HTTPS scenario, the three types of PDUs correspond to three different length sequences: message length sequence, TLS segment length sequence, and Application Data Unit (ADU) length sequence.

[0042] For encrypted multimedia content, due to the relatively large amount of video, short video, and audio content, these types are usually transmitted in segments of fixed duration to reduce the network pressure of a single request-response cycle. TS, M4S, and MP4 are commonly used formats in this segmented transmission process. Images, on the other hand, are typically transmitted in one request-response cycle per image. Therefore, image transmission can be compared to the transmission of each segment in audio and video. When audio and video are segmented into fixed duration segments, the content of each segment differs from other multimedia content, and these differences in content volume are reflected in the length of the segment data. Segment data is usually encapsulated in the HTTP header and sent to the client after multiple message segments. The length sequence of the ADU (Application Not Recommended) is the length sequence of the segment data transmitted in this batch of request-response, reflecting the characteristics of the information volume. Therefore, using the ADU length sequence as a fingerprint can effectively distinguish different multimedia content.

[0043] Previously, large platforms could obtain the video ADU segment length fingerprint by directly acquiring the corresponding MPD file of the video. This file contained the length information of each segment of the target video, which was consistent with the ADU length information and could be directly used as a fingerprint. However, with the improvement of privacy and copyright protection technologies on various platforms, most platforms can no longer directly obtain the MPD file during video access, making it impossible to directly read the video ADU length sequence fingerprint.

[0044] In this context, the inventors conducted a more detailed study on video transmission on large platforms. For platforms that do not contain MPD files, the video encoding and segmentation parameters of the platform can be analyzed and extracted. A high-resolution video file can then be re-encoded and segmented according to these parameters to generate video segments for the platform, which can then be used to extract video fingerprints. Taking the large video platform bilibili as an example, the project first collected videos from the website in batches. After obtaining the bilibili video links in batches, the playlist files were parsed and video segments were downloaded. After performing operations such as segment merging and video transcoding, MP4 file samples containing residual segment information were obtained for subsequent fingerprint extraction.

[0045] First, video links were obtained in batches through the website's public API. Then, by searching popular keywords and crawling popular blogger video submissions from different sections, the latest and most representative Bilibili video links of different types were obtained in batches to ensure the timeliness, representativeness, and sufficient sample size of the sample.

[0046] After obtaining the video link, the video transmission segmentation method is parsed and the video segments are downloaded. For specific Bilibili videos, the project cannot determine the streaming media protocol and segmentation format used. Using the open-source tool yt-dlp, automatic adaptation to different transmission segmentation methods can be achieved, allowing for automated processing through a unified interface. For AES-128 encrypted segments of some Bilibili videos, the key is obtained by parsing the playlist and then decrypted. During the video download process, the playlist returned by the webpage or API (such as an m3u8 file under the HLS protocol) is parsed, and the open-source tool Aria2 is used to download video segments (such as TS files under the HLS protocol), significantly improving download speed.

[0047] After obtaining the video segments, the open-source FFmpeg tool is used to perform operations such as segment merging and transcoding, ultimately obtaining a complete mp4 file with residual segment information for subsequent processing.

[0048] S102: Video YUV raw file reconstruction technology for video segmentation.

[0049] Multimedia files, such as video files, transmitted over the internet differ from other files in that their original form differs from their transmission form. Original multimedia files typically exist in an unprocessed or simply packaged form, with complex and diverse internal data structures and encoding methods. They generally contain high-quality uncompressed video streams, rich audio tracks, and detailed image information. During transmission, to adapt to network bandwidth limitations and improve transmission efficiency, these multimedia files undergo a series of conversions and optimizations. The data is re-encoded and compressed, possibly using different encapsulation formats, resulting in a transmitted file different from the original. This difference allows multimedia files to be transmitted more efficiently and stably over the internet, but it also places demands on file decoding and re-encoding.

[0050] Video file samples obtained from large platforms are typically encoded and compressed using various methods, with MP4 being a common one. These encoding methods significantly reduce data size without affecting subjective visual quality by removing redundant information and utilizing inter-frame correlation techniques. Uncompressed files, on the other hand, store raw YUV data directly, containing all pixel information without any compression. YUV is a color encoding method that directly reflects the raw data from an image sensor, separating the luminance (Y) and chrominance (U / V) components. Y represents luminance, reflecting the brightness and darkness of the image; U and V represent chrominance, encoding the blue and red color differences respectively, which together determine the image's color. This separation characteristic makes it more suitable for subsequent compression processing. However, uncompressed YUV files are extremely large. For example, a common 1080p resolution, 30 frames per second video, if in YUV4:2:0 format and uncompressed, has a data size of about 6.22MB per second; a 2-hour 1080p@60fps video in YUV format can reach about 1.35TB, while the compressed MP4 file can be reduced to several GB or even smaller.

[0051] Therefore, in practical network environments, to ensure network bandwidth and improve video transmission efficiency, video transmission commonly adopts compression encoding formats such as MP4. This encoding process removes redundant information and details imperceptible to the human eye, achieving a high compression ratio. However, this compression method also has limitations. Compressed MP4 files have already undergone encoding processing, and their data structure and encoding parameters are relatively fixed, making direct decoding, recoding, and flexible segmentation impossible. This makes it difficult to meet the research needs of converting video from one platform to another encoding method to adapt to another platform. In contrast, YUV files offer greater flexibility and operability. Because they are uncompressed raw video data, various decoding, recoding, and segmentation operations can be easily performed, better meeting the research needs for converting video encoding methods between different platforms. By decoding MP4 files back to YUV files, the limitations of compression formats can be overcome, enabling in-depth analysis and flexible processing of video data, facilitating research on video encoding conversion and adaptability across different platforms.

[0052] To convert MP4 files to YUV files, the FFmpeg library can be used. FFmpeg is an open-source, cross-platform multimedia processing framework that supports audio and video recording, conversion, streaming, and analysis. It consists of multiple libraries (such as libavcodec and libavformat) and command-line tools (such as FFmpeg, ffprobe, and ffplay), and is widely used in video transcoding, live streaming, and format conversion. Since YUV has multiple formats, to ensure format matching after conversion, it's necessary to check the MP4 file's encoding information before starting the conversion. This information confirms the original video's resolution, frame rate, encoding format, and pixel format, providing parameters for subsequent conversion. After determining the encoding information, specify the MP4 file to be converted. FFmpeg automatically detects the source file format and separates the audio and video streams; simultaneously, specify the original video encoder, bypassing the encoder to directly output the decoded raw pixel data; then, select a suitable pixel format based on the encoding information determined in the previous step to ensure output compatibility. Frame rate synchronization needs to be disabled during output; otherwise, FFmpeg may discard or duplicate some frames due to timestamp issues, resulting in a discrepancy between the input and output frame counts. The above steps will output the raw YUV file data stored frame by frame. In formal research, it may sometimes be necessary to trim segments from long video segments to output files of a specific time or size. In this case, FFmpeg's `frame` and `t` parameters can be used to perform secondary operations on the already output YUV file, or the corresponding frame count and time can be determined during the actual YUV file output, allowing for simultaneous output and trimming.

[0053] S103: Video fingerprinting for small and medium-sized platforms, building a video fingerprint database.

[0054] After obtaining the original YUV file of a video from a playback format such as MP4, the video needs to be re-encoded and then segmented to generate video segments for video transmission, serving as a method for extracting video fingerprints. During video encoding, numerous parameters control the encoding speed, video quality, video format, frame settings, and other related aspects, thus affecting the video size. The total video size directly influences the size of the segmented video fragments, i.e., the video fingerprint. Similarly, video segmentation involves numerous parameters controlling the size, format, and alignment of the segmented fragments. If the segmentation points cannot be nearly identical to those in the platform's video, the obtained fingerprint will deviate from the platform's actual fingerprint and become completely unusable.

[0055] This invention aims to achieve a video size that is as close as possible to the platform's video file size and to ensure consistency in segmentation points. Experiments show that the core factors influencing the final video file size during encoding are the bitrate, resolution, frame rate, and frame arrangement. Other factors causing size deviations are generally within acceptable limits. Since video encoding parameters are extensive and many cannot be obtained from the platform or video file, only parameters highly correlated with bitrate, resolution, frame rate, and frame arrangement are considered. Resolution and frame rate generally have limited selectable values, and these parameters can be directly extracted from the video file, making them relatively easy to obtain. Bitrate and frame arrangement are dynamically adjusted within the video, changing with the video content, and their behavior varies across different positions within the same video. The project first searches for residual video encoding methods in the platform's video file, extracting the obtainable encoding parameters. Based on this, FFmpeg is used to extract keyframe positions from the platform video. Each frame's `pict_type` field is set to "I" to indicate a keyframe, determining whether the video uses constant-interval keyframes and extracting the corresponding keyframe intervals. During the slicing process, parameters are generally not retained in the video file and need to be inferred manually from the platform's slicing parameters. For the streaming media protocol and file format of the slices used by the platform, browser plugins and browser developer tools are needed to search for streaming media-related information in network interactions for judgment. For the reference duration of video segment slicing, video segments need to be obtained first through plugins or video download tools, and inferred from the duration of a large number of video segments. If the segment duration fluctuates significantly, possible slicing durations need to be tried and compared during the slicing process to determine the specific slice duration. The encoding and slicing parameters of multiple platforms are statistically analyzed, and after determining all possible parameter values, they are permuted and combined to obtain a parameter library. The target video YUV file is first re-encoded, retaining only a portion of the original frames as keyframes. Other frames are supplemented with a certain number of reference frames according to parameter settings, and encoded into a new mp4 file based on parameters such as bitrate, resolution, and frame rate from the parameter library. In the re-slicing stage, combinations of slice durations and streaming media protocols from the parameter library are used to slice the video into corresponding slice segments to extract fingerprint sequences.

[0056] In addition to directly extracting video slice fingerprints, the generated video segments can also be used to generate network transmission video fingerprints. By setting up a network transmission environment and simulating the video transmission process between the client and server, video slices are used as the actual network transmission data, and network traffic is collected to generate video transmission fingerprints. This allows for the simulation of various network protocol combinations during network transmission, generating network transmission video fingerprints for multiple scenarios, thus eliminating the need for video traffic ADU reconstruction.

[0057] S104: Accurate reconstruction of video ADU length sequence features.

[0058] Since there is a clear correspondence between TLS layer PDU sequences and ADU sequences in video traffic, and the difference is usually determined only by the length of HTTP header fields, there is a correspondence between PDU sequences and ADU sequences under the same HTTP protocol. The inventors attempted to use the sequence-to-sequence generation capability of the Transformer model to restore the PDU sequences extracted from encrypted traffic to ADU sequences. PDU sequences were obtained by collecting video samples from various platforms in an experimental environment, and corresponding ADU features were extracted through traffic decryption to obtain feature pairs, which were then used as the input and output to train the model. It is recommended that the number of fingerprint pairs in the training set be greater than 1000 for each platform to ensure high restoration accuracy, thereby guaranteeing that the generated video ADU fingerprint has a small error and ensuring accurate matching in the matching process. When processing video traffic from this platform subsequently, after restoring the traffic fingerprint, a linear regression model is used to directly restore the fingerprint to the corresponding video ADU fingerprint.

[0059] This method allows for the construction of a video fingerprint database. For videos from small to medium-sized platforms, the ADU fingerprint can be extracted and matched with the fingerprint database to accurately identify the video content.

[0060] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. A cross-platform video content recognition method based on fingerprint multiplication and expansion, characterized in that, Includes the following steps: Step 1: Obtain high-resolution video samples in batches from a large video platform to build a video sample library, which will serve as the original files for fingerprint multiplication and expansion; Step 2: Reconstruct the MP4 video sample file from Step 1 into a YUV original file, breaking through the limitations of compression format, enabling in-depth analysis and flexible processing of video data, and providing convenience for video encoding conversion and adaptability research across different platforms; Step 3: Based on Step 2, for multiple video platforms, statistically analyze the platform encoding parameters and segmentation parameters, arrange and combine the possible values ​​of each parameter to obtain the parameter library under all parameter combinations, and re-encode the target video YUV file first. Only a portion of the original frames are retained as keyframes, and a certain number of reference frames are added to the other frames according to the parameter settings. The number of frames and frame quality are set according to the bitrate, resolution, and frame rate parameters. In the re-segmentation stage, the YUV file and all possible encoding and segmentation parameters in the parameter library are used to segment the video into corresponding segment sequences to extract the fingerprint sequence of the video length. Each encoding and segmentation parameter corresponds to a sequence, and each video obtains several fingerprint sequences under different encoding and segmentation parameters. The fingerprint sequences under different encoding and segmentation parameters are extracted from all videos in the video sample library in step 1 to obtain the fingerprint library that has been multiplied and expanded. This fingerprint library can cover multiple video platforms. Step 4: Accurately reconstruct the length sequence features of the video ADU from the video traffic, and match the cross-platform video fingerprint based on the fingerprint database obtained in Step 3.

2. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 1, characterized in that, This tool analyzes and downloads video segments from large platforms, using the open-source yt-dlp tool to automatically adapt to different transmission segmentation methods. By parsing the playlist returned by the large platform's webpage or API, it uses the open-source Aria2 tool to download video segments, significantly improving download speed.

3. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 1, characterized in that, Step 2 uses the FFmpeg library to implement the process of restoring MP4 files to YUV files.

4. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 3, characterized in that, Before officially starting the YUV file restoration, it is necessary to check the encoding information of the MP4 file to confirm the resolution, frame rate, encoding format, and pixel format of the original video, providing a basis for subsequent restoration parameters. After determining the encoding information, specify the MP4 file to be restored, and use FFmpeg to automatically detect the source file format and separate the audio and video streams; at the same time, specify the video encoder as the original video, bypassing the encoder and directly outputting the decoded original pixel data; then, select an appropriate pixel format based on the encoding information determined in the previous step to ensure output compatibility. Frame rate synchronization needs to be disabled during output; otherwise, FFmpeg may discard or duplicate some frames due to timestamp issues, resulting in the input and output frame counts not being exactly the same. Through the above operations, the YUV original file data stored frame by frame can be output.

5. A cross-platform video content recognition method based on fingerprint multiplication and amplification as described in claim 3 or 4, characterized in that, If you need to extract segments from a long video to output a file of a specific time or size, you can use FFmpeg's frame and t parameters to perform secondary operations on the already output YUV file, or determine the corresponding frame number and time when outputting the YUV file, and simultaneously output and extract the YUV file.

6. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 5, characterized in that, Step 3 requires extracting and analyzing the encoding and segmentation parameters that can be used by each video platform; The system searches for residual video encoding methods in the video files of the platform, extracts the available encoding parameters, counts the parameters that differ across platforms, and calculates the maximum and minimum values ​​of these parameters to obtain their value range. The calculated encoding parameters are then combined with the calculated segmentation parameters to form a parameter library.

7. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 6, characterized in that, The video segmentation method extracts keyframe intervals. The ffmpeg tool is used to extract the keyframe positions from the platform's video. Each frame's `pict_type` field is set to "I" to indicate a keyframe, determining whether the video uses constant interval keyframes and extracting the corresponding keyframe interval. During the segmentation process, segmentation parameters are generally not directly stored in the video file; keyframe intervals need to be extracted to infer the platform's segment duration. For the streaming media protocol and file format of the segment segments used by the platform, browser plugins and browser developer tools are used to search for streaming media-related information in network interactions for judgment. By statistically analyzing all possible segment durations and streaming media protocols for each video platform, all possible segmentation parameters are obtained. The statistically analyzed encoding parameters are then combined with all possible values ​​of these segmentation parameters, with each parameter combination representing a possible parameter combination for a platform, forming a parameter library.

8. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 7, characterized in that, The FFmpeg tool is used to re-encode and segment the restored YUV files according to the parameter library. Each YUV generates a set of video slices under each parameter, and the video fingerprint corresponding to the set is extracted to build a cross-platform video fingerprint library.

9. A cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 8, characterized in that, Step 4 includes: Step 4-1: In the experimental environment, collect and decrypt video traffic, extract the Protocol Data Unit (PDU) of each video segment at the TLS layer, and combine the PDUs corresponding to all video segments of a video to form the PDU sequence of that video; and extract the actual length of the content-length field in the HTTP header of each video segment after decryption, which is the Application Data Unit (ADU), and combine the ADUs corresponding to all video segments of a video to form the ADU sequence of that video; the PDU sequence and ADU sequence of the same video constitute a sample; Step 4-2: Collect video PDU sequences and ADU sequences from different platforms. It is recommended that the number of videos from each platform be greater than 1000. Use the Transformer model, with the PDU sequence as the encoder input and the ADU sequence as the decoder input, to train the model to restore the video ADU sequence from the video PDU sequence. Since data from multiple platforms are used for training, the model has the ability to restore the ADU sequence from the PDU sequence across platforms.

10. The cross-platform video content recognition method based on fingerprint multiplication and expansion according to claim 1, characterized in that, For a video stream, the PDU sequence of the TLS layer is extracted, and the trained Transformer model is used to restore the PDU sequence to obtain the ADU sequence of the video. This sequence is then used to perform matching in the cross-platform fingerprint database generated in claim 8 to achieve cross-platform recognition of the video.

Images