Electronic screen security verification system based on dynamic screen refresh rate and real-time watermark

Abstract

The present invention relates to the field of electronic screen content security detection technology, and in particular to an electronic screen security verification system based on dynamic screen refresh rate and real-time watermark, wherein a plurality of network electronic screens are connected to a cloud server via a network, and a security monitoring terminal communicates with the cloud server via a network connection; the security monitoring terminal transmits the acquired network electronic screen device information and the device code of the security monitoring terminal to the cloud server; the cloud server embeds a dynamic watermark into the video to be played on the network electronic screen; the security monitoring terminal requests the cloud server to obtain the watermark timing coding sequence of the video to be played, and the network electronic screen terminal downloads the video to be played and plays it; the security monitoring terminal performs abnormal detection, judgment and response on the video played on the network electronic screen. The present invention has strong anti-attack capability and low latency, and even if an attacker obtains the video content, he cannot reproduce the specific dynamic watermark timing sequence.

Description

Technical Field

[0001] The present invention relates to the field of network electronic screen content security information technology, and in particular to an electronic screen security verification system based on dynamic screen refresh rate and real-time watermark. Background Art

[0002] With the advancement of smart city development, electronic screens have become deeply integrated into public transportation, commercial centers, public venues, and other scenarios, becoming a core vehicle for information dissemination and interaction. However, the openness and high visibility of electronic screens in public places present serious content security challenges. Malicious attackers can tamper with displayed content through signal hijacking and protocol vulnerabilities, inserting illegal advertisements, false information, or even confidential content. This not only disrupts public order but also potentially triggers a crisis of social trust. Traditional protection methods based on manual inspections or fixed rule-based filtering are unable to cope with these highly real-time and covert digital attacks.

[0003] The Chinese patent number CN114650447A discloses a method for determining the degree of abnormality in video content using a deep learning-based neural network model. While the method has a certain level of semantic understanding capabilities by training the neural network model to identify abnormal content, it has significant drawbacks: (1) excessive reliance on computing resources. Model training requires massive amounts of labeled data and high-performance computing power. The inference phase requires feature extraction and classification for each frame, resulting in a surge in system load and difficulty meeting the needs of multi-terminal parallel monitoring. (2) Real-time bottlenecks. Complex model inference has high latency, making it impossible to complete tampering determination and blocking within a millisecond time window, posing a risk of response lag.

[0004] The Chinese patent number CN117596407A discloses a video stream tampering detection system and method based on layered feature code embedding. This method achieves anti-tampering by embedding digital feature codes into the video stream. However, it has the following limitations: (1) Dependence on the encoding format. Feature code embedding requires deep coupling with the video encoding and decoding algorithm (such as H.264 / HEVC). When facing heterogeneous encoding formats or private protocols, the adaptation cost increases sharply, and the system scalability is limited. (2) Weak anti-attack capability. Static feature codes are easily reverse analyzed or directly stripped. Attackers can bypass detection by means of re-encoding, area coverage, etc., and the protection effect is easily ineffective.

[0005] Therefore, there is an urgent need for a security protection technology that can be seamlessly integrated into the playback process, has real-time response capabilities, and is difficult to counterfeit and destroy. Summary of the Invention

[0006] The purpose of the present invention is to provide an electronic screen security verification system based on dynamic screen refresh rate and real-time watermark, by constructing a more secure and low-latency detection method for detecting tampering of electronic screen playback content, to monitor the legality of video content played on the electronic screen, automatically identify and block the playback of abnormal content, and prevent the illegal dissemination of bad content.

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: an electronic screen security verification system based on dynamic screen refresh rate and real-time watermark, wherein multiple network electronic screens are connected to a cloud server via a network, and a security monitoring terminal communicates with the cloud server via a network connection;

[0008] The security monitoring terminal transmits the acquired network electronic screen device information and the device code of the security monitoring terminal to the cloud server via TLS encrypted communication;

[0009] The cloud server embeds a dynamic watermark into the video to be played on the network electronic screen, and obtains the video embedded with the watermark after watermark modulation;

[0010] The security monitoring terminal requests the cloud server to obtain the watermark timing coding sequence of the video to be played through TLS encrypted communication, and the network electronic screen terminal downloads the video to be played and plays it;

[0011] The camera module of the security monitoring terminal captures the screen refresh timing;

[0012] Detect, judge and respond to video playback anomalies.

[0013] In some embodiments, the network electronic screen device information includes the network electronic screen refresh frequency and the network electronic screen device hardware serial number.

[0014] In some embodiments, embedding a dynamic watermark includes:

[0015] Generate a watermark basic sequence, including the serial number of the network electronic screen device and the number of the current video;

[0016] Generate a watermark temporal coding sequence based on a watermark basic sequence;

[0017] Divide each video frame into multiple refresh cycles;

[0018] In the embedded area The video pixel brightness at the moment is watermark modulated to obtain a video embedded with the watermark.

[0019] In some embodiments, the calculation expression for watermark modulation of the brightness of the video pixel at time t in the embedding area is:

[0020] ;

[0021] in,

[0022] ;

[0023] Where, is the pixel brightness value after modulation; For the original video Moment Image The original brightness value at ; is the modulation amplitude; It is an element in the watermark temporal coding sequence, and its function is to determine the positive and negative modulation of the pixel. , is the watermark temporal coding sequence; is an empirical constant; is the average brightness of the local area; are the global maximum and minimum values respectively.

[0024] In some embodiments, the security monitoring terminal requests the cloud server to obtain the watermark timing coding sequence of the video to be played through TLS encrypted communication, including:

[0025] Collect the screen image and calculate the average brightness of the watermark embedded area as the timing signal for screen refresh;

[0026] Extract the periodic signal of the time series signal and perform Fourier transform;

[0027] Decode the watermark sequence and perform coherent modulation on the periodic signal;

[0028] Calculate and output the bit error rate.

[0029] In some embodiments, the computational expression for coherently adjusting the periodic signal is:

[0030]

[0031] in

[0032]

[0033] Where, is the cycle length, ; is the screen refresh rate; is the index of the refresh cycle, ; is a symbolic function.

[0034] In some embodiments, the security monitoring terminal detects and responds to abnormalities in the video played on the network electronic screen, including:

[0035] Timing deviation abnormality judgment: judge whether the current playback is abnormal based on the degree of refresh rate deviation;

[0036] Timing coding error rate determination: Based on the timing coding error rate tolerance, determine whether the current playback is abnormal;

[0037] Set up a playback anomaly detection response mechanism: when multiple consecutive video frames have anomalies or the cumulative number of anomalies within a fixed time exceeds the threshold, it is determined that the current playback content has been tampered with and the network electronic screen is shut down.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. This invention is highly resistant to attacks: The screen refresh rate is dynamic and non-fixed. Static screenshots or regular screen recordings cannot capture the screen's physical refresh timing (such as the 78Hz periodic brightness fluctuations). The watermark sequence is bound to the dynamic screen refresh rate, device serial number, and video ID. The dynamic watermark is unique, and even if an attacker obtains the video content, they cannot reproduce the specific dynamic watermark timing sequence.

[0040] 2. The present invention has low delay detection and real-time performance: a single check cycle is For example, at a 60Hz screen refresh rate, the single verification time is 16.7ms, and a test is performed every 16.7ms. The detection has low latency and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the overall structure of the present invention;

[0042] Figure 2 A schematic diagram of the safety verification process of the present invention;

[0043] Figure 3 Schematic diagram of the structure of a security monitoring terminal according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] See also Figure 1-Figure 3The electronic screen security verification system based on dynamic screen refresh rate and real-time watermarking includes a cloud server, a network electronic screen, and a security monitoring terminal. The security monitoring terminal is the core execution unit of the real-time watermark embedding and verification solution, responsible for dynamically controlling the screen refresh rate, capturing screen signals, detecting anomalies, and triggering blocking mechanisms. It includes:

[0046] Camera module: The core hardware is the camera, which supports frame rates above 240Hz and can use the IMX477 camera.

[0047] Electronic screen dynamic frequency setting module: uses a control chip that supports the HDMI 2.1 VRR protocol, such as the STM32H743 microcontroller, through which the refresh frequency of the network electronic screen is dynamically set (such as 78Hz, 90Hz, or any other value) according to the watermark embedding requirements.

[0048] Anomaly Detection Module: This module processes the brightness signal input from the camera module in real time, performs periodic signal analysis, watermark sequence decoding, and anomaly determination, and synchronizes the data to the cloud server. The processor supports multi-threaded parallel processing and can use the following processor models: T507 4×A53 @1.8GHz + G52 MP2.

[0049] Electronic screen power blocking module: uses solid-state relay to cut off the power supply to the screen after detecting an abnormality. The relay model can be G5V-2-H1 DC5V solid-state relay.

[0050] The four modules of the safety monitoring terminal work together through the hardware interface, data bus and control signal line to form a closed-loop control system.

[0051] Multiple network electronic screens are connected to a cloud server via a network to obtain video content to be played. The network electronic screens are connected to a security monitoring terminal and are displays that support variable refresh rate VRR protocols (such as FreeSync and G-Sync).

[0052] The security monitoring terminal communicates with the cloud server via a network connection.

[0053] The safety verification system implements verification by the following method:

[0054] S1. The security monitoring terminal obtains the information of the electronic screen device connected to it, and randomly sets the refresh frequency of the network electronic screen within a certain range (for example, 60-120Hz), and stores the hardware serial number and refresh frequency of the network electronic screen device. The above information and security monitoring terminal device number and other information are transmitted to the cloud server through TLS encrypted communication.

[0055] S2. The cloud server embeds a dynamic watermark into the video to be played on the network electronic screen.

[0056] The original video frame sequence is recorded as , network electronic screen display screen refresh rate , original video frame rate The specific steps are as follows:

[0057] (1) Generate watermark base sequence.

[0058] Obtain the unique serial number SID of the network electronic screen device, obtain the number id of the current video, and convert the serial number SID and the number id of the current video into a binary sequence through the encrypted hash function SHA256(SID,id) to obtain the dynamic watermark basic sequence .

[0059] (2) Generate watermark timing coding sequence based on watermark basic sequence B Where, The new element whose value is 1 or -1 is obtained by converting the binary element of the watermark basic sequence; is the Kth element in the base sequence generated in step (1).

[0060] (3) Divide each video frame into multiple refresh cycles ,For example, , then each video frame is divided into refresh cycles.

[0061] (4) Watermark modulation, in a specific embedding area , such as the four corners of the screen, The video pixel brightness at the moment is watermarked:

[0062] ;

[0063] in,

[0064] ;

[0065] Where, It is an element in the watermark temporal coding sequence, and its function is to determine the positive and negative modulation of the pixel. , is the watermark temporal coding sequence.

[0066] For each watermark modulation, the timing coding elements are obtained from the watermark timing coding sequence in sequence order as the basis of the watermark information to refresh the number of cycles. For example, in the first refresh cycle of the first frame video (start and end time are 0, ), pixel brightness modulation watermark information , in the second refresh cycle (start and end time are 、 ) Pixel brightness modulation watermark information , and so on, when the watermark timing coding sequence is obtained The last element is the watermark temporal encoding sequence After watermark modulation, the video embedded with watermark is obtained, and the sequence is recorded as .

[0067] S3. The security monitoring terminal requests the cloud server to obtain the watermark timing coding sequence of the video to be played through TLS encrypted communication. , the network electronic screen terminal downloads the video to be played and plays it;

[0068] S4. The camera module of the security monitoring terminal captures the refresh timing of the network electronic screen, including the following steps:

[0069] S41, the camera module of the security monitoring terminal collects the screen image, the frame rate of the camera module ; Calculate the watermark embedding area in the screen capture image The average brightness of the screen is used as the timing signal for screen refresh: .

[0070] Collected for the camera module The instantaneous brightness value at the moment, To find the average function; The image of the original video at time t The original brightness value at Represents the pixel position coordinates of the image.

[0071] S42, extract the periodic signal of the timing signal, Perform Fourier transform to extract the fundamental frequency Element: , where is the Fourier transform, peak frequency ,in is the Fourier transform peak, and the frequency at the corresponding position is the peak frequency .

[0072] S43. Decode the watermark sequence and perform coherent adjustment on the periodic signal:

[0073] ;

[0074] in,

[0075] ;

[0076] Where, is the decoded watermark timing sequence; is the cycle length, ; is the screen refresh rate; is the index of the refresh cycle, ; is a symbolic function.

[0077] S44. Calculate and output the bit error rate.

[0078] Calculate the actual decoding sequence Watermark timing Bit error rate of the coded sequence , It is an indicator function, which outputs 1 when the condition is met, and 0 otherwise.

[0079] S5. Video playback anomaly detection, judgment, and response. This includes the following steps:

[0080] Timing deviation abnormality judgment: Determine whether the current playback is abnormal based on the degree of refresh rate deviation. When the detected refresh rate deviation exceeds the threshold, hour( is the threshold value, which can be 2Hz), and determines that the current playback is abnormal.

[0081] Timing coding error rate judgment: Based on the timing coding error rate tolerance, judge whether the current playback is abnormal. When the timing coding error rate exceeds the tolerance, that is, hour( is the threshold value, which can be 0.05), and determines that the current playback is abnormal.

[0082] Set up a playback anomaly detection response mechanism: When anomalies occur in multiple consecutive frames (for example, 5 frames) of video or the number of accumulated anomalies in a short period of time is too high (for example, more than 10 anomalies within 3 seconds), it is determined that the current playback content has been tampered with and is illegally played. The illegal playback report information is reported to the cloud server, and the power supply to the screen is disconnected through the power relay to shut down the network electronic screen, blocking the spread of illegal content.

[0083] In a specific embodiment, the refresh rate of the network electronic screen was set to 60Hz and 120Hz, and full HD and ultra-HD videos were used as the original video to test the watermark embedding efficiency of this embodiment. The real-time verification test of this embodiment was tested in the use scenarios of normal video playback, fast motion pictures, and strong ambient light interference. The anti-attack capability was tested and compared using screenshot replacement attack, same-screen replay attack, and simulated watermark attack as attack types. The specific embodiment scheme and results are as follows:

[0084] 1. Hardware equipment parameters are shown in the following table:

[0085] Table 1 Test hardware equipment parameters

[0086]

[0087] 2. Watermark embedding efficiency test.

[0088] For the same video to be played, the watermark capacity depends on the refresh frequency of the electronic screen. , the size of the watermark capacity determines the total duration of video watermark embedding. The watermark capacity size is: ,in, is the number of refresh cycles, The original video frame rate and total length are respectively: the video frame rate is 30, the video is 10 seconds long, the total number of video frames is 300, and the refresh rate is On the network electronic screen, the watermark capacity is 2*30*10=600 bits.

[0089] According to the above method, the device serial number and the video number to be played are converted into a binary sequence by the encrypted hash function to obtain the dynamic watermark basic sequence. , the sequence length is 256, and then converted into a watermark time sequence coding sequence .

[0090] An element in the watermark sequence is allocated to each refresh cycle of the video frame. For the first frame, -1 is allocated to cycle 1, 1 is allocated to cycle 2, and so on. This continues until a watermark sequence is allocated to each cycle of the 300 video frames (if there are not enough elements in the watermark sequence, they are reused from the beginning of the sequence).

[0091] Watermarks are embedded in the four corners of the screen, that is, the brightness of the video pixels is modulated according to the assigned watermark sequence elements. After watermark modulation, the video with embedded watermarks is obtained. The test results are shown in the following table:

[0092] Table 2 Watermark embedding efficiency test results

[0093]

[0094] 3. Real-time performance verification.

[0095] In the real-time performance verification, the original video frame rate is 30. When the network electronic screen terminal downloads the video to be played and plays it, it replaces it with another video with the same frame rate. The camera module of the security monitoring terminal captures the screen refresh timing and performs video playback anomaly detection. The condition is set to be when 5 consecutive frames of video are abnormal. When the condition is met, it is judged as abnormal playback. The test results are shown in the following table:

[0096] Table 3 Real-time performance test results

[0097]

[0098] 4. Anti-attack capability test

[0099] In the anti-attack capability test, the traditional DCT frequency domain signature embedding (H.264 encoding, medium embedding complexity) was selected as the comparison scheme. The screenshot replacement attack, the same screen replay attack, and the simulated watermark attack were used as the attack types for anti-attack capability testing and comparison. The results are shown in the following table:

[0100] Table 4. Test results of the anti-attack capability of the present invention and DCT frequency domain signature code.

[0101]

[0102] The present invention is immune to screenshot replacement attacks (depending on the physical refresh characteristics of the screen, screenshot attacks cannot capture the refresh timing of the screen). For the same model of multiple screens with wrong screen playback, since the watermark information is bound to the unique serial number of the device, and the same model of screens support the variable refresh rate VRR protocol, the refresh frequency of the same model of electronic screens in the present invention is also different, so the wrong screen playback of the same model of electronic screens can also be detected in time. For simulated watermarks, the watermark information of the present invention is obtained by hashing the variable dynamic screen refresh rate, the device serial number, and the video number. For the same video to be played, the watermark information of any two devices and the watermark information of the same device at different times are also different. The possibility of simulating the same watermark by cracking the watermark is almost zero, which effectively improves the anti-attack ability of the present invention.

[0103] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. An electronic screen security verification system based on dynamic screen refresh rate and real-time watermark, characterized by: Multiple network electronic screens are connected to the cloud server via the network, and the security monitoring terminal communicates with the cloud server via the network connection; The security monitoring terminal transmits the acquired network electronic screen device information and the device code of the security monitoring terminal to the cloud server; The cloud server embeds a dynamic watermark into the video to be played on the network electronic screen, including: generating a watermark basic sequence, including the network electronic screen device serial number and the current video number; generating a watermark timing coding sequence based on the watermark basic sequence; dividing each video frame into multiple refresh cycles; In the embedded area The brightness of the video pixels at the moment is watermarked and modulated to obtain a video embedded with the watermark; The security monitoring terminal requests the cloud server to obtain the watermark timing code sequence of the video to be played. The network electronic screen terminal downloads the video to be played and plays it. The camera module of the security monitoring terminal captures the refresh timing of the network electronic screen, including: capturing the screen image, calculating the average brightness of the watermark embedded area as the timing signal for screen refresh; extracting the periodic signal of the timing signal and performing Fourier transform; decoding the watermark sequence and performing coherent adjustment on the periodic signal; and calculating and outputting the bit error rate. The security monitoring terminal detects and judges abnormalities in the video played on the network electronic screen and responds to them, including: timing offset abnormality judgment: judging whether the current playback is abnormal based on the degree of refresh rate deviation; timing coding error rate judgment: judging whether the current playback is abnormal based on the timing coding error rate tolerance; setting a playback abnormality detection and response mechanism: when abnormalities occur in multiple consecutive frames of video or the cumulative number of abnormalities within a fixed time exceeds the threshold, it is judged that the current playback content has been tampered with and the network electronic screen is shut down.

2. The electronic screen security verification system based on dynamic screen refresh rate and real-time watermark according to claim 1 is characterized in that: The network electronic screen device information includes the network electronic screen refresh frequency and the network electronic screen device hardware serial number.

3. The electronic screen security verification system based on dynamic screen refresh rate and real-time watermark according to claim 1 is characterized in that: The calculation expression for watermark modulation of the video pixel brightness at time t in the embedding area is: ； in, ； Where, is the pixel brightness value after modulation; For the original video Moment Image The original brightness value at ; is the modulation amplitude; is the element in the watermark temporal coding sequence, , is the watermark temporal coding sequence; is an empirical constant; is the average brightness of the local area; are the global maximum and minimum values respectively.

4. The electronic screen security verification system based on dynamic screen refresh rate and real-time watermark according to claim 1 is characterized in that: The calculation expression for coherent adjustment of periodic signals is: in, Where, is the decoded watermark timing sequence; is the cycle length, ; is the screen refresh rate; is the index of the refresh cycle, ; is a symbolic function.

Images