A method for embedding and extracting a hidden watermark in a screen interface of a relay protection device

Abstract

This invention discloses a method for embedding and extracting invisible watermarks on the screen interface of a relay protection device in the fields of copyright protection and digital image watermarking. It aims to solve technical problems such as easy copying of interfaces and easy tampering of menu layouts. The method includes: acquiring the original screen interface image; extracting business information through image recognition, generating a verification digest, and converting it into a binary copyright watermark; standardizing and normalizing the same original screen interface image; extracting macroscopic, mesoscopic, and microscopic features using CNN and weighted fusion; performing first-level DWT decomposition on the normalized image to obtain subbands; inputting the low-frequency and high-frequency subbands into a CNN to predict an adaptive intensity matrix; binding the encoded watermark with the fused features, and embedding the horizontal and vertical high-frequency subbands in conjunction with the intensity matrix; reconstructing and inverse normalizing the image using inverse DWT, and outputting the watermarked image. This invention can achieve highly concealed watermark embedding and disturbance-resistant blind extraction, improving the reliability of copyright tracing and infringement evidence collection.

Description

Technical Field

[0001] This invention relates to a method for embedding and extracting invisible watermarks on the screen interface of a relay protection device, belonging to the fields of copyright protection of human-machine interfaces for power equipment, digital image watermarking and intelligent recognition technology. Background Technology

[0002] Relay protection devices are core equipment for the safe and stable operation of power systems. Their screen interfaces integrate unique menu layouts, text typography, and functional partitioning designs, representing the core intellectual property of the R&D team and possessing clear copyright ownership. Therefore, effective copyright protection measures are urgently needed. Currently, copyright protection for relay protection device screen interfaces mainly relies on design patents, lacking supporting technical protection measures, which is clearly insufficient.

[0003] Infringers typically copy interface designs by manually imitating them, redrawing the interface, adjusting menu order, modifying font line height, and deleting icons. Relying solely on appearance patents is insufficient for quick evidence collection and accurate infringement determination. Existing digital watermarking technologies largely rely on embedding image pixel information, which is not robust to disturbances such as structural alterations to the interface or adjustments to local elements. Once the interface is imitated or partially altered, the watermark easily becomes invalid, making copyright traceability impossible.

[0004] Although wavelet transform frequency domain watermarking and CNN feature extraction technologies exist, they are not optimized for the structured characteristics of relay protection device screens, which are characterized by "fixed menus, text-based content, and clearly defined functional areas." They cannot simultaneously address the requirements for watermark invisibility, anti-disturbance properties, and blind extraction, making them unsuitable for copyright protection scenarios and unable to effectively solve the problem of evidence collection after interface imitation. Therefore, a targeted method for embedding and extracting invisible watermarks is urgently needed to overcome these shortcomings. Summary of the Invention

[0005] The purpose of this invention is to provide a method for embedding and extracting invisible watermarks on the screen interface of a relay protection device, which can solve the problems that the screen interface of a relay protection device is easily copied and imitated, the menu layout is easily tampered with, traditional watermarks cannot resist interface redrawing and structural disturbances, and copyright is difficult to effectively prove.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution.

[0007] In a first aspect, the present invention provides a method for embedding an invisible watermark on the screen interface of a relay protection device, comprising: Get the original screen interface image; Business information is extracted from the original screen interface image using image recognition methods to generate a corresponding verification digest, and the verification digest is converted into a binary copyright watermark to be embedded. The same original screen interface image is then subjected to size standardization and normalization processing in sequence to obtain a normalized interface image; The binary copyright watermark to be embedded is combined with the encryption key for obfuscation encoding to obtain the encoded watermark; The normalized interface image is input into a CNN feature extraction network to extract macro-level business partition features, meso-level digital layout features, and micro-level stroke and line detail features, and then the fused features are obtained through weighted fusion. The normalized interface image is decomposed using a first-order discrete wavelet transform to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands. The low-frequency sub-band, horizontal high-frequency sub-band, and vertical high-frequency sub-band are input into the CNN adaptive embedding intensity prediction network, and the adaptive intensity matrix is ​​output. The encoded watermark and the fusion feature are bound by element-wise multiplication and combined with an adaptive intensity matrix to generate a watermark signal to be embedded. The watermark signal to be embedded is then embedded into the horizontal high-frequency sub-band and the vertical high-frequency sub-band to obtain the modified horizontal high-frequency sub-band and the vertical high-frequency sub-band. The modified horizontal high-frequency sub-band, vertical high-frequency sub-band, low-frequency sub-band, and diagonal high-frequency sub-band are reconstructed using inverse discrete wavelet transform to obtain the interface image with embedded watermark. The interface image with the embedded watermark is denormalized to output a relay protection device screen interface image containing an invisible watermark.

[0008] In conjunction with the first aspect, the expression for the encoded watermark is further as follows: ; in, This indicates the watermark after encoding; Represents binary copyright watermark information; Indicates a bitwise XOR operation; This represents the encryption key.

[0009] In conjunction with the first aspect, the macro-level business partitioning characteristics further include the overall functional partitioning ratio and the overall structural distribution; The meso-level digital typesetting features include text spacing, line height ratio, and the proportion of typesetting within a region; The microscopic stroke and line detail features include stroke details and high-frequency information of line edges.

[0010] In conjunction with the first aspect, the CNN feature extraction network is further trained using a multi-task loss function, which includes macro-feature loss, meso-feature loss and micro-feature loss. The expression for the multi-task loss function is: ; ; ; ; in, Indicates loss of macroscopic features; Indicates macroscopic layout characteristics; Indicates the macroscopic feature annotation value; Represents the macroscopic feature dimension; i represents the index of the i-th dimension of the macro / meso / micro feature vector; This represents the value of the i-th dimension of the macroscopic feature vector; This represents the value of the i-th dimension of the macroscopic feature annotation value vector; This indicates the loss of mesoscopic features; Indicates mid-level typesetting characteristics; Indicates the labeled values ​​of mesoscopic features; Indicates the mesoscopic feature dimension; This represents the value of the i-th dimension of the mesoscopic eigenvector; This represents the value of the i-th dimension of the mesoscopic feature annotation value vector; This indicates the loss of microscopic features; Indicates the dimension of microscopic features; This represents the value of the i-th dimension of the micro-feature annotation value vector; This represents the value of the i-th dimension of the micro-feature vector; Represents the multi-task loss function; This represents the mean square error.

[0011] In conjunction with the first aspect, further, obtaining the adaptive intensity matrix includes: Based on the edge intensity of the normalized interface image, a human visual masking model is constructed. Based on the aforementioned human visual masking model, visual sensitivity is differentiated for different functional display zones on the screen interface of the relay protection device, and the optimal watermark embedding strength for each zone is calculated. ; The low-frequency sub-band, horizontal high-frequency sub-band, and vertical high-frequency sub-band are input into a trained CNN adaptive embedding intensity prediction network, which outputs an adaptive intensity matrix. ; wherein, with the optimal embedding strength The mean squared error loss function is used to train the CNN adaptive embedding strength prediction network as training labels.

[0012] In conjunction with the first aspect, further, the optimal embedding strength The expression is: ; in, This represents the optimal embedding strength of the normalized image; Indicates the maximum embedding strength; k represents the adjustment coefficient; This represents the edge intensity at pixel (x, y) in the normalized image; This represents the exponential decay factor based on edge strength.

[0013] In conjunction with the first aspect, further, obtaining the modified horizontal high-frequency sub-band and vertical high-frequency sub-band includes: The encoded watermark is bound to the fusion feature by multiplying element by element to obtain the bound watermark signal. The dimensions of the bound watermark signal are adjusted to be the same as the dimensions of the horizontal high-frequency sub-band / vertical high-frequency sub-band to obtain the adapted bound feature map. The adapted binding feature map is multiplied element-wise with the adaptive intensity matrix to obtain the watermark signal to be embedded. The watermark signal to be embedded is superimposed pixel by pixel onto the horizontal high-frequency sub-band and the vertical high-frequency sub-band to obtain the modified horizontal high-frequency sub-band and the vertical high-frequency sub-band respectively.

[0014] In conjunction with the first aspect, the expression for the output of the relay protection device screen interface image containing the invisible watermark is further as follows: ; in, This represents the output screen interface image of the relay protection device, which includes an invisible watermark. This represents the normalized interface image obtained after embedding the watermark and undergoing inverse discrete wavelet transform. Represents the original screen interface image; This represents the maximum value of the original screen interface image; This represents the minimum value of the original screen interface image.

[0015] Secondly, a method for extracting invisible watermarks from the screen interface of a relay protection device includes: Acquire images of suspicious interfaces to be detected; The suspicious interface image is normalized and its size is standardized to obtain a standardized suspicious image; The standardized suspicious image is decomposed using a first-order discrete wavelet transform to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands corresponding to the embedding stage. The horizontal and vertical high-frequency subbands corresponding to the embedding stage are input into the CNN watermark robust extraction network, and the coded state is output to extract the watermark. In conjunction with the second aspect, it further includes: The watermark extracted from the encoded state is decoded using an encryption key to obtain the restored binary watermark information as the decoded watermark, and the corresponding verification digest is recovered from the binary watermark information. For the currently suspected interface image to be detected, key business information is extracted through image recognition, and the current verification digest is recalculated; Calculate the normalization coefficient and bit error rate of the decoded watermark and the original binary copyright watermark; If the normalization coefficient is not greater than a preset threshold, or the bit error rate is not less than a preset threshold, then the suspicious interface image is determined to be without a valid copyright watermark. If the normalization coefficient is greater than a preset threshold and the bit error rate is less than a preset threshold, then it is determined that a valid copyright watermark is obtained from the suspicious image and a verification digest is restored from the valid copyright watermark. The restored verification digest is compared with the current verification digest. If they match, the suspicious interface image is determined to originate from a protected design and its content has not been tampered with. If they do not match, the suspicious interface image is determined to originate from a protected design but has been illegally tampered with.

[0016] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: This invention addresses the shortcomings of existing relay protection device screen interfaces, such as susceptibility to copying and imitation, easy tampering with menu layouts, inability of traditional watermarks to withstand interface redrawing and structural disturbances, and difficulty in effectively proving copyright infringement. It achieves a watermark core by using a deep fusion architecture based on Discrete Wavelet Transform (DWT) and Convolutional Neural Network (CNN), combined with a service partition adaptive embedding strength mechanism. This architecture generates a verification digest from the fixed structure of the interface and configuration-related service information (including channel identifiers, setpoint configurations, pressure plate status, menu layouts, and area divisions), enabling highly concealed watermark embedding and disturbance-resistant blind extraction. Through service partition adaptive strength allocation, it ensures clear display of key areas such as setpoints and waveforms without affecting normal device operation. Simultaneously, it constructs a local closed-loop evidence collection system, enabling copyright tracing and tamper identification without relying on external systems, effectively solving the problem of evidence collection after interface counterfeiting and tampering. Compared with traditional solutions, this invention combines practicality, robustness, and security. It not only overcomes the limitations of traditional watermarks in resisting disturbances, but also fills the gap in copyright protection for power industrial control interfaces, significantly improving the reliability of copyright confirmation and evidence collection. It has outstanding substantial progress and commercial application value. Attached Figure Description

[0017] Figure 1 The diagram shows a flowchart of the invisible watermark embedding method provided in an embodiment of the present invention. Figure 2 The diagram shows a flowchart of the invisible watermark extraction method provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other. Example 1

[0019] This embodiment describes in detail the complete implementation process of a method for embedding and extracting invisible watermarks on the screen interface of a relay protection device. The whole is based on a fusion architecture of discrete wavelet transform (DWT) and convolutional neural network (CNN), including three parts: watermark embedding, watermark extraction and network training.

[0020] 1. Image and watermark preprocessing: Acquire the original screen interface image of the relay protection device; Static, fixed key business information such as set values, pressure plate status, channel identifiers, menu layout, area division, and waveform channel information in the interface is extracted using image recognition methods. Dynamic content such as real-time changing sampled values ​​and instantaneous monitoring values ​​is ignored. Based on the above fixed information, a corresponding verification summary is generated as the binary copyright watermark to be embedded.

[0021] The same original screen interface image is then subjected to size standardization and pixel normalization processing in sequence, so that the processed image values ​​are uniform between 0 and 1, which facilitates subsequent wavelet decomposition and network operation. At the same time, a binary copyright watermark to be embedded is prepared, and it is obfuscated by using a bitwise XOR method with an encryption key (one or more combinations of device number, on-site timestamp and fixed version number) to improve the security of the watermark and prevent it from being forged or cracked.

[0022] 2. Perform multi-level feature extraction The preprocessed interface image is input into a CNN feature extraction network, which uses a three-layer convolution and two-layer pooling structure to extract macroscopic business partition features, mesoscopic digital layout features, and microscopic stroke and line detail features of the image from shallow to deep.

[0023] The first convolutional layer extracts macroscopic business partition features to characterize the overall functional partitions and structural distribution of the interface; the first pooling layer and the second convolutional layer extract mesoscopic digital layout features, covering information such as menu spacing and line height ratio; the second pooling layer and the third convolutional layer extract microscopic detail features, including text strokes, lines and waveform details.

[0024] Finally, the three features are weighted and fused according to preset weights to obtain the fused features used for watermark binding.

[0025] 3. Discrete Wavelet Transform Decomposition A first-order two-dimensional wavelet transform is performed on the standardized and normalized interface image to decompose it into four sub-matrices: low-frequency sub-band (LL), horizontal high-frequency sub-band (LH), vertical high-frequency sub-band (HL), and diagonal high-frequency sub-band (HH). LL represents the overall structure of the interface, while LH and HL represent the horizontal and vertical edge information. In this invention, watermarks are embedded only in LH and HL to ensure that the interface display is not affected.

[0026] 4. Adaptive Embedding Strength Prediction The three sub-bands LL, LH and HL obtained by wavelet decomposition are input into the adaptive embedding intensity prediction network. This network is a lightweight structure with two convolutional layers and one output layer, and the output is an adaptive intensity matrix that matches the visual characteristics of the human eye. The values ​​of the adaptive intensity matrix are limited to a reasonable range, and the CNN adaptive embedding intensity prediction network dynamically allocates the embedding intensity according to the importance of the service partition to ensure that there is no visual distortion after the watermark is embedded. Through this intensity allocation rule, it is possible to ensure that the key data is clear and without distortion, and also to ensure the robustness of the watermark in the background area, so as not to affect the normal display and operation of the relay protection device.

[0027] 5. Perform watermark embedding after intensity prediction. The encoded watermark is bound to the fused features through element-wise multiplication, and the dimensions are adjusted to match the wavelet subband size. The watermark is then embedded into the LH and HL subbands using an adaptive intensity matrix to obtain LH' and HL' with embedded watermarks. LL and HH are kept unchanged. The four subbands are reconstructed through inverse wavelet transform, and then the image is inversely normalized to obtain the final interface image with the hidden watermark.

[0028] 6. Watermark extraction adopts an end-to-end blind extraction method. The suspicious interface images to be detected undergo the same preprocessing and wavelet decomposition as in the embedding stage to extract LH and HL subbands; Input the two sub-bands into the trained watermark robust extraction network and output the encoded state to extract the watermark. The watermark extracted from the encoded state is decoded using the same encryption key as that used in the embedding stage, and the restored binary watermark information is the decoded watermark. The corresponding verification digest is then recovered from the binary watermark information. Calculate the normalization coefficient and bit error rate of the decoded watermark and the original binary copyright watermark; If the normalization coefficient is not greater than a preset threshold, or the bit error rate is not less than a preset threshold, then the suspicious interface image is determined to be without a valid copyright watermark. If the normalization coefficient is greater than a preset threshold and the bit error rate is less than a preset threshold, then a valid copyright watermark is determined to be decoded from the suspicious image. Then, key business information is extracted from the current suspicious interface image to be detected, the current verification digest is recalculated, and the restored verification digest is further compared with the current verification digest. If the two are consistent, then the suspicious interface image is determined to be from a protected design and the content has not been tampered with. If the two are inconsistent, then the suspicious interface image is determined to be from a protected design but has been illegally tampered with.

[0029] 7. Network Training Process The training process consists of three parts: a feature extraction network, an intensity prediction network, and a watermark robust extraction network.

[0030] The training set contains a large number of relay protection interface samples, as well as perturbation samples such as menu order adjustments, font modifications, icon deletions, and screenshot noise. Each network is trained using the Adam optimizer with corresponding learning rates, batch sizes, and training epochs set, and an early stopping strategy is employed to prevent overfitting. The robust watermark extraction network uses a robust loss function composed of binary cross-entropy loss and NC loss to ensure stable watermark extraction even under perturbations such as screen interface imitation and tampering. Example 2

[0031] See Figure 1 This embodiment describes a method for embedding invisible watermarks on the screen interface of a relay protection device, including the following steps: Step 1: Image and watermark preprocessing: 1.1 Obtain the original screen interface image ; 1.2 Extracting the original screen interface image using image recognition methods The business information is used to generate a corresponding verification digest, and the verification digest is converted into binary copyright watermark information to be embedded. The business information includes static and fixed key business information such as set values, pressure plate status, channel identification, menu layout, area division and waveform channel information, while ignoring dynamic content such as real-time changing sampled values ​​and instantaneous monitoring values. 1.3. Convert the original screen interface image First, the dimensions are standardized (to a uniform 512*512 pixels), then a normalization operation is performed to obtain a normalized interface image. Its expression is as follows: (1) in, Represents a normalized interface image; Represents the original screen interface image; This represents the minimum value of the original screen interface image; This represents the maximum value of the original screen interface image.

[0032] 1.4. Use an encryption key to embed the binary copyright watermark information. Perform bitwise XOR obfuscation encoding to obtain the encoded watermark. ; The encryption key is generated by one or more combinations of the relay protection device number, the field timestamp, and the setting version number. For interfaces without a setting, it is generated by only one or more combinations of the device number and the timestamp, ensuring the uniqueness and security of the key.

[0033] The encoded watermark The expression is: ; (2) in, This indicates the watermark after encoding; Represents binary copyright watermark information; Indicates a bitwise XOR operation; This represents the encryption key.

[0034] Step 2, Multi-level Feature Extraction and Fusion: 2.1 Normalize the interface image Input to CNN feature extraction network The structure consists of 3 convolutional layers, 2 pooling layers, and 1 fusion layer, used to extract macroscopic business partition features from the screen interface. Mid-level digital typesetting characteristics and microscopic strokes and line details ; Among them, macro business zoning characteristics Characterizes the overall functional area ratio and overall structural distribution of the real-time display value, setting value display area, waveform display area, menu operation area, etc. of the relay protection device screen interface; mesoscopic digital layout characteristics. Characterization of text spacing, line height ratio, and layout proportion within a region; microscopic stroke and line detail features. It represents the stroke details and high-frequency information of the line edges of the screen interface.

[0035] Furthermore, CNN feature extraction network The training process is as follows: A training set containing a large number of relay protection device screen interface samples and various disturbance samples (menu order adjustment, font modification, etc.) was constructed. Each sample image was manually and accurately labeled, with macroscopic feature labels, mesoscopic feature labels and microscopic feature labels respectively. Each label corresponds one-to-one with the feature to be extracted by the network to ensure the training is targeted. Among them, the macroscopic feature labels are the coordinates of each functional area of ​​the interface (such as the menu area). Fixed value display area Waveform display area The text describes a series of data structures and labels, including: a section ratio (radio = section area / total screen area), a section for guiding the network to learn the overall structure distribution of the interface, a section for assigning the highest weight during training to ensure the accuracy of macroscopic structural feature extraction, a section for mesoscopic feature labels (including the ratio of menu item spacing to interface width, line height to interface height, and fixed-value number spacing to corresponding display area width), a section for guiding the network to learn interface layout rules to match the text- and data-dominated structure of relay protection interfaces, and a section for microscopic feature labels (marking the coordinates of text stroke edges, waveform lines, and fixed-value decimal point outlines) to guide the network to capture subtle interface features, laying the foundation for subsequent watermark and feature binding and anti-disturbance extraction.

[0036] Due to CNN feature extraction network Since three types of features need to be output simultaneously, a multi-task loss function is adopted. Simultaneously constrain the extraction accuracy of the three-layer features, and update the network parameters through backpropagation, the expression of which is: (3) in, Indicates loss of macroscopic features; This indicates the loss of mesoscopic features; This indicates the loss of microscopic features; This represents the multi-task loss function.

[0037] Specifically, the macroscopic feature loss uses mean squared error (MSE) regression loss to calculate the macroscopic layout features of the network output. Macroscopic feature annotation values Differences between them: (4) in, Indicates mean square error; Indicates macroscopic layout characteristics; Indicates the macroscopic feature annotation value; Represents the macroscopic feature dimension; represents the index of the i-th dimension of the macroscopic feature vector; represents the value of the i-th dimension of the macroscopic feature vector; represents the value of the i-th dimension of the macroscopic feature annotation vector.

[0038] The loss for mesoscopic features also uses mean squared error, and the expression is as follows: (5) in, Indicates mid-level typesetting characteristics; Indicates the labeled values ​​of mesoscopic features; Indicates the mesoscopic feature dimension; This represents the value of the i-th dimension of the mesoscopic eigenvector; This represents the value of the i-th dimension of the mesoscopic feature annotation value vector.

[0039] The micro-feature loss employs cross-entropy loss, applying a binary classification constraint to stroke edges. Its expression is: (6) in, Indicates the dimension of microscopic features; This represents the value of the i-th dimension of the micro-feature annotation value vector; This represents the value of the i-th dimension of the micro-feature vector.

[0040] It should be noted that in terms of training parameter settings, the CNN feature extraction network takes the preprocessed interface image of the relay protection device as input and outputs microscopic stroke and line detail features, mesoscopic layout features, and macroscopic layout features. The Adam optimizer is used for iterative parameter updates, with an initial learning rate of [value missing]. The batch size is set to 16 during training, for a total of 30 training epochs, and an early stopping mechanism is implemented: when the validation set shows a total loss... The decline in each of the five consecutive rounds was less than If the network overfits, training should be terminated immediately to avoid overfitting while ensuring the accuracy and efficiency of feature extraction. In the later stages of training, the learning rate can be fine-tuned based on the performance on the validation set to further improve the stability and generalization ability of feature extraction.

[0041] 2.2. The three types of features are weighted and fused to obtain the fused features. Its expression is: (7) in, Indicates fusion characteristics; Indicates the weight of macroscopic features; Indicates the weight of mesoscopic features; This represents the weight of micro-features.

[0042] In this embodiment of the invention, the weight is taken as... , , And satisfy Priority should be given to ensuring the dominance of macro-layout characteristics.

[0043] Step 3, Discrete Wavelet Transform (DWT) Decomposition: The normalized image is decomposed using a first-order discrete wavelet transform. The decomposition yields a low-frequency subband LL, a horizontal high-frequency subband LH, a vertical high-frequency subband HL, and a diagonal high-frequency subband HH. Specifically, the input normalized image is subjected to low-pass / high-pass filtering and downsampling operations in both the row and column directions to obtain four sub-bands decomposed using the db4 wavelet basis, as follows: (8) in, The wavelet transform operation represents the normalized image; LL represents the low-frequency subband; LH represents the horizontal high-frequency subband; HL represents the vertical high-frequency subband; and HH represents the diagonal high-frequency subband.

[0044] The LL sub-band is obtained by first low-pass filtering the rows and then low-pass filtering the columns, preserving the overall structure, brightness, and large-area contours of the image, corresponding to the overall layout and menu area of ​​the relay protection interface; the LH sub-band is obtained by first low-pass filtering the rows and then high-pass filtering the columns, mainly extracting horizontal edges, horizontal lines, and text lines; the HL sub-band is obtained by first high-pass filtering the rows and then low-pass filtering the columns, used to extract vertical edges, vertical lines, and menu columns; the HH sub-band is obtained by first high-pass filtering the rows and then low-pass filtering the columns, mainly reflecting diagonal textures, noise, and small specks.

[0045] The mathematical expression of the above decomposition process is as follows: (9) in, Indicates low-pass filtering operation; Indicates a high-pass filter operation; Represents the normalized interface image The image obtained after performing row-direction low-pass filtering is used to preserve the low-frequency components of the image (corresponding to the overall outline of the interface and smooth areas, while filtering high-frequency noise). Represents the normalized interface image The image obtained after performing row-direction high-pass filtering is used to extract high-frequency components (corresponding to interface edges and detailed features, providing a carrier for subsequent watermark embedding).

[0046] Step 4: Adaptive Embedding Intensity Prediction: Low-frequency subband LL, horizontal high-frequency subband LH, and vertical high-frequency subband HL are input into the CNN adaptive embedding intensity prediction network. (2 convolutional layers + 1 output layer) The network learns the visual sensitivity of the human eye in different areas of the screen interface and outputs an adaptive intensity matrix. Its expression is: (10) Constraints: Range of values ​​constraint: ,in This value ensures that there is no visual distortion on the interface after the watermark is embedded (PSNR≥40dB), and does not affect the normal display and operation of the relay protection device screen; Size constraints: The matrix size is perfectly matched with the LH and HL subband sizes, ensuring that the intensity value corresponds to the embedding position pixel by pixel, thus achieving refined adaptive watermark embedding.

[0047] It should be noted that although the watermark is only embedded in the LH and HL subbands, the LL subband, as a low-frequency subband, carries the overall structure of the interface, brightness distribution, and functional zoning features (such as the global distribution of dense text areas, blank areas, and menu borders). It is the core basis for judging the visual sensitivity of the human eye. The LH and HL subbands only reflect local edge features. The combination of the three allows the network to accurately identify "visually sensitive areas (such as the core structure area of ​​the interface)" and "visually insensitive areas (such as the edge area of ​​text strokes)", avoiding inaccurate intensity prediction caused by only inputting LH and HL.

[0048] Furthermore, the adaptive intensity matrix is ​​output. The process specifically includes: Based on the edge intensity of the normalized interface image, a human visual masking model is constructed. Based on a human visual masking model, visual sensitivity is differentiated among different functional display zones on the screen interface of a relay protection device, and the optimal embedding strength for each zone is calculated. Its expression is: ; (11) in, This represents the optimal embedding strength of the normalized image; This represents the maximum embedding strength, in this embodiment. Set to 0.02 to ensure no visual distortion; k represents the adjustment coefficient, which is set to 10 in this embodiment; This represents the edge intensity at pixel (x, y) in the normalized image; This represents the exponential decay factor based on edge strength.

[0049] The low-frequency subband, horizontal high-frequency subband, and vertical high-frequency subband are input into the trained CNN adaptive embedding intensity prediction network, which outputs an adaptive intensity matrix. Among them, the optimal embedding strength As training labels, the mean squared error loss function is used to constrain the error between the predicted intensity and the optimal intensity as follows, and the network parameters are updated through backpropagation: (12) in, This represents the mean squared error loss function; Indicates the optimal embedding strength; H represents the size of the intensity matrix; W represents the height (number of rows) of the embedded intensity matrix; x represents the row index of the matrix; y represents the column index of the matrix. Coordinates representing network predictions The watermark embedding strength at the location.

[0050] It should be noted that the CNN adaptive embedding strength prediction network dynamically allocates embedding strength according to the importance of business partitions, with the embedding strength of key information display areas being lower than that of non-key background areas. Key information display areas include real-time display values, waveform display areas, fixed value areas, and data annotation areas; non-key background areas include blank areas of the interface, backgrounds without information filling for function buttons, and border decoration areas. Through this strength allocation rule, both the clarity and distortion-free nature of key data and the robustness of watermarks in background areas are ensured.

[0051] To achieve the above functions, the network is trained using the low-frequency subband LL, horizontal high-frequency subband LH, and vertical high-frequency subband HL obtained from wavelet transform as input, and an adaptive intensity matrix as output. The training process employs the Adam optimizer for parameter updates, with an initial learning rate set to... The batch size was set to 32, and a total of 20 iterations of training were completed to ensure that the network could accurately learn the visual sensitivity of different regions of the interface and output an intensity matrix that met the constraints. .

[0052] Step 5: Watermark embedding operation: 5.1. Encode the watermark With fusion features Perform element-wise multiplication binding and combine it with an adaptive strength matrix. Generate the watermark signal to be embedded; 5.2 Embed the watermark signal into the horizontal high-frequency subband LH and the vertical high-frequency subband HL to obtain the modified horizontal high-frequency subband. and vertical high-frequency subband ; Specifically, the encoded watermark is bound to the fusion feature by multiplying element by element to obtain the bound watermark signal; After binding, the watermark signal is adjusted to the same dimension as the horizontal / vertical high-frequency sub-band to obtain the adapted binding feature map, the expression of which is: (13) in, This indicates element-wise multiplication; Indicates dimensional reshaping operation, ensuring and Same size; This represents the feature map after binding; This indicates the size of the horizontal high-frequency subband.

[0053] The adapted binding feature map is multiplied element-wise with the adaptive intensity matrix to obtain the watermark signal to be embedded. ; The watermark signal to be embedded The modified horizontal and vertical high-frequency sub-bands are obtained by superimposing pixel-by-pixel onto the horizontal and vertical high-frequency sub-bands, respectively, and their expressions are as follows: (14) (15) in, This indicates the modified horizontal high-frequency subband; This indicates the modified vertical high-frequency subband; Step 6, Inverse Wavelet Transform and Post-processing: 6.1. The modified horizontal high-frequency sub-band, vertical high-frequency sub-band, low-frequency sub-band, and diagonal high-frequency sub-band are reconstructed using inverse discrete wavelet transform to obtain the interface image with embedded watermark. Its expression is: (16) in, This represents the interface image after the watermark is embedded.

[0054] 6.2. For the interface image after embedding the watermark Inverse normalization is performed to restore the pixel range to its original range (e.g., [0, 255]), ultimately outputting a relay protection device screen interface image containing an invisible watermark. And satisfy the peak signal-to-noise ratio .

[0055] The expression for the screen interface image of the relay protection device containing the invisible watermark is: (17) in, This indicates that the final output is an image of the relay protection device screen interface containing an invisible watermark. Example 3

[0056] See Figure 2 A method for extracting invisible watermarks from the screen interface of a relay protection device includes the following steps: S1. Preprocessing of the image to be detected: Acquire suspicious interface images to be detected (Including disturbances such as imitation / menu adjustments / font modifications); The suspicious interface image is normalized and its size is standardized (to 512*512 to ensure consistency with the input dimension during the embedding stage) to obtain a standardized suspicious image. Its expression is: (18) in, Indicates standardized suspicious images; This indicates a suspicious interface image.

[0057] S2. Wavelet Subband Decomposition: First-order discrete wavelet transform is used to normalize suspicious images. Decomposition yields the low-frequency subbands corresponding to the embedding stage. Horizontal high-frequency subband Vertical high-frequency subband and diagonal high-frequency subband Its expression is: (19) in, This represents the low-frequency subband corresponding to the embedding stage; This represents the horizontal high-frequency subband corresponding to the embedding stage; This represents the vertical high-frequency subband corresponding to the embedding stage; This represents the diagonal high-frequency subband corresponding to the embedding stage.

[0058] S3. End-to-end watermark extraction: The horizontal and vertical high-frequency subbands corresponding to the embedding stage are input into the CNN watermark robust extraction network. (Convolution + pooling + upsampling structure, LeakyReLU activation), the output encoding state is used to extract the watermark, and its expression is: (20) in, This indicates an encoded watermark.

[0059] The watermark extracted from the encoded state is decoded using the encryption key, and the restored binary watermark information is the decoded watermark, whose expression is: ;(twenty one) in, This indicates the watermark after decoding.

[0060] Furthermore, CNN's robust watermark extraction network The training process is as follows: For each original sample and perturbed sample, a complete watermark embedding process is first performed to obtain a watermarked image. ; After normalization and wavelet decomposition, the horizontal and vertical high-frequency sub-bands are taken as network input, and the original binary copyright watermark is used. As training labels, a supervised training pair of "perturbation subband - original watermark" is constructed to enable the network to learn watermark features in scenarios of interface tampering and imitation. The loss function employs a robust loss function that balances watermark extraction accuracy and perturbation similarity, and its expression is: ;(twenty two) in, Represents the robust loss function; This represents the binary cross-entropy loss function; Indicates the loss weighting coefficient; This represents the normalized correlation coefficient loss (NC loss).

[0061] It should be noted that the base loss is the binary cross-entropy loss. The expression for (ensuring the accuracy of watermark extraction bits) is: ;(twenty three) in, denoted as binary cross-entropy loss; M represents the number of rows in the watermark matrix; N represents the number of columns in the watermark matrix; i represents the row index of the watermark matrix; j represents the column index of the watermark matrix; This represents the bit value of the original embedded watermark in the i-th row and j-th column (the value is 0 or 1, corresponding to the binary bits of the binary watermark). This represents the bit value of the watermark extracted by the network in the i-th row and j-th column (the value ranges from 0 to 1, representing the probability that the watermark bit at that position is predicted to be 1).

[0062] Robust Loss: NC Loss The expression for (improving watermark similarity under perturbation) is: ;(twenty four) in, This represents the normalized loss.

[0063] Training optimization uses the Adam optimizer with an initial learning rate of [missing information]. After 20 rounds of training, it decays to The batch size is set to 8, and an early stopping mechanism is set: if the NC value of the validation set does not improve significantly for 5 consecutive rounds (the increase is less than 0.01), the training is terminated to prevent overfitting and ensure the network's generalization ability and extraction stability in perturbation scenarios.

[0064] S4. Similarity determination: Simultaneously, copyright watermark similarity determination and summary watermark comparison are required to double-verify the copyright ownership of suspicious interface images and whether they have been tampered with. The specific process is as follows: S41. Copyright Watermark Similarity Determination The normalization coefficients and bit error rates of the decoded watermark and the original binary copyright watermark are calculated, and their expressions are as follows: (25) (26) in, Represents the normalization coefficient, with a value range of 1. The closer the value is to 1, the higher the watermark matching degree. Represents the bit error rate, with a value range of 100%. The closer the value is to 0, the higher the accuracy of watermark extraction.

[0065] If the normalization coefficient (NC) is not greater than the preset threshold, or the bit error rate (BER) is not less than the preset threshold, then it is directly determined that there is no valid copyright watermark and the interface image does not belong to the protected design. If NC is greater than the preset threshold and BER is less than the preset threshold, then it is determined that a valid copyright watermark has been decoded from the suspicious image. The interface image originates from a protected design, and the next step is to verify the digest comparison.

[0066] S42. Verify digest consistency comparison The verification summary is restored from the valid copyright watermark. At the same time, image recognition is performed on the current suspicious interface image to extract static fixed business information such as set value, pressure plate status, channel identification, menu layout, waveform display area, and functional partition structure, while ignoring dynamic changes such as real-time sampling value and instantaneous monitoring value. Based on this, the current verification summary is calculated. Perform a consistency comparison between the restored checksum and the current checksum.

[0067] S43. Comprehensive Judgment Rules If the verification digest matches and the watermark similarity meets the preset threshold requirement, then the suspicious interface image is determined to originate from the protected design and the interface structure has not been tampered with. If the verification digest does not match, but the watermark similarity meets the threshold requirement, then the suspicious interface image is determined to originate from a protected design, but the interface structure has been illegally tampered with. If the watermark similarity does not meet the threshold requirement, it is directly determined that there is no valid copyright watermark and it is not a protected design.

[0068] In this embodiment of the invention, a normalization coefficient threshold is set. Bit error rate threshold The judgment rule is as follows: .

[0069] In summary, the method of the present invention has the following characteristics and functions: (1) The CNN network training adopts an optimization strategy that combines multi-task loss and robust loss to specifically adapt to three types of network training needs: the feature extraction network adopts a three-layer convolution and two-layer pooling structure, and accurately extracts the structural features of the relay protection interface through multi-dimensional loss constraints of macro business partitioning, meso digital layout, and micro stroke and line details; the embedding intensity prediction network is trained based on the human eye visual masking model, and dynamically allocates the embedding intensity in combination with the importance of business partitioning (the intensity of the key information display area is lower than that of the non-key background area), ensuring the watermark invisibility and not affecting the display of key data on the interface; the watermark robust extraction network integrates binary cross-entropy loss and NC (normalized correlation coefficient) loss, which greatly improves the robustness of watermark extraction in scenarios where the interface is imitated, tampered with, or subject to screenshot noise, and adapts to complex working conditions in industrial sites.

[0070] (2) The training dataset is specifically constructed to fully cover the infringement scenarios in industrial sites: it includes a large number of original interface samples of relay protection devices, as well as typical disturbance samples such as menu order shuffling, font / line height modification, icon deletion, screenshot noise, interface redrawing, resolution adjustment, and partial occlusion. It fully covers the interface tampering and imitation scenarios commonly encountered in actual infringement evidence collection, enabling the network to fully learn the characteristic rules of interface imitation / tampering and accurately adapt to the actual needs of copyright tracing and infringement evidence collection in industrial sites.

[0071] (3) Hierarchical network training + refined parameter design, taking into account training accuracy, anti-overfitting ability and industrial real-time performance: For the three types of networks, namely feature extraction, intensity prediction and watermark extraction, the Adam optimizer, corresponding learning rate, batch size and training rounds are designed respectively. Combined with Dropout regularization, LeakyReLU activation function and early stopping strategy (stop training when the validation set loss does not decrease for several consecutive rounds), overfitting is effectively avoided while ensuring training accuracy. According to the actual test, the inference time of a single image is ≤500ms, which fully meets the industrial real-time requirements of on-site inspection and real-time evidence collection of relay protection devices, and realizes local closed-loop verification of "instant verification".

[0072] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for embedding an invisible watermark on the screen interface of a relay protection device, characterized in that, include: Get the original screen interface image; Business information is extracted from the original screen interface image using image recognition methods to generate a corresponding verification digest, and the verification digest is converted into a binary copyright watermark to be embedded. The same original screen interface image is then subjected to size standardization and normalization processing in sequence to obtain a normalized interface image; The binary copyright watermark to be embedded is combined with the encryption key for obfuscation encoding to obtain the encoded watermark; The normalized interface image is input into a CNN feature extraction network to extract macro-level business partition features, meso-level digital layout features, and micro-level stroke and line detail features, and then the fused features are obtained through weighted fusion. The normalized interface image is decomposed using a first-order discrete wavelet transform to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands. The low-frequency sub-band, horizontal high-frequency sub-band, and vertical high-frequency sub-band are input into the CNN adaptive embedding intensity prediction network, and the adaptive intensity matrix is ​​output. The encoded watermark and the fusion feature are bound by element-wise multiplication and combined with an adaptive intensity matrix to generate a watermark signal to be embedded. The watermark signal to be embedded is then embedded into the horizontal high-frequency sub-band and the vertical high-frequency sub-band to obtain the modified horizontal high-frequency sub-band and the vertical high-frequency sub-band. The modified horizontal high-frequency sub-band, vertical high-frequency sub-band, low-frequency sub-band, and diagonal high-frequency sub-band are reconstructed using inverse discrete wavelet transform to obtain the interface image with embedded watermark. The interface image with the embedded watermark is denormalized to output a relay protection device screen interface image containing an invisible watermark.

2. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 1, characterized in that, The expression for the encoded watermark is: ； in, This indicates the watermark after encoding; Represents binary copyright watermark information; Indicates a bitwise XOR operation; This represents the encryption key.

3. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 1, characterized in that, The macro-level business partition characteristics include the overall functional partition ratio and the overall structural distribution; The meso-level digital typesetting features include text spacing, line height ratio, and the proportion of typesetting within a region; The microscopic stroke and line detail features include stroke details and high-frequency information of line edges.

4. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 1, characterized in that, The CNN feature extraction network is trained using a multi-task loss function, which includes macro-feature loss, meso-feature loss and micro-feature loss. The expression for the multi-task loss function is: ； ； ； ； in, Indicates loss of macroscopic features; Indicates macro-level layout characteristics; Indicates the macroscopic feature annotation value; Represents the macroscopic feature dimension; i represents the index of the i-th dimension of the macro / meso / micro feature vector; This represents the value of the i-th dimension of the macroscopic feature vector; This represents the value of the i-th dimension of the macroscopic feature annotation value vector; This indicates the loss of mesoscopic features; Indicates mid-level typesetting characteristics; Indicates the labeled values ​​of mesoscopic features; Indicates the mesoscopic feature dimension; This represents the value of the i-th dimension of the mesoscopic eigenvector; This represents the value of the i-th dimension of the mesoscopic feature annotation value vector; This indicates the loss of microscopic features; Indicates the dimension of microscopic features; This represents the value of the i-th dimension of the micro-feature annotation value vector; This represents the value of the i-th dimension of the micro-feature vector; Represents the multi-task loss function; This represents the mean square error.

5. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 1, characterized in that, The process of obtaining the adaptive intensity matrix includes: Based on the edge intensity of the normalized interface image, a human visual masking model is constructed. Based on the aforementioned human visual masking model, visual sensitivity is differentiated for different functional display zones on the screen interface of the relay protection device, and the optimal watermark embedding strength for each zone is calculated. ; The low-frequency sub-band, horizontal high-frequency sub-band, and vertical high-frequency sub-band are input into a trained CNN adaptive embedding intensity prediction network, which outputs an adaptive intensity matrix. ; wherein, with the optimal embedding strength The mean squared error loss function is used to train the CNN adaptive embedding strength prediction network as training labels.

6. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 5, characterized in that, The optimal embedding strength The expression is: ； in, This represents the optimal embedding strength of the normalized image; Indicates the maximum embedding strength; k represents the adjustment coefficient; This represents the edge intensity at pixel (x, y) in the normalized image; This represents the exponential decay factor based on edge strength.

7. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 1, characterized in that, The obtained modified horizontal high-frequency subband and vertical high-frequency subband include: The encoded watermark is bound to the fusion feature by multiplying element by element to obtain the bound watermark signal. The dimensions of the bound watermark signal are adjusted to be the same as the dimensions of the horizontal high-frequency sub-band / vertical high-frequency sub-band to obtain the adapted bound feature map. The adapted binding feature map is multiplied element-wise with the adaptive intensity matrix to obtain the watermark signal to be embedded. The watermark signal to be embedded is superimposed pixel by pixel onto the horizontal high-frequency sub-band and the vertical high-frequency sub-band to obtain the modified horizontal high-frequency sub-band and the vertical high-frequency sub-band respectively.

8. The method for embedding an invisible watermark on the screen interface of a relay protection device according to claim 1, characterized in that, The output of the relay protection device screen interface image containing the invisible watermark is expressed as follows: ； in, This represents the output screen interface image of the relay protection device, which includes an invisible watermark. This represents the normalized interface image obtained after embedding the watermark and undergoing inverse discrete wavelet transform. Represents the original screen interface image; This represents the maximum value of the original screen interface image; This represents the minimum value of the original screen interface image.

9. A method for extracting invisible watermarks from the screen interface of a relay protection device, characterized in that, include: Acquire images of suspicious interfaces to be detected; The suspicious interface image is normalized and its size is standardized to obtain a standardized suspicious image; The standardized suspicious image is decomposed using a first-order discrete wavelet transform to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands corresponding to the embedding stage. The horizontal and vertical high-frequency subbands corresponding to the embedding stage are input into the CNN watermark robust extraction network, and the output is the encoded state to extract the watermark.

10. The method for extracting invisible watermarks from the screen interface of a relay protection device according to claim 9, characterized in that, Also includes: The watermark extracted from the encoded state is decoded using an encryption key to obtain the restored binary watermark information as the decoded watermark, and the corresponding verification digest is recovered from the binary watermark information. For the currently suspected interface image to be detected, key business information is extracted through image recognition, and the current verification digest is recalculated; Calculate the normalization coefficient and bit error rate of the decoded watermark and the original binary copyright watermark; If the normalization coefficient is not greater than a preset threshold, or the bit error rate is not less than a preset threshold, then the suspicious interface image is determined to be without a valid copyright watermark. If the normalization coefficient is greater than a preset threshold and the bit error rate is less than a preset threshold, then it is determined that a valid copyright watermark is obtained from the suspicious image and a verification digest is restored from the valid copyright watermark. The restored verification digest is compared with the current verification digest. If they match, the suspicious interface image is determined to originate from a protected design and its content has not been tampered with. If they do not match, the suspicious interface image is determined to originate from a protected design but has been illegally tampered with.

Images