Neural network-based web application leak trace evidence method and system

Abstract

The present application belongs to the technical field of information security, and particularly relates to a Web application leak traceability evidence collection method and system based on a neural network, which generates an event by setting a watermark on a third-party application system, and uses the watermark event to obtain screen capture image watermark encryption data containing key parameter information, wherein the watermark event encodes the key parameters in combination with the third-party application system scene and embeds the watermark encoding information into the screen capture image based on the base color positioning, and the key parameter information includes a user identifier, a system identifier, a timestamp and a BCH error correction supervision bit; for a screen capture leak traceability request, the watermark generation event is triggered to obtain the relevant screen capture leak image; the trained image classification deep neural network is used to extract the key parameters in the relevant screen capture leak image watermark encryption data, the extracted key parameters are matched with the database to generate an evidence collection report of the feedback third-party application system, and the precise positioning of the leak source can be realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of information security, and particularly relates to a Web application leakage traceability evidence method and system based on a neural network. BACKGROUND

[0002] With the popularity of portable electronic products and the development of computer networks, problems such as stealthy shooting and screen shooting are increasingly prominent. And especially under the impetus of the Internet, it is very common to use Web sites to handle general businesses. Especially in some banks, government and other internal network environments with high security requirements, although there are firewall, USB abnormal storage medium detection technology to support to prevent data leakage. But it cannot resist the effect of mobile phones and other image capture devices. The enterprise sector also often has similar web-based cases that cannot be traced, such as the 2016 event of a certain enterprise illegally pre-purchasing customized moon cakes. In the traditional traceability method, a watermark is embedded into the original carrier image in the Fourier Mellin domain, and then inverse log-polar mapping (ILPM) is used to resist lens distortion such as scaling, translation and rotation, and finally the watermark information is extracted from the leaked picture to complete the traceability evidence. These watermark methods cannot solve the distortion problems such as color, illumination and moire caused by screen shooting. SUMMARY

[0003] Therefore, the application provides a Web application leakage traceability evidence method and system based on a neural network, which can be used for the scene of screen shooting secret application in real life, for example, in the scene of using a screen shooting terminal to take a picture of a work computer without USB drive and prohibiting public network transmission of e-mail, and then performing leakage traceability after the screen shooting picture stream is transmitted to the Internet, so as to realize accurate positioning of the leakage source.

[0004] According to the design scheme provided by the application, a Web application leakage traceability evidence method based on a neural network is provided for third-party application system screen shooting leakage traceability, which includes the following contents:

[0005] A watermark generation event is set on the third-party application system, and watermark encryption data of a screen shooting image containing key parameter information is obtained by triggering the watermark generation event, wherein the watermark generation event encodes the key parameters based on the scene of the third-party application system and embeds the watermark encoding information into the screen shooting image based on the texture positioning, and the key parameter information includes a user identifier, a system identifier, a timestamp and a BCH error correction supervision bit;

[0006] For a third-party application system screen shooting leakage traceability request, a related screen shooting leakage image is obtained by triggering a watermark generation event by itself;

[0007] The trained image classification deep neural network is used to extract key parameters in the relevant screen leakage image watermark encryption data, and the extracted key parameters are matched with the database to generate a feedback third-party application system forensics report.

[0008] As the Web application leakage traceability forensics method based on the neural network in the application, further, in the watermark generation event, first, the watermark encoding is generated by the snowflake generation algorithm based on the BCH error correction, and the watermark encoding composed of the user identifier, the system identifier, the timestamp, the BCH error correction supervision bit and the extension bit; then, the RGB color value of the screen camera image is converted into the YCbCr color space, and the Cb and Cr channels are used as the carrier of the undercoat, and the watermark encoding is embedded in the Y channel matrix, and the watermark screen camera image containing the watermark encryption data is generated by the data fusion of the Cb, Cr and Y channels.

[0009] As the Web application leakage traceability forensics method based on the neural network in the application, further, for the watermark encoding, the watermark encoding is embedded in the Y channel matrix according to the preset embedding rule, wherein the preset embedding rule is that for the binary sequence of the watermark encoding, a matrix of M*N size is used as the carrier for embedding each 1bit information in the binary sequence, and the matrix is equally divided into M / 2*N matrix blocks; if the embedded 1bit information is 0, the pixel values of the upper matrix block are all assigned as 0, and the pixel values of the lower matrix block are all assigned as 255, if the embedded 1bit information is 1, the pixel values of the upper matrix block are all assigned as 255, and the pixel values of the lower matrix block are all assigned as 0.

[0010] As the Web application leakage traceability forensics method based on the neural network in the application, further, for the relevant screen leakage image, first, the image is preprocessed to obtain a sub-image containing complete watermark information, and the coordinates of all watermark data in the leakage image are located by using the row and column equal interval division algorithm, and the located coordinates are used to correct the watermark contour, and the corrected image is used as the input of the image classification neural network, wherein the preprocessing operation includes image enhancement, image cropping and geometric transformation.

[0011] As the neural network-based Web application leakage trace evidence method of the application, further, the row-column equidistant division algorithm is used to locate the coordinates of all watermark data in the leakage image, including the following processes: first, the first key point and the second key point in the image are set according to the width and height dimensions of the screen camera leakage image, and the width and height dimensions of the complete watermark block are obtained according to the xy-axis coordinates of the two key points; then, the starting point of the line is set, and straight lines are drawn along the x and y axes with the width and height of the complete watermark block as the interval, and the corner points of the straight lines are taken as the watermark key points; then, starting from the first key point, the starting point that meets the preset condition distance is found along the upper left corner direction, and the positioning coordinates of the complete watermark block composed of the other three points that meet the preset condition distance are obtained clockwise, the starting point position is moved under the preset condition distance according to the row priority principle, the positioning coordinates of the complete watermark block are repeatedly selected, and the entire screen camera leakage image is traversed, and the watermark blocks that do not meet the width and height dimensions of the complete watermark block are filtered from the selected watermark blocks, wherein the preset condition distance is Δx and Δy are the width and height dimensions of the complete watermark block, respectively.

[0012] As the neural network-based Web application leakage trace evidence method of the application, further, in the watermark block contour correction, the obtained watermark block positioning coordinates are used to correct the image contour through space transformation, so that the corrected watermark block contour is in the positive direction.

[0013] As the neural network-based Web application leakage trace evidence method of the application, further, the image classification neural network model uses the DenseNet image classification network to complete the image watermark decoding classification, and uses the dense block as the component unit of the neural network to unify the feature map size in each dense block.

[0014] As the neural network-based Web application leakage trace evidence method of the application, further, in the extraction of the image watermark key parameters by the image classification neural network model, first, the neural network model is used for classification, a plurality of groups of watermark encryption information are obtained through confidence filtering, and then the BCH code is used for error correction to obtain the final watermark decoding information.

[0015] Further, the application also provides a neural network-based Web application leakage trace evidence system for screen camera leakage trace of a third-party application system, including a watermark generation module and a leakage trace module, wherein,

[0016] The watermark generation module is used for setting a watermark generation event on the third-party application system, and obtaining screen camera image watermark encryption data containing key parameter information by triggering the watermark generation event, wherein the watermark generation event encodes the key parameters and embeds the watermark encoding information into the screen camera image based on the undercoat positioning, and the key parameter information includes a user identifier, a system identifier, a timestamp and BCH error correction supervision bits.

[0017] The leakage tracing module is used for a third-party application system screen camera leakage tracing request, obtains relevant screen camera leakage images by triggering the watermark generation event, extracts key parameters in the relevant screen camera leakage image watermark encryption data by using the trained image classification deep neural network, and generates a forensic report of the feedback third-party application system by matching the extracted key parameters with a database.

[0018] The present application has the following advantages:

[0019] The present application embeds a graph containing a user identifier and a timestamp watermark in a Web application, and the watermark information can still be extracted from the graph when the application is screened and photographed; in order to increase the robustness of the web screen camera steganography, the DenseNet neural network and the BCH encoding error correction method are combined to realize the web screen leakage tracing; in the implementation process, the Cr and Cb channels in the YCrCb three channels are used for positioning block embedding, the Y channel is used for writing user information to generate a watermark, and then the identification watermark containing the user information of the third-party system is embedded in any third-party Web system; once the third-party system is maliciously screened, the leakage information can be extracted through the system, and the test accuracy of a single watermark is about 91%; the accuracy can reach 99.1% after BCH encoding error correction of 10 repeated watermarks, and the precise positioning and tracing of the screen camera leakage source can be realized, which has good application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The figure is a schematic diagram of the Web application leakage tracing process based on the neural network in the embodiment;

[0021] Figure 2 The figure is a schematic diagram of the self-defined code identification in the embodiment;

[0022] Figure 3 The figure is a schematic diagram of the undercoat generation in the embodiment;

[0023] Figure 4 The figure is a schematic diagram of the encryption information embedding in the embodiment.

[0024] Figure 5 The figure is a schematic diagram of the whole watermark generation process in the embodiment;

[0025] Figure 6For the contrast diagram before and after the color balance enhancement algorithm in the embodiment;

[0026] Figure 7 For the overall flow of watermark positioning in the embodiment;

[0027] Figure 8 For the Dense Block structure in the embodiment;

[0028] Figure 9 For the DenseNet structure in the embodiment;

[0029] Figure 10 For the network model training data set in the embodiment;

[0030] Figure 11 For the combination verification diagram in the embodiment. DETAILED DESCRIPTION:

[0031] In order to make the purpose, technical scheme and advantages of the present application more clear, specific, the present application is further described in detail below in combination with the drawings and technical scheme.

[0032] At present, the general business of Web site processing is very common, although there is firewall, USB abnormal storage medium detection technology support to prevent data leakage, but with the popularization of portable electronic products and the development of computer technology, the screen leakage problem cannot be resisted. The traceability technology used at home and abroad, such as bar code technology and radio frequency identification technology, can be well integrated with information tracking, but the steganographic information is small and easy to be tampered. The embodiment of the present application provides a Web application leakage traceability evidence collection method based on neural network, which is used for third-party application system screen leakage traceability, and includes the following contents:

[0033] S101, setting a watermark generation event on the third-party application system, obtaining screen image watermark encryption data containing key parameter information by triggering the watermark generation event, wherein the watermark generation event encodes the key parameters in combination with the third-party application system scene and embeds the watermark encoding information into the screen image based on the undercoat positioning, and the key parameter information includes user identification, system identification, timestamp and BCH error correction supervision bit;

[0034] S102, for the third-party application system screen leakage traceability request, obtaining the relevant screen leakage image by triggering the watermark generation event by itself; using the trained image classification deep neural network to extract the key parameters in the relevant screen leakage image watermark encryption data, and matching the extracted key parameters with the database to generate the evidence report of the feedback third-party application system.

[0035] Referring to Figure 1As shown, a series of technical means such as watermark generation and watermark extraction are used to form a complete closed loop from embedding to extraction of watermark encryption information, and by providing watermark generation services and leakage tracing evidence services to third-party Web systems, when a screen leakage event occurs in the third-party system, the source of the leakage can be traced through the streaming screen image.

[0036] As a preferred embodiment, in the watermark generation event, first, the watermark code composed of user identification, system identification, timestamp, BCH error correction supervision bit and extension bit is generated by performing watermark encoding based on the snowflake generation algorithm of BCH error correction; then, the RGB color value of the screen image is converted to YCbCr color space, and the Cb and Cr channels are used as the carrier of the undercoat, and the watermark code is embedded in the Y channel matrix, and the data fusion of Cb, Cr and Y channels is used to generate a watermark screen image containing watermark encryption data.

[0037] The snowflake generation algorithm based on CRC check is used to encode the watermark information, and in combination with the Web system scenario, the final watermark code is composed of user identification (12 bits) + system identification (4 bits) + timestamp (32 bits) + BCH error correction supervision bit (15 bits) + extension bit (1 bit), a total of 64 bits. The specific design of the code identifier is as shown in the figure. Figure 3 When the third-party Web application system calls the watermark generation service, these key parameters can be passed in to generate the watermark code.

[0038] Further, for the watermark code, the watermark code is embedded in the Y channel matrix according to the preset embedding rule, wherein the preset embedding rule is that for the binary sequence of the watermark code, a matrix of size M*N is used as the carrier for embedding each 1bit information in the binary sequence, and the matrix is equally divided into M / 2*N matrix blocks; if the embedded 1bit information is 0, the pixel values of the upper matrix block are all assigned to 0, and the pixel values of the lower matrix block are all assigned to 255, if the embedded 1bit information is 1, the pixel values of the upper matrix block are all assigned to 255, and the pixel values of the lower matrix block are all assigned to 0.

[0039] In view of the problems of low image resolution and local missing caused by shooting angle, shooting distance, performance of electronic device and the like, the watermark image in the embodiment adopts YCbCr color space channels, wherein the Cb channel and the Cr channel realize underprint embedding to realize watermark positioning effect. For example, a complete watermark image embeds 64-bit watermark information, 1-bit watermark information is stored in a 16*4 matrix block, and the upper and lower parts are divided into two 8*4 matrixes. The upper black and lower white represent 0, and the upper white and lower black represent 1. In a complete 64*64 watermark image, 1-bit watermark information 16*4 matrix block is tiled in row priority to Y channel to realize embedding of watermark information, and finally Y, Cb and Cr three channels are fused to obtain a watermark image containing watermark information.

[0040] For a 128*128 size white image I, its RGB color value is converted into YCbCr color space, and Cb and Cr channels are used as the carrier of underprint. M cb and M cr represent the pixel matrix of Cb and Cr channels. For embedding of 64-bit encrypted information, a 64*64 matrix is used as the encryption carrier, so for adjacent complete watermark blocks, underprint with clear contrast is designed. M cb and M cr are divided into four equal parts, and The values in M and M are all assigned as 0. Taking the Cb channel matrix as an example, the underprint rendering is shown in Figure 3 , and two colors of underprint are generated.

[0041] The image is converted into YCbCr color space By positioning the underprint of multiple complete watermarks, the Y channel matrix M y corresponding to the complete watermark is taken, and the size is 64*64. The embedding definition rule of encrypted information can be described as follows: the encrypted binary sequence is S=1011010…101010. For embedding of 1-bit information, a 16*4 matrix is used as the embedding carrier, and the upper and lower parts are divided into two 8*4 matrix blocks. If the embedded information is 0, the pixel values of the upper 8*4 matrix are all assigned as 0, and the pixel values of the lower 8*4 matrix are all assigned as 255. If the embedded information is 1, the pixel values of the upper 8*4 matrix are all assigned as 255, and the pixel values of the lower 8*4 matrix are all assigned as 0. The corresponding visual effect display is shown in Figure 4 , wherein M represents a 16*4 two-dimensional matrix embedding information 1, A two-dimensional matrix of 16*4 represents embedded information 0, and 64-bit encrypted information is embedded into a 64*64 matrix according to the embedding rule described above, forming a watermark carrier with encrypted information.

[0042] The final generated watermark image flow is shown in Figure 5 First, 64-bit encrypted information is generated, then the watermark positioning undercoat is embedded into the Cb and Cr channels, then the encrypted information is embedded into the Y channel, and then the Y, Cb and Cr channels need to be fused to finally generate a watermark image containing encrypted information.

[0043] As a preferred embodiment, further, for a related screen camera leak image, first, the image is preprocessed to obtain a sub-image containing complete watermark information, and the row and column equidistance division algorithm is used to locate the coordinates of all watermark data in the leak image, and the located coordinates are used to correct the watermark contour, and the corrected image is used as the input of the image classification neural network, wherein the preprocessing operation includes image enhancement, image cropping and geometric transformation.

[0044] When tracing the source of the leak, js script form can be used to support watermark embedding of any dom element of the interface of a three-party Web application system, and the number of watermark embedding blocks and the transparency of the watermark can be customized to finally complete the security upgrade of the third-party system. When a screen camera leak event occurs in a third-party Web application system, the tracing evidence service interface is called, and the input parameter is a screen camera image. The screen camera image watermark block positioning first performs color equalization processing on the entire image, which improves the contrast between the undercoats of the complete watermark blocks, and enhances the before-and-after effect as shown in Figure 6 .

[0045] The image after image enhancement is cropped and geometrically transformed to obtain an image containing key watermark information. The key point information (the information of two points at the top left and the bottom right) of a complete watermark in the image is manually selected. Through the set row and column equidistance division algorithm, the coordinate information of all complete watermark blocks contained in the leak image can be located. The specific operation steps are as follows: (1) the screen camera leak image has a width of W and a height of H, and the manually selected key point information is P0(x0, y0) and P2(x2, y2); Δx = |x0-x2| and Δy = |y0-y2| are calculated as the width and height of the complete watermark block. (2) Taking P1 as the starting point and Δx as the interval, draw straight lines in the x-axis direction; taking P1 as the starting point and Δy as the interval, draw straight lines in the y-axis direction; the intersection points of the straight lines are the key point coordinates of the watermark. (3) key point screening: find the initial point that meets the condition distance from P0(x0, y0) in the top left direction clockwise to obtain the four-point coordinates as the positioning coordinates of a complete watermark, and Move to Repeat the selection of complete watermark block coordinates, move the starting point position according to the row priority principle, and repeat the above point selection rule. The specific selection is as shown in the following table: Figure 7

[0046] The four-point coordinates of the complete watermark block need to satisfy that the difference between the two-point coordinates in the same row is approximately equal to Δx, and the difference between the two-point coordinates in the same column is approximately equal to Δy. The four-point positioning coordinates that do not satisfy this condition are filtered out. Thus, the watermark block positioning of the screen camera leakage image is completed.

[0047] Further, in the watermark block contour correction, according to the obtained watermark block positioning coordinates, the image contour is corrected through spatial transformation, so that the corrected watermark block contour is in the positive direction.

[0048] The obtained multiple sets of watermark block coordinates are subjected to watermark square contour correction, and the contour of the photo is corrected through spatial transformation. In the specific application process, a 2*3 matrix N can be used to represent the spatial transformation, and finally the watermark block correction becomes a square contour.

[0049]

[0050]

[0051] After the watermark contour correction, the size of a complete watermark block is 64*64. The screen camera image is converted to the YCrCb color space, the two-dimensional matrix of the Y channel is taken, and a complete watermark block of 64*64 is obtained through the watermark positioning coordinates. Multiple 16*16 two-dimensional matrix blocks are obtained through row and column four equal division, which are used as the input of the subsequent neural network.

[0052] As a preferred embodiment, further, the image classification neural network model uses a DenseNet image classification network to complete the image watermark decoding classification, and uses a dense block as a component unit of the neural network to unify the feature map size in each dense block.

[0053] The model can convert the information decoding problem in the image into an image classification problem by encoding the watermark features. Specifically, the DenseNet image classification network can be used to complete the watermark decoding work. When information is written, 16*4 pixel blocks are encoded, and up black and down white represent 0, and up white and down black represent 1. Therefore, 64 bits represent 16 color blocks.

[0054]

[0055] ​The DenseNet network architecture directly connects the layers, allowing the output of each layer to serve as the input for subsequent layers, thus ensuring maximum information transfer between layers. The dense block (DB) is the basic building block of DenseNet, and its structural diagram is shown below. Figure 8 As shown. The layers in the DB are a combination structure of BN-ReLU-Conv. The input of each layer in the DB comes from the output of all the previous layers. If the number of layers in the DB is L, then the number of connections between layers is L(L+1) / 2. Figure 9 This is a schematic diagram of the DenseNet structure used for image classification. The reason for using DB as the building block of DenseNet is to unify the size of the feature maps within each DB block and avoid inconsistent sizes when merging feature maps.

[0056] After initial sampling and labeling, a total of 3168 datasets were created. 2368 datasets were selected as the training set, and 800 as the test set, for model training and optimization. For 16 types of test data structures (such as...),... Figure 10 (a) shows the image preview of category 0. Figure 10 As shown in (b), there are approximately 200 images in each category.

[0057] Furthermore, in this embodiment, the key parameters of the image watermark are extracted using an image classification neural network model. First, the model is used for classification, and multiple sets of watermark encryption information are obtained through confidence filtering. Then, BCH codes are used for error correction to obtain the final watermark decoding information.

[0058] After watermark decoding via neural network and confidence filtering, multiple sets of encrypted watermark information are obtained. Following BCH encoding error correction, if the result matches, the user's encoded data prediction is successful; otherwise, the candidate sequences are further combined for verification until the entire candidate set is verified, yielding the final decoded information. A verification diagram is shown below. Figure 11 As shown, the watermark decoding information is obtained, and the user information, system identifier, etc. contained therein are extracted for database matching. An evidence collection report is generated and returned to the third-party application system, completing the source tracing and evidence collection process.

[0059] Furthermore, based on the above method, this embodiment of the invention also provides a web application leakage tracing and forensics system based on neural networks, used for tracing and investigating leakage from third-party application systems via screen capture, comprising: a watermark generation module and a leakage tracing module, wherein,

[0060] The watermark generation module is configured to set a watermark generation event on the third-party application system, and obtain screen capture image watermark encryption data containing key parameter information by triggering the watermark generation event, wherein the watermark generation event encodes the key parameters and embeds the watermark encoding information into the screen capture image based on the undercoat positioning in combination with the third-party application system scene, and the key parameter information includes a user identifier, a system identifier, a timestamp, and BCH error correction supervision bits.

[0061] The leakage tracing module is configured to trigger the watermark generation event by itself to obtain the relevant screen capture leakage image for a screen capture leakage tracing request of the third-party application system, extract the key parameters in the relevant screen capture leakage image watermark encryption data by using the trained image classification deep neural network, and match the extracted key parameters with the database to generate a forensic report of the third-party application system.

[0062] Unless specifically stated, the relative steps, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0063] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0064] The units and method steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example are generally described in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation does not exceed the scope of the present application.

[0065] Those skilled in the art can understand that all or part of the steps in the above method can be instructed by a program to complete by relevant hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk. Alternatively, all or part of the steps of the above embodiments can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiments can be implemented in the form of hardware or in the form of a software function module. The present application is not limited to any specific form of combination of hardware and software.

[0066] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the present application, and the protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features therein, within the technical scope disclosed by the present application. Such modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A neural network-based web application leakage traceability forensics method for third-party application system screen leakage traceability, characterized in that, The application discloses a watermark generation method and device, and a watermark decoding method and device. The watermark generation event is triggered to obtain screen capture image watermark encryption data containing key parameter information, wherein the watermark generation event encodes the key parameters and embeds watermark encoding information into the screen capture image based on the texture positioning in combination with the third-party application system scene, and the key parameter information contains a user identifier, a system identifier, a timestamp and BCH error correction supervision bits; in the watermark generation event, first, watermark encoding is performed by a snowflake generation algorithm based on BCH error correction code to generate watermark encoding composed of the user identifier, the system identifier, the timestamp, the BCH error correction supervision bits and extension bits; then, the RGB color value of the screen capture image is converted into a YCbCr color space, and the Cb and Cr channels are used as carriers of the texture, and the watermark encoding is embedded into the Y channel matrix according to a preset embedding rule, and the watermark screen capture image containing the watermark encryption data is generated through data fusion of the Cb, Cr and Y channels; the preset embedding rule is that, for a binary sequence of the watermark encoding, a matrix of M*N size is used as a carrier for embedding each 1bit information in the binary sequence, and the matrix is equally divided into M / 2*N matrix blocks; if the embedded 1bit information is 0, the pixel values of the upper matrix block are all assigned as 0, and the pixel values of the lower matrix block are all assigned as 255, if the embedded 1bit information is 1, the pixel values of the upper matrix block are all assigned as 255, and the pixel values of the lower matrix block are all assigned as 0. For a third-party application system screen leakage traceability request, related screen leakage images are obtained by triggering a watermark generation event; and for the related screen leakage images, first, the images are preprocessed to obtain a sub-image containing complete watermark information, and a row and column equal-interval division algorithm is used to locate the coordinates of all watermark data in the leakage image, the located coordinates are used to correct the watermark contour, and the corrected image is used as the input of an image classification neural network, wherein the preprocessing operation includes image enhancement, image cropping and geometric transformation; the row and column equal-interval division algorithm is used to locate the coordinates of all watermark data in the leakage image, including the following processes: first, the first key point and the second key point in the image are set according to the width and height dimensions of the screen leakage image, and the width and height dimensions of the complete watermark block are obtained according to the xy-axis coordinates of the two key points; then, the starting point of the line is set, and straight lines are drawn along the x and y axes with the complete watermark block width and height as the interval, and the corner points of the straight lines are taken as the watermark key points; then, starting from the first key point, the starting point is found in the upper left corner direction that meets the preset condition distance, and the other three points that meet the preset condition distance are sequentially found clockwise to form the positioning coordinates of the complete watermark block, the starting point position is moved under the row priority principle and the preset condition distance, the positioning coordinates of the complete watermark block are repeatedly selected, and the entire screen leakage image is traversed, and the watermark blocks that do not meet the width and height dimensions of the complete watermark block are filtered from the selected watermark blocks, wherein the preset condition distance is , , respectively the width and height dimensions of the complete watermark block. The trained image classification deep neural network is used to extract key parameters in the relevant screen capture leakage image watermark encryption data, and the extracted key parameters are matched with a database to generate a forensic report of the feedback third-party application system.

2. The neural network-based web application leak tracing forensics method of claim 1, wherein, In the watermark block contour correction, the image contour is corrected through space transformation according to the obtained watermark block positioning coordinates, so that the corrected watermark block contour is in a positive direction.

3. The neural network-based web application leak tracing forensics method of claim 1, wherein, The image classification neural network model adopts a DenseNet image classification network to complete image watermark decoding classification, and uses dense blocks as component units of the neural network to unify the feature map sizes in the dense blocks.

4. The neural network-based web application leakage traceability forensics method according to claim 1 or 3, characterized in that, In the extraction of image watermark key parameters by using the image classification neural network model, first, classification is performed by the neural network model, a plurality of groups of watermark encryption information are obtained through confidence filtering, and then BCH code is used for error correction to obtain final watermark decoding information.

5. A neural network-based web application leakage traceability forensics system for third-party application system screen leakage traceability, characterized in that, The application discloses a watermark generation method and device, and a watermark decoding method and device. The watermark generation module is configured to set a watermark generation event on the third-party application system, and obtain screen capture image watermark encryption data containing key parameter information by triggering the watermark generation event. The watermark generation event encodes the key parameters based on the third-party application system scenario and embeds the watermark encoding information into the screen capture image based on the base color positioning. The key parameter information includes a user identifier, a system identifier, a timestamp, and BCH error correction supervision bits. In the watermark generation event, first, the watermark encoding is generated by a snowflake generation algorithm based on BCH error correction code to generate watermark encoding composed of a user identifier, a system identifier, a timestamp, BCH error correction supervision bits, and extension bits. Then, the RGB color value of the screen capture image is converted to YCbCr color space, and the Cb and Cr channels are used as the carrier of the base color. The watermark encoding is embedded in the Y channel matrix according to a preset embedding rule, and the data fusion of the Cb, Cr, and Y channels is used to generate a watermark screen capture image containing watermark encryption data. The preset embedding rule is that for the binary sequence of the watermark encoding, a matrix of M*N size is used as the carrier for embedding each 1bit information in the binary sequence, and the matrix is equally divided into M / 2*N matrix blocks. If the embedded 1bit information is 0, the pixel values of the upper matrix block are all assigned as 0, and the pixel values of the lower matrix block are all assigned as 255. If the embedded 1bit information is 1, the pixel values of the upper matrix block are all assigned as 255, and the pixel values of the lower matrix block are all assigned as 0. The leakage tracing module is used for shielding leakage tracing requests for third-party application systems, obtains relevant screen leakage images by triggering watermark generation events, extracts key parameters in relevant screen leakage image watermark encryption data using a trained image classification deep neural network, matches the extracted key parameters with a database to generate a forensic report for the third-party application system, and for relevant screen leakage images, first pre-processes the images to obtain a sub-image containing complete watermark information, and uses a row-column equal-interval division algorithm to locate the coordinates of all watermark data in the leakage image, corrects the watermark outline using the located coordinates, and uses the corrected image as the input of the image classification neural network, wherein the preprocessing operation includes image enhancement, image cropping, and geometric transformation; the row-column equal-interval division algorithm for locating the coordinates of all watermark data in the leakage image includes the following processes: first, set the first key point and the second key point in the image according to the width and height dimensions of the screen leakage image, and obtain the width and height dimensions of the complete watermark block according to the xy-axis coordinates of the two key points; then, set the starting point of the line, and draw straight lines along the x and y axes with the width and height of the complete watermark block as the interval, and take the corner points of the straight lines as the watermark key points; then, starting from the first key point, find the starting point that meets the preset condition distance along the top-left corner direction, and clockwise find other three points that meet the preset condition distance to form the positioning coordinates of the complete watermark block; move the starting point position under the preset condition distance according to the row priority principle, repeatedly select the positioning coordinates of the complete watermark block, and traverse the entire screen leakage image until the entire screen leakage image is traversed, and filter the watermark blocks that do not meet the width and height dimensions of the complete watermark block from the selected watermark blocks, wherein the preset condition distance is , 、 respectively the width and height dimensions of the complete watermark block.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-4.

Images