Image analysis method and device, electronic equipment and storage medium

Abstract

The invention discloses an image analysis method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a to-be-analyzed watermark file; performing format conversion processing on the to-be-analyzed watermark file to obtain a to-be-analyzed watermark image; inputting the to-be-analyzed watermark image into a first target sub-network of a target analysis model for feature extraction processing to obtain a target feature image; the target feature image comprises a target background color image and a target attention thermal image; performing feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; and inputting the target feature vector and the to-be-analyzed watermark image into a second target sub-network of the target analysis model for image analysis processing, and generating a target watermark-free image. The processing efficiency and accuracy of watermark image analysis can be improved, and the method can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image analysis method, device, electronic device and storage medium. Background Art

[0002] In the related technology, there are some technical solutions for parsing watermarked PDF (Portable Document Format) files. The related technical solutions mainly adopt traditional image processing methods and rely on hand-crafted heuristic algorithms to find watermark edge features. These technical solutions still have limitations in parsing complex watermarks, processing large-scale document collections and accuracy.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The embodiments of the present application aim to solve at least one of the technical problems in the related art to a certain extent. To this end, the main purpose of the embodiments of the present application is to propose an image analysis method, device, electronic device and storage medium, which can improve the processing efficiency and accuracy of watermark image analysis.

[0005] To achieve the above object, an embodiment of the present application provides an image analysis method, which includes the following steps:

[0006] Get the watermark file to be parsed;

[0007] Performing format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed;

[0008] Input the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image;

[0009] Performing feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector;

[0010] The target feature vector and the watermark image to be parsed are input into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image.

[0011] In some embodiments, the step of inputting the to-be-parsed watermark image into the first target subnetwork of the target parsing model for feature extraction to obtain a target feature image comprises:

[0012] Inputting the to-be-parsed watermark image into the first target sub-network of the target parsing model, and extracting image feature information of the to-be-parsed watermark image through the convolutional layer in the first target sub-network;

[0013] Compressing the image feature information through the global maximum pooling layer in the first target sub-network to obtain feature compression information;

[0014] Predicting the target three-primary color output values ​​corresponding to the watermark image to be parsed according to the feature compression information through the fully connected layer in the first target subnetwork;

[0015] The target background color image in the target feature image is generated based on the target three-primary color output values.

[0016] In some embodiments, the step of inputting the to-be-parsed watermark image into the first target subnetwork of the target parsing model for feature extraction to obtain a target feature image comprises:

[0017] Inputting the to-be-parsed watermark image into the first target sub-network of the target parsing model, and extracting image feature information of the to-be-parsed watermark image through the convolutional layer in the first target sub-network;

[0018] Constructing a feature map based on the image feature information;

[0019] Adjusting the image size of the feature map to the image size of the watermark image to be analyzed to obtain a target size feature map;

[0020] The target size feature map is superimposed on the watermark image to be analyzed according to the target image processing function to obtain the target attention thermal image in the target feature image.

[0021] In some embodiments, the step of inputting the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model to perform image parsing processing to generate a target watermark-free image includes:

[0022] The target feature vector and the watermark image to be parsed are input into the second target sub-network of the target parsing model, and the generator in the second target sub-network performs image parsing processing on the target feature vector and the watermark image to be parsed to generate the target watermark-free image.

[0023] In some embodiments, the method further includes the step of constructing the first target sub-network, wherein the step of constructing the first target sub-network includes:

[0024] Acquire a dataset of images to be trained; the dataset of images to be trained includes watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained;

[0025] Performing cluster analysis on the real watermark-free image through a Gaussian mixture model to obtain watermark-free content information and watermark-free background information corresponding to the real watermark-free image, and constructing a background color reference image according to the watermark-free background information;

[0026] Input the watermark image to be trained into the first initial sub-network for feature extraction processing to obtain a first feature image to be trained; the first feature image to be trained includes a first background color image to be trained and a first attention thermal image to be trained;

[0027] Calculating a background color deviation value between the first background color image to be trained and the background color reference image;

[0028] Constructing a first loss function of the first initial sub-network based on the background color deviation value;

[0029] The first initial sub-network is trained based on the first loss function to obtain the first target sub-network.

[0030] In some embodiments, the method further includes the step of constructing a second target sub-network, wherein the step of constructing the second target sub-network includes:

[0031] Acquire a dataset of images to be trained; the dataset of images to be trained includes watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained;

[0032] Inputting the watermark image to be trained into the first target subnetwork for feature extraction processing to obtain a second feature image to be trained; the second feature image to be trained includes a second background color image to be trained and a second attention thermal image to be trained;

[0033] Performing feature combination processing on the second background color image to be trained and the second attention thermal image to be trained to obtain a feature vector to be trained;

[0034] Inputting the feature vector to be trained and the watermark image to be trained into a second initial sub-network, performing image parsing processing through a generator in the second initial sub-network, and obtaining a candidate watermark-free image;

[0035] Performing discriminative processing on the candidate watermark-free image by the discriminator in the second initial sub-network to obtain a discriminative processing result;

[0036] Calculate the image quality deviation between the candidate watermark-free image and the real watermark-free image according to the discrimination processing result to obtain an image quality evaluation result;

[0037] Constructing a generative adversarial network loss function of the second initial sub-network based on the image quality assessment result;

[0038] The generator and the discriminator are optimized based on the generative adversarial network loss function, and the step of inputting the feature vector to be trained and the watermark image to be trained into the second initial sub-network and performing image parsing processing through the generator in the second initial sub-network to obtain a candidate watermark-free image is returned to be executed, until the image quality deviation value meets the preset image quality deviation threshold, and the second target sub-network is obtained.

[0039] In some embodiments, the method further comprises:

[0040] Obtain the captured file image to be parsed;

[0041] Performing color correction processing on the photographed document image to obtain a color-corrected image;

[0042] Affine transformation is performed on the color-corrected image to obtain a target corrected image, and the target corrected image is used as the watermark image to be parsed.

[0043] To achieve the above object, another aspect of the embodiment of the present application provides an image analysis device, the device comprising the following modules:

[0044] A watermark file acquisition module is used to obtain the watermark file to be parsed;

[0045] A watermark image acquisition module is used to perform format conversion processing on the watermark file to be parsed to obtain the watermark image to be parsed;

[0046] A feature extraction processing module is used to input the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image;

[0047] A feature combination processing module is used to perform feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector;

[0048] The image analysis processing module is used to input the target feature vector and the watermark image to be analyzed into the second target sub-network of the target analysis model to perform image analysis processing to generate a target watermark-free image.

[0049] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0050] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0051] The embodiments of the present application include at least the following beneficial effects: The present application provides an image parsing method, device, electronic device and storage medium, which obtains a watermark file to be parsed; performs format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed; inputs the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; performs feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; inputs the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image. The embodiment of the present application performs feature extraction on the watermark image corresponding to the watermark file through the first target sub-network of the target parsing model, and can quickly and accurately extract the background color image and attention thermal image of the watermark image, providing key information of the image for subsequent image analysis. Then, the second target sub-network of the target parsing model performs image parsing processing on the target feature vector composed of the background color image and the attention thermal image and the watermark image to be parsed, and can efficiently generate a high-quality target watermark-free image. That is, the embodiment of the present application decomposes the image parsing task into two stages, feature extraction and image parsing, through the constructed target parsing model, and can efficiently handle complex large-scale image watermark removal tasks, reduce a large amount of pixel calculations, and improve the processing efficiency, accuracy and reliability of watermark image parsing, so that the content of the watermark file can be quickly and accurately identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flowchart of the steps of an image analysis method provided in an embodiment of the present application;

[0053] Figure 2 is a flowchart of steps for correcting a photographed document image provided by an embodiment of the present application;

[0054] Figure 3 is a flowchart of steps for constructing a first target subnetwork provided by an embodiment of the present application;

[0055] Figure 4is a flowchart of steps for constructing a second target subnetwork provided by an embodiment of the present application;

[0056] Figure 5 This is a flowchart of the steps of extracting a target background color image provided by an embodiment of the present application;

[0057] Figure 6 It is a flowchart of the steps of extracting a target attention thermal image provided by an embodiment of the present application;

[0058] Figure 7 This is a schematic diagram of a file parsing system architecture provided by an embodiment of the present application;

[0059] Figure 8 It is a flowchart of an image analysis method provided in an embodiment of the present application;

[0060] Fig. 9 is a structural schematic diagram of an image analysis device provided in an embodiment of the present application;

[0061] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0063] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0064] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] In related technologies, PDF has become a standard format widely used for document communication and storage. However, with the popularization of information sharing, it is particularly important to protect the security of documents and prevent unauthorized copying and tampering. In order to meet this demand, many organizations and individuals have adopted watermarking technology to identify and protect PDF documents. A watermark is a visible or invisible mark embedded in a document to identify the owner, authorization status or other relevant information of the document. Traditional watermarking technology mainly includes visible watermarks and invisible watermarks. Visible watermarks are images or text embedded in the visible area of ​​the document so that they can be directly perceived by the observer; while invisible watermarks embed information in the content or structure of the document to achieve document tracking and authentication. Since PDF was developed as a document exchange format, this also explains why PDF cannot be directly read by machines and is difficult to be edited again, especially for watermarked PDFs. In order to obtain document content data, a complex parsing process is required.

[0067] At present, there are some technical solutions for parsing watermarked PDF documents. These technical solutions are mainly based on analyzing and parsing the PDF file structure, as well as identifying the watermark embedding position and features. However, the relevant technologies still face challenges in parsing complex watermarks, processing large-scale document collections, and accuracy. For example, in work, it is often necessary to convert PDF documents into editable text data for secondary editing. Manual extraction methods are often time-consuming and labor-intensive, and accuracy is difficult to guarantee. At present, most PDF documents with watermarks account for the majority. For parsing watermarked PDF documents, most methods use traditional image processing methods and rely on hand-crafted heuristic algorithms to find watermark edge features. These methods are often only effective for some PDF document images, and their accuracy and reliability are limited. In addition, traditional image processing methods require a large amount of pixel calculations, resulting in low processing performance and unable to meet the real-time requirements of large-scale PDF document processing.

[0068] In view of this, an image parsing method, device, electronic device and storage medium are provided in the embodiments of the present application. The scheme obtains a watermark file to be parsed; performs format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed; inputs the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; performs feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; inputs the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image. The embodiment of the present application performs feature extraction on the watermark image corresponding to the watermark file through the first target sub-network of the target parsing model, and can quickly and accurately extract the background color image and attention thermal image of the watermark image, providing key information of the image for subsequent image analysis. Then, the second target sub-network of the target parsing model performs image parsing processing on the target feature vector composed of the background color image and the attention thermal image and the watermark image to be parsed, and can efficiently generate a high-quality target watermark-free image. That is, the embodiment of the present application decomposes the image parsing task into two stages, feature extraction and image parsing, through the constructed target parsing model, and can efficiently handle complex large-scale image watermark removal tasks, reduce a large amount of pixel calculations, and improve the processing efficiency, accuracy and reliability of watermark image parsing, so that the content of the watermark file can be quickly and accurately identified.

[0069] The image parsing method provided in the embodiment of the present application relates to the field of image processing technology. The image parsing method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network) and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in the blockchain network; the software can be an application that implements the image parsing method, etc., but is not limited to the above forms.

[0070] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs (Personal Computers, personal computers), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0071] See also Figure 1 , Figure 1 is an optional step flow chart of an image analysis method provided in an embodiment of the present application, Figure 1 The method may include but is not limited to steps S101 to S105.

[0072] Step S101, obtaining a watermark file to be parsed;

[0073] In the embodiment of the present application, the watermark file to be parsed mainly refers to the PDF document with watermark to be parsed. The watermarked PDF documents used in the embodiment of the present application are all digital files or PDF scanned files. In the normal work of the user, there may be document pictures that are copied from the camera. The copied photos are affected by the lighting scene and shooting angle. It is worth mentioning that an image processing solution for processing copied document pictures is provided in the image parsing system of the embodiment of the present application. Specifically, on the user operation interface of the image parsing system, the user configures the option of "copying document pictures". The business layer will first perform color transformation and affine transformation on the input image data for correction, and then pass it to the algorithm layer for further parsing and processing to improve the accuracy of the parsing results.

[0074] It is easy to understand that in the embodiment of the present application, the image analysis method supports analysis and processing of various types of watermarked images, mainly for image analysis and processing of watermarked images of watermarked PDF documents. At the same time, the embodiment of the present application also supports image analysis and processing of photocopied document photos.

[0075] In the specific implementation, see Figure 2 , Figure 2is a flowchart of steps for correcting a photographed document image provided by an embodiment of the present application; in some embodiments, the step of correcting the photographed document image may also be included, and the step may include but is not limited to steps S201 to S203:

[0076] Step S201, obtaining a photographed document image to be analyzed;

[0077] As for the photographed document image, it is a document picture taken by a camera during the normal work of the user.

[0078] Step S202, performing color correction processing on the photographed document image to obtain a color-corrected image;

[0079] Among them, the purpose of color correction processing is to adjust the color of the captured document image to obtain a color-corrected image, so that the color of the color-corrected image is closer to the true color of the original document. Color correction processing may include white balance correction, brightness contrast adjustment, color space conversion and other processing to reduce the impact of ambient light on image quality, which is beneficial to subsequent image analysis processing. It should be noted that the implementation principle of color correction processing can be implemented with reference to the technical solutions of related technologies, and the embodiments of this application are not limited to this.

[0080] Step S203, performing affine transformation processing on the color-corrected image to obtain a target corrected image, and using the target corrected image as the watermark image to be parsed.

[0081] Among them, affine transformation processing is a geometric transformation operation, which can change the shape, size and direction of the image, but will not change the straight lines and parallel relationships in the image. In a specific implementation, affine transformation processing can perform operations such as rotation, scaling, and tilt correction to obtain a target corrected image to ensure that the geometric shape, size and direction of the image are consistent with the original document. It should be noted that the implementation principle of affine transformation processing can be performed with reference to the technical solutions of the relevant technology, and the embodiments of the present application are not limited to this.

[0082] The embodiment of the present application can not only process conventional watermarked PDF documents, but also effectively process reproduced document images, thereby improving the applicability of the system and the accuracy of the parsing results.

[0083] Step S102, performing format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed;

[0084] Optionally, the format conversion process is a process of converting the watermark file to be parsed into a digital image. In a specific implementation, if the watermark file to be parsed is a document in PDF format, a PDF conversion tool (such as a PDF reader, PDF to image software, etc.) can be used to convert it into a common image format, which is conducive to subsequent image parsing and processing.

[0085] For the watermark image to be analyzed, in the embodiment of the present application, the watermark image to be analyzed is mainly a digital image converted from the watermark file to be analyzed, that is, a digital image converted from a watermarked PDF document. In addition, the watermark image to be analyzed can also be a document picture taken by a camera, but the document picture taken by the camera needs to be corrected through color transformation and affine transformation before it can be input into the image analysis system for image analysis processing.

[0086] In step S102 shown in the embodiment of the present application, the watermark file to be parsed is converted into a digital image format suitable for subsequent processing through format conversion processing, which not only simplifies the processing flow, but also improves the processing efficiency and accuracy, and lays a data foundation for subsequent image parsing processing.

[0087] Step S103, inputting the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image;

[0088] In the embodiment of the present application, in view of the low accuracy and low performance of the related art, a BEWDSR-Net (Background Estimation Watermark Removal Network) model is designed. The BEWDSR-Net model is a model dedicated to parsing watermarked PDF image documents. The BEWDSR-Net model is mainly composed of two sub-networks, BE-Net (BackgroundEstimation Network) and WDR-Net (Watermark Removal Network). The input of BE-Net is a watermarked PDF image (watermarked image). According to the specific attributes of the PDF image, BE-Net can calculate the background color image (Background color) reflecting the color of the paper and the attention thermal image (Attention map) reflecting the distribution of the watermark. The background color image and the attention thermal image are extremely useful in improving image quality and reducing model parameters; after BE-Net obtains the background color image and the attention thermal image, the three images of the watermarked image, the background color image, and the attention thermal image are input into WDR-Net for image parsing, and finally the watermark-free image (Non-watermarked image) is obtained.

[0089] Among them, the target parsing model is the BEWDSR-Net model, the first target sub-network is BE-Net, and the second target sub-network is WDR-Net. For BE-Net, it is built based on a convolutional neural network with an attention mechanism, consisting of a convolutional layer, a global maximum pooling layer, and a fully connected layer. Specifically, it extracts image features through its multi-layer structure, and uses the attention mechanism to enable the network structure to dynamically focus on important parts of the image, such as text, watermarks, and the edges of the background. For WDR-Net, it is a module for segmenting the background of an image. It is built based on a conditional generative adversarial network. The components of WDR-Net are a generator and a discriminator. The generator uses the U-Net model, and the discriminator uses the PatchGAN model.

[0090] Optionally, the specific attributes of the PDF image specifically refer to: background pixels are generally white, text and watermark pixels are generally dark gray, and edge features are relatively obvious.

[0091] In the specific implementation, the process of acquiring the background color image is as follows: the trained BE-Net network can realize the extraction of the background color image. Specifically, the global maximum pooling layer in the BE-Net network reduces the spatial dimension of the feature map extracted by the convolution layer to 1x1, while retaining the maximum value in each feature map, realizing the pixel classification of the input image and then upsampling to obtain the background color image. For the attention feature map (attention thermal image), the attention feature map in the embodiment of the present application uses the feature map output by the intermediate convolution layer of the BE-Net network. The acquisition process of the attention feature map is as follows: The specific implementation method of extracting the feature map: the image size of the feature map output by the convolution layer is adjusted to the size of the input image (i.e., the watermarked image), and the resized feature map is superimposed on the watermarked image through the function in the cv2 library (image processing library) to obtain the attention feature map.

[0092] Before executing the image reasoning prediction stage of the embodiment of the present application, the step of constructing a target parsing model is also included, specifically the steps of constructing the first target sub-network and the second target sub-network in the target parsing model, see Figure 3 , Figure 3 is a flowchart of steps for constructing a first target subnetwork provided by an embodiment of the present application; in some embodiments, the steps for constructing the first target subnetwork may include but are not limited to steps S301 to S306:

[0093] Step S301, obtaining a dataset of images to be trained; the dataset of images to be trained includes watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained;

[0094] Optionally, the image dataset to be trained is data used to train the BEWDSR-Net model, and the image dataset to be trained includes the watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained.

[0095] Step S302, clustering analysis is performed on the real watermark-free image by using a Gaussian mixture model to obtain watermark-free content information and watermark-free background information corresponding to the real watermark-free image, and a background color reference image is constructed according to the watermark-free background information;

[0096] The background color reference image can be used to compare with the background color image extracted by the first initial sub-network in the subsequent step to confirm the similarity between the two.

[0097] Step S303, inputting the watermark image to be trained into the first initial sub-network for feature extraction processing to obtain a first feature image to be trained; the first feature image to be trained includes a first background color image to be trained and a first attention thermal image to be trained;

[0098] The first feature image to be trained refers to the image obtained during the training of the first target sub-network.

[0099] Step S304, calculating a background color deviation value between the first background color image to be trained and the background color reference image;

[0100] Step S305, constructing a first loss function of the first initial sub-network based on the background color deviation value;

[0101] Step S306: training the first initial sub-network based on the first loss function to obtain the first target sub-network.

[0102] In the specific implementation, the process of constructing the first target sub-network (BE-Net) is as follows: first, the training set of the BEWDSR-Net model of the embodiment of the present application is defined as The training set D consists of N pairs (S1, T i ) image composition, S i is the watermarked image, T i is the watermark-free image corresponding to the watermarked image, i is the serial number, and after training the BEWDSR-Net model based on the training set, the following image prediction function can be formed. The expression of the image prediction function is as follows:

[0103]

[0104] in, Expressed as an image prediction function, Expressed as a function, S represents the input watermarked image, Represented as the watermark-free image predicted by the function. It is impossible to be completely consistent with the training data, so the purpose of training is to minimize the loss function so that the obtained Infinitely close to the training set T i .

[0105] Specifically, after receiving the watermarked image S, the image prediction function returns a predicted watermark-free image that is similar to the true watermark-free image T.

[0106] In an embodiment of the present application, BE-Net is composed of a convolutional layer, a global maximum pooling layer, and a fully connected layer. The convolutional layer can effectively capture the feature information of the input image, the global maximum pooling layer can compress the feature information extracted by the convolutional layer to reduce the computational burden, and the fully connected layer can predict the RGB (Red, Green, Blue, three primary colors) value of the output image. It should be noted that in an embodiment of the present application, when selecting a background color image, in order to bring a better system experience to the user, the user can manually select the background watermark-free area, or use the automated script provided in the embodiment of the present application to automatically identify the background color of the real image. These two selection methods can be configured through the user interface.

[0107] In the specific implementation, the training process of the first target subnetwork (BE-Net) is as follows: first, the watermark-free images in the training set are processed, specifically, the Gaussian Mixture Model (GMM) of expectation maximization is used to perform cluster analysis on the watermark-free images according to the pixel intensity values, and the pixels of the watermark-free images are clustered into two groups, which usually represent the content and background of the image. Among them, since the watermark-free image only contains the background and content, does not contain a watermark, and the document background color is usually brighter, therefore, the embodiment of the present application classifies the brighter group as the background and the darker group as the document content. After obtaining the above-mentioned background color and watermarked image, the embodiment of the present application minimizes the color loss function in a supervised learning manner, and the expression of the color loss function is as follows:

[0108]

[0109] in, It is represented as the color loss function, N is the number of samples, b i Expressed as weight coefficient, is represented as the color feature vector of the i-th sample, and ‖·‖ is represented as the norm operation.

[0110] During the training process of BE-Net, the color loss function is minimized in a supervised learning manner. The color loss function measures the difference between the background color of the background color image generated by BE-Net and the background color of the real watermark-free image. Through optimization methods such as back propagation algorithm and gradient descent, the network parameters of BE-Net are continuously adjusted to reduce the value of the loss function, and finally the trained BE-Net is obtained.

[0111] See also Figure 4 , Figure 4 is a flowchart of steps for constructing a second target subnetwork provided by an embodiment of the present application; in some embodiments, the steps for constructing the second target subnetwork may include but are not limited to steps S401 to S408:

[0112] Step S401, obtaining a dataset of images to be trained; the dataset of images to be trained includes watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained;

[0113] Step S402, inputting the watermark image to be trained into the first target subnetwork for feature extraction processing to obtain a second feature image to be trained; the second feature image to be trained includes a second background color image to be trained and a second attention thermal image to be trained;

[0114] The second feature image to be trained refers to the image obtained during the training of the second target sub-network.

[0115] Step S403, performing feature combination processing on the second background color image to be trained and the second attention thermal image to be trained to obtain a feature vector to be trained;

[0116] Step S404, inputting the feature vector to be trained and the watermark image to be trained into the second initial sub-network, performing image parsing processing through the generator in the second initial sub-network, and obtaining a candidate watermark-free image;

[0117] Step S405, performing discriminative processing on the candidate watermark-free image through the discriminator in the second initial sub-network to obtain a discriminative processing result;

[0118] Step S406, calculating the image quality deviation value between the candidate watermark-free image and the real watermark-free image according to the discrimination processing result to obtain an image quality evaluation result;

[0119] Step S407, constructing a generative adversarial network loss function of the second initial sub-network based on the image quality assessment result;

[0120] Step S408, optimizing the generator and the discriminator based on the generative adversarial network loss function, and returning to execute the step of inputting the feature vector to be trained and the watermark image to be trained into the second initial subnetwork, performing image parsing processing through the generator in the second initial subnetwork, and obtaining a candidate watermark-free image, until the image quality deviation value meets the preset image quality deviation threshold, and obtaining the second target subnetwork.

[0121] In the specific implementation, WDR-Net is a conditional generative adversarial network, which consists of a discriminator and a generator. The discriminator adopts the PatchGAN model, and the generator adopts the U-Net model. It is divided into two parts: the encoder and the decoder. At each spatial resolution, the features of the decoder will be combined with the features of the encoder through a jump connection. In the embodiment of the present application, the data loss function of WDR-Net is defined as:

[0122]

[0123] in, Expressed as the data loss function of WDR-Net, Represents the deviation between the predicted watermark-free image and the real watermark-free image, represents the data loss function of the GAN network (i.e., the formula for the loss function of the Generative Adversarial Network (GAN)), The calculation formula is:

[0124]

[0125] in, represents the expected operation, S1 and T1 represent the input data pair and output data pair respectively, T i represents the original output, Indicates the result obtained through model prediction.

[0126] The calculation formula is:

[0127]

[0128] Among them, P data represents the real data distribution; D(·) represents the discriminator, which is used to determine whether the input data is real; It means maximizing 1-D(·), that is, making the probability that the discriminator thinks the generated sample is fake as large as possible. During the training process of the entire GAN network, it is expected that the generator can deceive the discriminator and make the discriminator think that the generated sample is real. Therefore, from the perspective of the generator, the generator will try to minimize the expression That is to say As close to 0 as possible, so that [log D(S i , T i )] is close to 1, that is, the generated samples look as real as possible.

[0129] Specifically, WDR-Net is part of the BEWDSR-Net network structure. The underlying principle is the Generative Adversarial Network (GAN). During the training process, the generator and the discriminator play a game. The generator tries to generate more and more real data to deceive the discriminator, while the discriminator becomes better at identifying true and false data. This process is completed by alternately training two networks. The conditional information in the embodiment of the present application is specifically three image data (watermarked image, attention thermal image, background color image), and the three images are provided to the generator and the discriminator at the same time. These pixel information is used to guide the generator to generate specific types of data and help the discriminator make more accurate judgments.

[0130] Among them, the specific implementation logic of training the network structure BEWDSR-Net of the embodiment of the present application is: through supervised training, the network learns meaningful visual patterns, and uses these patterns to perform pixel classification tasks. The specific principle is back propagation and optimization. The gradient of the loss function with respect to the network parameters is calculated through the back propagation algorithm, and then the optimizer (such as SGD (Stochastic Gradient Descent, random gradient descent), Adam, etc.) is used to update the weights and biases of the network. The training goal is to minimize the loss function.

[0131] After obtaining the trained BEWDSR-Net model, BE-Net based on the BEWDSR-Net model can perform feature extraction on the watermarked PDF image, calculate the background color image reflecting the color of the paper and the attention thermal image reflecting the watermark distribution. After BE-Net obtains the background color image and the attention thermal image, the three images of the watermarked image, the background color image, and the attention thermal image are input into WDR-Net for image analysis processing, and finally the watermark-free image with the watermark removed is obtained.

[0132] Optionally, the target feature image includes a target background color image and a target attention thermal image.

[0133] See also Figure 5 , Figure 5 is a flowchart of steps for extracting a target background color image provided by an embodiment of the present application; in some embodiments, step S103 may include but is not limited to steps S501 to S504:

[0134] Step S501, inputting the watermark image to be parsed into the first target sub-network of the target parsing model, and extracting image feature information of the watermark image to be parsed through the convolutional layer in the first target sub-network;

[0135] Step S502, compressing the image feature information through the global maximum pooling layer in the first target sub-network to obtain feature compression information;

[0136] Step S503, predicting the target three-primary color output values ​​corresponding to the to-be-parsed watermark image according to the feature compression information through the fully connected layer in the first target subnetwork;

[0137] Step S504: generating the target background color image in the target feature image based on the target three primary color output values.

[0138] In the specific implementation, the first target subnetwork (BE-Net) consists of a convolutional layer, a global maximum pooling layer, and a fully connected layer. The convolutional layer can effectively capture the feature information of the input image, the global maximum pooling layer can compress the feature information extracted by the convolutional layer to reduce the computational burden, and the fully connected layer can predict the RGB value of the output image, and finally generate a background color map based on the RGB value. Specifically, the global maximum pooling layer in the BE-Net network reduces the spatial dimension of the feature map extracted by the convolutional layer to 1x1, while retaining the maximum value in each feature map, achieving pixel classification of the input image and then upsampling to obtain a background color image.

[0139] See also Figure 6 , Figure 6 is a flowchart of steps for extracting a target attention thermal image provided by an embodiment of the present application; in some embodiments, step S103 may also include but is not limited to steps S601 to S604:

[0140] Step S601, inputting the watermark image to be parsed into the first target sub-network of the target parsing model, and extracting image feature information of the watermark image to be parsed through the convolutional layer in the first target sub-network;

[0141] Step S602, constructing a feature map based on the image feature information;

[0142] Step S603, adjusting the image size of the feature map to the image size of the watermark image to be analyzed, to obtain a target size feature map;

[0143] Step S604, superimposing the target size feature map onto the watermark image to be analyzed according to the target image processing function to obtain the target attention thermal image in the target feature image.

[0144] For the attention feature map (attention thermal image), the attention feature map in the embodiment of the present application uses the feature map output by the intermediate convolutional layer of the BE-Net network. The process of obtaining the attention feature map is as follows: the image size of the feature map output by the convolutional layer is adjusted to the size of the input image (i.e., the watermarked image), and the resized feature map is superimposed on the watermarked image through the function in the cv2 library (image processing library) to obtain the attention feature map.

[0145] In step S103 shown in the embodiment of the present application, feature extraction is performed on the watermark image corresponding to the watermark file through the first target sub-network of the target parsing model, so that the background color image and attention thermal image of the watermark image can be quickly and accurately extracted, thereby providing key information of the image for subsequent image analysis.

[0146] Step S104, performing feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector;

[0147] Step S104 shown in the embodiment of the present application can enrich the content of the feature vector by combining feature information from different images, thereby improving the accuracy of image analysis.

[0148] Step S105, inputting the target feature vector and the watermark image to be analyzed into the second target subnetwork of the target analysis model for image analysis processing to generate a target watermark-free image.

[0149] In some embodiments, step S105 may include: inputting the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model, performing image parsing processing on the target feature vector and the watermark image to be parsed by the generator in the second target subnetwork, and generating a target watermark-free image.

[0150] Step S105 shown in the embodiment of the present application, by inputting the target feature vector and the watermark image to be parsed into the generator, can utilize the powerful learning ability of the deep learning model to efficiently remove the watermark in the image. At the same time, the generator has an efficient processing speed and can support batch processing of multiple images to meet the real-time requirements of large-scale applications.

[0151] In the specific implementation, in order to improve the accuracy and efficiency of parsing watermarked PDF documents, the embodiments of the present application combine technologies such as image processing, text analysis and machine learning. By converting watermarked PDF documents into digital images, the image PDF is analyzed and understood, and features are extracted. Different types of content such as text, pictures, tables, formulas, etc. are processed accordingly. The purpose is to accurately parse and identify the semantic content of PDFs with complex watermarks, thereby providing users with a more reliable and convenient watermark parsing solution.

[0152] Steps S101 to S105 shown in the embodiment of the present application are as follows: obtaining a watermark file to be parsed; performing format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed; inputting the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; performing feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; inputting the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image. The embodiment of the present application performs feature extraction on the watermark image corresponding to the watermark file through the first target sub-network of the target parsing model, and can quickly and accurately extract the background color image and attention thermal image of the watermark image, providing key information of the image for subsequent image analysis. Then, the second target sub-network of the target parsing model performs image parsing processing on the target feature vector composed of the background color image and the attention thermal image and the watermark image to be parsed, and can efficiently generate a high-quality target watermark-free image. That is, the embodiment of the present application decomposes the image parsing task into two stages, feature extraction and image parsing, through the constructed target parsing model, and can efficiently handle complex large-scale image watermark removal tasks, reduce a large amount of pixel calculations, and improve the processing efficiency, accuracy and reliability of watermark image parsing, so that the content of the watermark file can be quickly and accurately identified.

[0153] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0154] See also Figure 7 , Figure 7 Schematic diagram of a file parsing system architecture provided by an embodiment of the present application; Figure 7As shown, the embodiment of the present application adopts a deep layered architecture design, with clear responsibilities between each level, reducing the coupling degree of the system and improving the reusability of the code. This allows developers to focus more on the specific implementation of each level without paying too much attention to other levels. The system architecture is mainly divided into six parts: the front-end user interface (User Interface, referred to as UI), the display layer, the business logic layer, the algorithm layer, the database layer, and the operating environment layer. These levels each have specific functions and responsibilities, and together constitute the core architecture of the entire system. Figure 7 The system architecture design and the collaborative relationship between each layer are clearly explained, among which the BEWDSR-Net model constructed by this application is applied to the algorithm layer. Figure 7 As shown, the architecture of each layer of the file parsing system architecture is explained as follows:

[0155] (1) Front-end user interface (UI) layer: Based on the progressive framework Vue, the ElementUI component library is used to create a dynamic and interactive user interface and interact with the back-end data.

[0156] (2) Presentation layer: responsible for page rendering, receiving data from the front-end user interface layer, and presenting it to the user in an appropriate manner. The presentation layer handles HTML rendering.

[0157] (3) Business logic layer: responsible for processing user requests transmitted by the presentation layer, and logically processing ordinary digital PDFs and watermarked image PDFs separately. For digital PDFs, directly parse the detailed information of the PDF object and extract the text content and table information. You can use the PyPDF2 library, PDFMiner, and six libraries in Python for parsing, which is more efficient than extracting using OCR (Optical Character Recognition) technology; for image PDFs, the business logic layer implements image preprocessing before inputting the algorithm layer. Specifically: first convert the page number of the image PDF into an image array of the same length, and input the image into the algorithm layer in the form of batching. After the algorithm layer obtains the output result, it will return the output result to the business logic layer. The business logic layer then processes the output result obtained by the algorithm layer into the business file required by the user, and then performs specific business rule verification and business operations, such as PDF merging or splitting, logging, permission verification, and other tasks.

[0158] (4) Algorithm layer: It is the core part of the system, which includes the implementation of various algorithms and data processing logic, and is used to handle complex problems that the business layer cannot handle. The BEWDSR-Net model is applied to the algorithm layer to parse PDF images with watermarks, and finally returns the parsing results to the business layer. The algorithm layer schedules different algorithm models to perform different tasks based on the data transmitted by the business logic layer, and then returns the results to the business logic layer. For example, the algorithm capabilities of text detection and text recognition involved in the embodiments of the present application: the algorithm layer calls the text detection model to obtain the coordinate position of the text line in the figure, and then calls the text recognition model to obtain the text line string, and returns the text string to the business logic layer to provide corresponding computing support for the business logic layer.

[0159] (5) Database layer: The relational database MySQL and the non-relational database Redis are used. MySQL is responsible for persistent storage and management of data, and provides operations such as user data query, update, and deletion; Redis is responsible for cooperating with MySQL for high-speed caching, and its specific data structure can support the system to complete registration, statistics, comments and other functions.

[0160] (6) Operation environment layer: It provides various environment configurations required for system operation, including configuration and management of infrastructure such as servers, operating systems, and network environments. The operation environment layer ensures that the system can operate stably and efficiently, and provides security and scalability guarantees. The operation environment layer is also responsible for monitoring the performance and resource usage of the system, and optimizing and adjusting it in a timely manner. The parsing system of the embodiment of the present application can be operated on a cloud server ( Figure 7 The parsing system of the embodiment of the present application can also provide a deployment and operation solution on a third-party virtual host.

[0161] It is easy to understand that the embodiment of the present application constructs a method and system for parsing a PDF document with a watermark, which implements the parsing and data management method of a PDF document and can be used to process large-scale PDF documents and manage systematic cases.

[0162] See also Figure 8 , Figure 8 is a flow chart of an image analysis method provided in an embodiment of the present application; Figure 8As shown, the process of the image parsing method is divided into two stages: the first stage is the processing stage of BE-Net, specifically: first, the watermarked image is input into the BEWDSR-Net model, and then the convolution layer of BE-Net in the BEWDSR-Net model is used to extract the features of the watermarked image to obtain a feature map, and then the spatial dimension of the feature map extracted by the convolution layer is reduced to 1x1 by the global maximum pooling layer of BE-Net, while retaining the maximum value in each feature map, and the RGB value of the output image is predicted by the fully connected layer of BE-Net, and the pixel classification of the input image is realized and then up-sampled to obtain a background color image; for the acquisition of the attention thermal image, the attention feature map of the embodiment of the present application uses the feature map output by the intermediate convolution layer of the BE-Net network, and the acquisition process of the attention feature map is: the image size of the feature map output by the convolution layer is adjusted to the size of the input image (i.e., the watermarked image), and the resized feature map is superimposed on the watermarked image through the function in the cv2 library (image processing library) to obtain the attention feature map, where w1...w n is the weight value, and each convolution kernel contains a set of weights (w1-wn), which perform dot product operations (i.e. weighted summation) with the local area of ​​the input feature map to generate a pixel value of the output feature map; when adjusting the size of the feature map, the interpolation functions in the image processing library (such as OpenCV, i.e. cv2) are usually used. These functions calculate the interpolation points based on the feature dimension and target size of the feature map to generate a new feature map with the required size. The second stage is the processing stage of WDR-Net, which is as follows: after BE-Net obtains the background color image and the attention thermal image, the background color image, the attention thermal image and the watermarked image are input into WDR-Net, and the generator and discriminator in WDR-Net work together to generate a watermark-free image.

[0163] Among them, the embodiment of the present application uses NVIDIA tritonserver module to load the BEWDSR-Net model and utilizes its GPU (Graphics Processing Unit) performance optimization advantages to accelerate the BEWDSR-Net model. Compared with the traditional digital image processing method, the embodiment of the present application greatly improves the efficiency of text extraction in watermarked PDF documents and provides more possibilities for actual production deployment.

[0164] It should be pointed out that this embodiment only briefly illustrates the general process of the image analysis method. The detailed description of each step can refer to the relevant content in the aforementioned embodiment and will not be repeated here. It can be understood that the present invention is not limited to this.

[0165] The embodiment of the present application obtains a watermark file to be parsed; performs format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed; inputs the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; performs feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; inputs the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image. The embodiment of the present application performs feature extraction on the watermark image corresponding to the watermark file through the first target sub-network of the target parsing model, and can quickly and accurately extract the background color image and attention thermal image of the watermark image, providing key information of the image for subsequent image analysis. Then, the second target sub-network of the target parsing model performs image parsing processing on the target feature vector composed of the background color image and the attention thermal image and the watermark image to be parsed, and can efficiently generate a high-quality target watermark-free image. That is, the embodiment of the present application decomposes the image parsing task into two stages, feature extraction and image parsing, through the constructed target parsing model, and can efficiently handle complex large-scale image watermark removal tasks, reduce a large amount of pixel calculations, and improve the processing efficiency, accuracy and reliability of watermark image parsing, so that the content of the watermark file can be quickly and accurately identified.

[0166] In summary, the embodiment of the present application designs the BEWDSR-Net model in combination with the generative adversarial network. The BEWDSR-Net model consists of BE-Net and WDR-Net. BE-Net is used to extract the background color map and the attention heat map with rich watermark information from the watermarked image, and combine the background color map and the attention heat map into a feature vector. WDR-Net is used to connect the feature vectors output by BE-Net, combined with the original watermarked image, to generate the final watermark-free image. Through the design of the BEWDSR-Net model network, the distance of the image watermark information in the image pixel space can be expanded, thereby improving the accuracy of the extracted text results.

[0167] In addition, related methods for parsing watermarked PDF documents all use some hand-crafted inspirations and traditional image processing methods to find the edge features of the watermark in the document image. These algorithms are usually only effective for some document images and have average accuracy. In addition, traditional image processing methods often require a large number of image pixel calculations, so the performance is also average. The embodiment of the present application provides a method and application system for parsing watermarked PDF documents, which can process large-scale PDF documents and realize systematic management. It can solve the problem that PDF watermarks are difficult to parse efficiently and accurately.

[0168] In addition, the application prospects of the embodiments of the present application are broad, and can be applied to electronic document management, copyright protection, legal evidence collection, finance and accounting, scientific research, and personalized needs. By parsing the watermarked PDF document, document resources can be better managed and utilized to meet various practical application needs. The image parsing method provided by the embodiments of the present application has higher parsing accuracy and processing efficiency, providing users with a more reliable and convenient parsing solution for watermarked PDFs.

[0169] See also Fig. 9 The present application also provides an image analysis device 900, which can implement the above-mentioned image analysis method. The device includes the following modules:

[0170] The watermark file acquisition module 901 is used to acquire the watermark file to be parsed;

[0171] The watermark image acquisition module 902 is used to perform format conversion processing on the watermark file to be parsed to obtain the watermark image to be parsed;

[0172] The feature extraction processing module 903 is used to input the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image;

[0173] A feature combination processing module 904 is used to perform feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector;

[0174] The image analysis processing module 905 is used to input the target feature vector and the watermark image to be analyzed into the second target sub-network of the target analysis model to perform image analysis processing to generate a target watermark-free image.

[0175] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0176] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned image analysis method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0177] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0178] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0179] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0180] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the image analysis method of the embodiment of this application;

[0181] Input / output interface 1003, used to implement information input and output;

[0182] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0183] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0184] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0185] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the computer program implements the above-mentioned image analysis method when executed by a processor.

[0186] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0187] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0188] The image parsing method, image parsing device, electronic device and storage medium provided by the embodiments of the present application obtain a watermark file to be parsed; perform format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed; input the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; perform feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; input the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image. The embodiment of the present application performs feature extraction on the watermark image corresponding to the watermark file through the first target sub-network of the target parsing model, and can quickly and accurately extract the background color image and attention thermal image of the watermark image, providing key information of the image for subsequent image analysis. Then, the second target sub-network of the target parsing model performs image parsing processing on the target feature vector composed of the background color image and the attention thermal image and the watermark image to be parsed, and can efficiently generate a high-quality target watermark-free image. That is, the embodiment of the present application decomposes the image parsing task into two stages, feature extraction and image parsing, through the constructed target parsing model, and can efficiently handle complex large-scale image watermark removal tasks, reduce a large amount of pixel calculations, and improve the processing efficiency, accuracy and reliability of watermark image parsing, so that the content of the watermark file can be quickly and accurately identified.

[0189] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0190] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0191] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0192] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0193] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0194] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0195] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0196] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0197] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0198] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0199] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An image analysis method, characterized in that: The method comprises the following steps: Get the watermark file to be parsed; Performing format conversion processing on the watermark file to be parsed to obtain a watermark image to be parsed; Input the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; Performing feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; The target feature vector and the watermark image to be parsed are input into the second target subnetwork of the target parsing model for image parsing processing to generate a target watermark-free image.

2. The method according to claim 1, characterized in that The step of inputting the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image comprises: Inputting the to-be-parsed watermark image into the first target sub-network of the target parsing model, and extracting image feature information of the to-be-parsed watermark image through the convolutional layer in the first target sub-network; Compressing the image feature information through the global maximum pooling layer in the first target sub-network to obtain feature compression information; Predicting the target three-primary color output values ​​corresponding to the watermark image to be parsed according to the feature compression information through the fully connected layer in the first target subnetwork; The target background color image in the target feature image is generated based on the target three-primary color output values.

3. The method according to claim 1, characterized in that The step of inputting the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image comprises: Inputting the to-be-parsed watermark image into the first target sub-network of the target parsing model, and extracting image feature information of the to-be-parsed watermark image through the convolutional layer in the first target sub-network; Constructing a feature map based on the image feature information; Adjusting the image size of the feature map to the image size of the watermark image to be analyzed to obtain a target size feature map; The target size feature map is superimposed on the watermark image to be analyzed according to the target image processing function to obtain the target attention thermal image in the target feature image.

4. The method according to claim 1, characterized in that: The step of inputting the target feature vector and the watermark image to be parsed into the second target subnetwork of the target parsing model to perform image parsing processing to generate a target watermark-free image comprises: The target feature vector and the watermark image to be parsed are input into the second target sub-network of the target parsing model, and the generator in the second target sub-network performs image parsing processing on the target feature vector and the watermark image to be parsed to generate the target watermark-free image.

5. The method according to claim 1, characterized in that The method further comprises the step of constructing the first target sub-network, wherein the step of constructing the first target sub-network comprises: Acquire a dataset of images to be trained; the dataset of images to be trained includes watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained; Performing cluster analysis on the real watermark-free image through a Gaussian mixture model to obtain watermark-free content information and watermark-free background information corresponding to the real watermark-free image, and constructing a background color reference image according to the watermark-free background information; Input the watermark image to be trained into the first initial sub-network for feature extraction processing to obtain a first feature image to be trained; the first feature image to be trained includes a first background color image to be trained and a first attention thermal image to be trained; Calculating a background color deviation value between the first background color image to be trained and the background color reference image; Constructing a first loss function of the first initial sub-network based on the background color deviation value; The first initial sub-network is trained based on the first loss function to obtain the first target sub-network.

6. The method according to claim 1, characterized in that The method further comprises the step of constructing a second target sub-network, wherein the step of constructing the second target sub-network comprises: Acquire a dataset of images to be trained; the dataset of images to be trained includes watermarked images to be trained and real watermark-free images corresponding to the watermarked images to be trained; Inputting the watermark image to be trained into the first target subnetwork for feature extraction processing to obtain a second feature image to be trained; the second feature image to be trained includes a second background color image to be trained and a second attention thermal image to be trained; Performing feature combination processing on the second background color image to be trained and the second attention thermal image to be trained to obtain a feature vector to be trained; Inputting the feature vector to be trained and the watermark image to be trained into a second initial sub-network, performing image parsing processing through a generator in the second initial sub-network, and obtaining a candidate watermark-free image; Performing discriminative processing on the candidate watermark-free image by the discriminator in the second initial sub-network to obtain a discriminative processing result; Calculate the image quality deviation between the candidate watermark-free image and the real watermark-free image according to the discrimination processing result to obtain an image quality evaluation result; Constructing a generative adversarial network loss function of the second initial sub-network based on the image quality assessment result; The generator and the discriminator are optimized based on the generative adversarial network loss function, and the step of inputting the feature vector to be trained and the watermark image to be trained into the second initial sub-network and performing image parsing processing through the generator in the second initial sub-network to obtain a candidate watermark-free image is returned to be executed, until the image quality deviation value meets the preset image quality deviation threshold, and the second target sub-network is obtained.

7. The method according to claim 1, characterized in that The method further comprises: Obtain the captured file image to be parsed; Performing color correction processing on the photographed document image to obtain a color-corrected image; Affine transformation is performed on the color-corrected image to obtain a target corrected image, and the target corrected image is used as the watermark image to be parsed.

8. An image analysis device, characterized in that: The device comprises the following modules: A watermark file acquisition module is used to obtain the watermark file to be parsed; A watermark image acquisition module is used to perform format conversion processing on the watermark file to be parsed to obtain the watermark image to be parsed; A feature extraction processing module is used to input the watermark image to be parsed into the first target subnetwork of the target parsing model for feature extraction processing to obtain a target feature image; the target feature image includes a target background color image and a target attention thermal image; A feature combination processing module is used to perform feature combination processing on the target background color image and the target attention thermal image to obtain a target feature vector; The image analysis processing module is used to input the target feature vector and the watermark image to be analyzed into the second target sub-network of the target analysis model to perform image analysis processing to generate a target watermark-free image.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images