Model training method, watermark text recognition method and related device
By generating a set of watermarked images and constructing a bidirectional recurrent neural network model, the problem of difficult identification of pixelated watermarks was solved, and accurate identification and tracing of pixelated watermarked text was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING VOLCANO ENGINE TECH CO LTD
- Filing Date
- 2022-06-23
- Publication Date
- 2026-06-26
AI Technical Summary
In existing technologies, pixelated watermarks are difficult to identify, especially in the case of background interference and light colors, making image tracing difficult.
By generating a set of watermarked images and performing pixelation processing, a bidirectional recurrent neural network model is constructed. The model is trained using a training dataset to recognize pixelated watermarked text, and a heuristic search algorithm is used to remove blank and overlapping parts.
It improves the recognition accuracy of watermark text in pixelated watermark images, avoids the problem of being unable to trace the source after pixelation, and achieves effective recognition of clear watermarks.
Smart Images
Figure CN117333879B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a model training method, a watermark text recognition method, and related equipment. Background Technology
[0002] In existing technologies, adding watermarks to various digital images allows for quick identification of the image's source information, such as copyright holder details. However, to avoid tracing the image's origin, the watermarks may be pixelated, making them difficult to recognize.
[0003] In order to accurately trace the source of an image, it is necessary to perform watermark text recognition on the pixelated watermark. However, because the original watermark text is light in color and easily affected by the background of the image, it is not easy to restore the original watermark text in the image. Summary of the Invention
[0004] In view of this, the purpose of this application is to propose a model training method, a watermark text recognition method, and related equipment to solve or partially solve the above-mentioned technical problems.
[0005] To achieve the above objectives, the first aspect of this application provides a method for training a watermark restoration model, comprising:
[0006] Obtain watermark style information and background style information; the watermark style information is used to indicate the style of the watermark character content; the background style information is used to indicate the style of the background image content.
[0007] A watermark image set is generated by combining the watermark style information and the background style information, and the watermark image set includes multiple images with visible watermarks;
[0008] Each watermark image in the watermark image set is pixelated and the pixel values in the pixel block are extracted as training samples. The clear watermark corresponding to the watermark image is used as the sample label. The training samples and their corresponding sample labels are combined to generate a training dataset.
[0009] A bidirectional recurrent neural network model is constructed, and the training dataset is used to train the bidirectional recurrent neural network model to obtain a bidirectional trained neural network model that meets the training termination condition, which is used as a watermark restoration model. The watermark restoration model is used to restore the visible watermark characters in the image.
[0010] Based on the same inventive concept, a second aspect of this application proposes a watermark text recognition method, including:
[0011] Acquire the target image and extract the color values from the pixel blocks of the target image;
[0012] The color values are processed by a pre-trained watermark restoration model to obtain the watermark restoration image corresponding to the target image. The watermark restoration model is trained and generated based on a set of watermark images synthesized from predefined watermark style information and background style information. The watermark restoration model takes the color values of the pixel blocks corresponding to the image as the input of the model and takes the image restored by pixelation as the output.
[0013] The watermark restoration result corresponding to the target image is obtained by removing the blank and / or overlapping parts in the watermark restoration image corresponding to the target image through a heuristic search algorithm.
[0014] Based on the same inventive concept, a third aspect of this application proposes a training device for a watermark restoration model, comprising:
[0015] The watermark and background acquisition module is configured to acquire watermark style information and background style information; the watermark style information is used to indicate the style of the watermark character content; the background style information is used to indicate the style of the background image content.
[0016] The watermark image generation module is configured to generate a watermark image set based on the watermark style information and the background style information, wherein the watermark image set includes multiple images with visible watermarks.
[0017] The pixelation processing module is configured to perform pixelation processing on each watermark image in the watermark image set and extract the pixel values in the pixel blocks as training samples, and use the clear watermark corresponding to the watermark image as the sample label, and combine each training sample and its corresponding sample label to generate a training dataset.
[0018] The training module is configured to construct a bidirectional recurrent neural network model and call the training dataset to train the bidirectional recurrent neural network model to obtain a bidirectional trained neural network model that meets the training termination condition as a watermark restoration model. The watermark restoration model is used to restore the visible watermark characters in the image.
[0019] Based on the same inventive concept, a fourth aspect of this application proposes a watermarked text recognition device, comprising:
[0020] The color value extraction module is configured to acquire a target image and extract color values from pixel blocks in the target image.
[0021] The text restoration module is configured to call a pre-trained watermark restoration model to process the color value to obtain the watermark restoration image corresponding to the target image; the watermark restoration model is trained and generated based on a set of watermark images synthesized from predefined watermark style information and background style information, and the watermark restoration model takes the color value of the pixel block corresponding to the image as the input of the model and takes the image restored by pixelation watermark as the output.
[0022] The filtering module is configured to remove blank and / or overlapping portions in the watermark restoration image corresponding to the target image using a heuristic search algorithm to obtain the watermark restoration result corresponding to the target image.
[0023] Based on the same inventive concept, the fifth aspect of this application proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of the first aspect or the second aspect.
[0024] Based on the same inventive concept, the sixth aspect of this application proposes a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the method of the first aspect or the second aspect.
[0025] As can be seen from the above, the training method, watermark text recognition method, and related equipment provided in this application can synthesize watermark images using pre-set watermark and background styles. This automatic generation method is fast, has low generation cost, and the number of generated images is relatively arbitrary. The generated watermark images are basically indistinguishable from the collected watermark images. These watermark images are pixelated, and the watermark text content is associated with the pixelated watermark images to form a training dataset. The training dataset is automatically synthesized in this way. The constructed bidirectional recurrent neural network is trained using this training dataset. During the training process, the accuracy of the bidirectional recurrent neural network in recognizing watermark text in pixelated watermark images is continuously improved. After training, the watermark restoration model can accurately recognize watermark text in pixelated watermark images, thus avoiding the problem of being unable to trace the source of watermark images after pixelation. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this application;
[0028] Figure 2 This is a flowchart illustrating the training method of the watermark restoration model according to an embodiment of this application;
[0029] Figure 3 This is a flowchart of the watermark text recognition method according to an embodiment of this application;
[0030] Figure 4 This is a structural block diagram of the training device for the watermark restoration model according to an embodiment of this application.
[0031] Figure 5 This is a structural block diagram of the watermark text recognition device according to an embodiment of this application;
[0032] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0033] The principles and spirit of this application will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this application, and are not intended to limit the scope of this application in any way. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.
[0034] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.
[0035] Based on the above background description, the following situations also exist in the related technologies:
[0036] In related technologies, the recognition of pixelated text is usually done on text in normal images. However, watermarked text is often tilted and light in color, which makes it relatively difficult to recover pixelated watermarks.
[0037] Furthermore, pixelated text recognition in related technologies often requires the background of the pixelated text to be white or solid color, without taking into account the presence of textures or other interference in the background (e.g., dark watermarks). In particular, when the watermark itself is shallow, the impact of such interference will be amplified.
[0038] Pixelated text recognition technology has very strict requirements on the font size of text in images and the size of pixel grids in pixelated regions, which cannot adapt to real-world scenarios with multiple relative sizes.
[0039] Based on the above description, the principles and spirit of this application will be explained in detail below with reference to several representative embodiments.
[0040] refer to Figure 1 This diagram illustrates an application scenario of the training method for the watermark restoration model or the watermark text recognition method provided in this application embodiment. The application scenario includes a terminal device 101, a server 102, and a data storage system 103. The terminal device 101, server 102, and data storage system 103 can all be connected via wired or wireless communication networks. The terminal device 101 includes, but is not limited to, desktop computers, mobile phones, mobile computers, tablets, media players, smart wearable devices, personal digital assistants (PDAs), or other electronic devices capable of performing the aforementioned functions. The server 102 and data storage system 103 can both be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.
[0041] Server 102 can automatically generate several pixelated watermark images and associate them with corresponding watermark text content as labels, which then serve as training data. Multiple training datasets are combined to form the training dataset. Server 102 constructs a bidirectional recurrent neural network, sequentially inputting the training data from the training dataset into the network for training, resulting in a watermark restoration model capable of recognizing watermark text from pixelated watermark images. Then, server 102 receives a pixelated target image from terminal device 101, processes it using the watermark restoration model, obtains one or more watermark text results, and sends these results back to terminal device 101. Data storage system 103 provides data storage support for the operation of server 102, for example, storing program code that implements the above processes.
[0042] The following is combined with Figure 1 The application scenarios described above illustrate the training method for the watermark restoration model and the watermark text recognition method according to exemplary embodiments of this application. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect. Rather, the embodiments of this application can be applied to any applicable scenario.
[0043] Watermarks in watermarked images can be visible or hidden. Visible watermarks are those that are directly visible to the naked eye, while hidden watermarks require technical processing to become visible. This application primarily addresses the pixelation problem in visible watermark images.
[0044] This application provides a method for training a watermark restoration model, applied to a server. For example... Figure 2 As shown, the training methods include:
[0045] Step 201: Obtain watermark style information and background style information; the watermark style information is used to indicate the style of the watermark character content; the background style information is used to indicate the style of the background image content.
[0046] In practice, the pixelated watermark images can be those from a stock image library, where the watermark text content is known. However, obtaining these pixelated watermark images from the library is relatively complex, and the resulting images are inconsistent and difficult to train. To avoid this, this application chooses to automatically synthesize these pixelated watermark images.
[0047] In some embodiments, step 201 includes:
[0048] The system displays a watermark style information configuration interface and a background style information configuration interface; the watermark style information configuration interface is used to configure at least one of the following: watermark content, watermark color, watermark font, watermark size, and watermark tilt; the background style information interface is used to configure the background color, dark watermark, or texture.
[0049] Receive watermark style information configured by the user through the watermark style information configuration interface, and receive background style information configured by the user through the background style information configuration interface.
[0050] Step 202: Generate a watermark image set by combining the watermark style information and the background style information. The watermark image set includes multiple images with visible watermarks.
[0051] In practice, a watermarked image refers to an image containing watermark content. An image may contain one or multiple watermarks. These watermarked images can be pre-obtained from a library or automatically synthesized. For watermarked images pre-obtained from a library, the watermark text content is determined before subsequent pixelation processing. For automatically synthesized watermarked images, the watermark text content is known. The watermark text content is pre-assigned to each watermark image, for example, through key-value pairs, connectors, or table relationships.
[0052] Multiple background images are determined based on a set background style. The set background style can include: a set background color, background pattern, watermark, or texture, etc. The resulting multiple background images can be the same or different.
[0053] Multiple watermarked images are generated by adding watermarks to multiple background images according to a set watermark style. The set watermark style includes: watermark content, watermark color (RGBA), watermark font, watermark size, watermark tilt, etc. After adding some random factors within a certain range (such as randomly similar font sizes, randomly increasing or decreasing watermark transparency, randomly adjusting watermark tilt, and randomly adjusting watermark position), a batch of watermarked images (i.e., digital images with visible watermarks) is generated.
[0054] Compared to collecting watermarked images, this method of automatically synthesizing watermarked images is cheaper, faster, and allows for a greater number of images to be generated. It also avoids the privacy issues associated with collecting watermarked images, and the synthesized watermarked images are almost indistinguishable from the collected ones.
[0055] In some embodiments, before performing the following steps, a watermark adjustment strategy is invoked to randomly adjust the watermark images in the watermark image set to obtain an adjusted watermark image set. The watermark adjustment strategy is used to randomly extract watermark images and randomly adjust at least one dimension of watermark font size, watermark transparency, watermark tilt, and watermark position.
[0056] Step 203: Pixelate each watermark image in the watermark image set and extract the pixel values in the pixel blocks as training samples. Use the watermark corresponding to the watermark image as the sample label. Combine each training sample and its corresponding sample label to generate a training dataset.
[0057] In some embodiments, a preset pixelation strategy is invoked to perform pixelation processing on each watermark image in the watermark image set. The pixelation strategy is used to indicate at least one of the following: the size of the pixel grid, the shape and size of the pixelated region, and the position coordinates of the pixelated region.
[0058] In some embodiments, step 203 includes:
[0059] Step 2031: Pixelate multiple watermark images according to a predetermined pixelation rule to form pixelated regions in each watermark image. The pixelation rule includes at least one of the following: pixel grid size, pixelated region, and pixelated position.
[0060] In practice, users can pre-set pixelation rules, such as pixel size, pixelation area, and pixelation position. Alternatively, users can select one or more pre-stored pixelation rules for pixelation processing.
[0061] Step 2032: Adjust the range of the pixelated regions of the multiple watermark images to obtain multiple pixelated watermark images.
[0062] In practice, the watermark image is pixelated according to the pre-set pixelation rules. After pixelation, the size and position of the pixelated areas in the pixelated watermark image are randomly adjusted, or manually adjusted. This adjustment makes the pixelated areas have diverse characteristics, thus obtaining multiple pixelated watermark images.
[0063] In addition, the obtained multiple pixelated watermark images can be de-noiseed to remove identical pixelated watermark images, effectively improving the efficiency of subsequent training.
[0064] The above method can automatically synthesize pixelated watermark images, speeding up the acquisition rate of pixelated watermark images. This automated synthesis method does not infringe on the privacy of others, making pixelated watermark images more legitimate.
[0065] In practice, if the watermark image is captured directly, the watermark location is determined, and its text content is identified and extracted. If the watermark image is automatically synthesized using the above steps, the corresponding watermark text content is directly obtained. The watermark text content includes numbers, characters, symbols, etc.
[0066] In each training dataset, the pixelated watermark image and its corresponding label can be associated using key-value pairs, tables, or connectors. The specific association method can be set or selected according to actual needs.
[0067] Step 203 also includes:
[0068] Step 2033, the processing procedure for each pixelated watermark image is as follows: extract at least one pixel cell from the pixelated region of the pixelated watermark image, arrange the at least one pixel cell in a matrix according to its position in the pixelated watermark image to obtain a pixel cell matrix, wherein the pixelated region includes at least one pixel cell; extract the color value of each column of pixel cells in the pixel cell matrix as a time sequence, normalize the time sequence, and associate the watermark text content with the normalized time sequence to form training data.
[0069] In practice, the pixelated region is the same as the mosaic region. Each pixelated region contains multiple pixel cells, and the color value within each pixel cell is the same. Each pixelated region contains at least one column of pixel cells. The color value of each column of pixel cells is extracted, and these color values are arranged according to the corresponding pixel cells to form a time-series sequence. To facilitate the training of the time-series sequence, it needs to be normalized, and then associated with the corresponding watermark text to form the training data.
[0070] Step 2034: Integrate the training data obtained corresponding to each pixelated watermark image to form a training dataset.
[0071] In practice, each pixelated watermark image is processed according to the above scheme, thus obtaining training data with an equal number of pixelated watermark images. These are then integrated to form the training dataset.
[0072] If the amount of training data in the training dataset is insufficient, it is necessary to supplement the amount of training data by following the steps described above. If the amount of training data is too large, some training data can be randomly discarded.
[0073] Step 204: Construct a bidirectional recurrent neural network model and call the training dataset to train the bidirectional recurrent neural network model to obtain a bidirectional trained neural network model that meets the training termination condition as a watermark restoration model. The watermark restoration model is used to restore the visible watermark characters in the image.
[0074] In practice, the bidirectional recurrent neural network (BRNN) model includes an input layer, multiple hidden layers, and an output layer, possessing a certain degree of autonomous learning capability. The time-series data from the training data is input into the BRNN model for processing. The parameters of the BRNN model are adjusted according to the corresponding labels, thus completing the training of the BRNN model. Then, the next training data is used to train the BRNN model, continuously adjusting its parameters and thereby continuously improving its accuracy in recognizing watermarked text. After training, the final BRNN model is used as the watermark restoration model.
[0075] Among them, the bidirectional recurrent neural network (BiRNN) consists of a forward neural network and a backward neural network. Each layer of the two neural networks is interconnected, and the two networks are connected to an output layer.
[0076] The above training scheme combines bidirectional recurrent neural networks with joint mechanism time classification for training, resulting in a watermark restoration model with higher recognition accuracy.
[0077] In some embodiments, step 205 includes:
[0078] Step 2051: Construct a bidirectional recurrent neural network model;
[0079] Step 2052: Call the training dataset and train the bidirectional recurrent neural network model iteratively using the connection time-series classification algorithm to obtain a model that meets the training termination condition as the watermark restoration model.
[0080] In practice, the temporal sequence composed of pixel color values in each training data in the training dataset is used as the input of the bidirectional recurrent neural network for training. After bidirectional recurrent training is performed in the forward and backward neural networks of the bidirectional recurrent neural network using the Connectionist temporal classification (CTC) method, the forward output result and the backward output result are obtained.
[0081] The training result is obtained based on the forward and backward output results. The loss function is calculated based on the difference between the training result and the labels in the training data. The bidirectional recurrent neural network is adjusted by backpropagation based on the loss function to complete the training of the training data.
[0082] In practice, the loss function characterizes the difference between the training results and the corresponding labels. The more convergent the loss function, the smaller its value, indicating a smaller difference and higher recognition accuracy of the bidirectional recurrent neural network. Based on the convergence of the loss function, the parameters between neurons in each layer of the bidirectional recurrent neural network are adjusted according to the principle of backpropagation, thereby completing the training of a training dataset.
[0083] In response to the determination that the trained bidirectional recurrent neural network meets the predetermined degree of convergence, or that all training data in the training dataset has been trained, the trained bidirectional recurrent neural network is used as the watermark restoration model.
[0084] In practice, the conditions for determining the completion of training of a bidirectional recurrent neural network can be:
[0085] (1) Determine the degree of convergence based on the obtained loss function. If the degree of convergence reaches the predetermined degree of convergence, it proves that the bidirectional recurrent neural network training is complete.
[0086] (2) A training dataset consisting of a predetermined number of training data can be obtained in advance. If the entire training dataset is trained, it proves that the bidirectional recurrent neural network training is complete.
[0087] If either or both of the above two conditions are met, the final bidirectional recurrent neural network will be used as the watermark restoration model capable of recognizing watermark text in the pixelated watermark image.
[0088] In some embodiments, step 205 further includes:
[0089] Step 205a: Determine the size of the pixel grid within the pixelated region of multiple training data in the training dataset, and classify the multiple training data according to the size, with each size corresponding to a category training dataset.
[0090] In practice, the training data can be divided into multiple categories according to the size of the pixel grid, and each category's training dataset can be ensured to have more than a predetermined number of training data, thereby ensuring that there is enough training data for that category in the future.
[0091] Step 205b: Using the training datasets of each category, the bidirectional recurrent neural network is trained using the joint mechanism time classification method to obtain multiple watermark restoration models, wherein each watermark restoration model corresponds one-to-one with the category training dataset.
[0092] In this step, the training datasets for each category are trained according to the process described above, resulting in a watermark restoration model for each category. This watermark restoration model can then perform watermark text recognition processing on pixelated watermark images of corresponding pixel grid sizes. This approach improves the accuracy and efficiency of watermark text recognition.
[0093] In some embodiments, the training method further includes:
[0094] Step 206: Add a heuristic search algorithm (Beam search) capable of blank removal to the output layer of the obtained watermark restoration model.
[0095] In practice, a heuristic search algorithm can be used to perform blank removal on the watermark text recognition results to obtain an output with the same length as the original watermark text.
[0096] The above-described embodiments enable the association of watermark text content with pixelated watermark images to form a training dataset. This training dataset is then used to train the constructed bidirectional recurrent neural network model. During the training process, the accuracy of the bidirectional recurrent neural network model in recognizing watermark text in pixelated watermark images is continuously improved. After training, the resulting watermark restoration model can accurately recognize watermark text in pixelated watermark images, thus avoiding the problem of being unable to trace the source of watermark images after they have been pixelated.
[0097] Based on the same inventive concept, this embodiment provides a watermarked text recognition method, applied to the server side. For example... Figure 3 As shown, the watermark text recognition method includes:
[0098] Step 301: Obtain the target image and extract the color values from the pixel blocks of the target image.
[0099] In some embodiments, step 301 includes:
[0100] Step 3011: Obtain the target image and determine whether the target image has been compressed during network transmission based on its source.
[0101] Step 3012: When it is determined that the target image has been compressed during network transmission, mean filtering is performed on the central part of the pixel block in the target image and then the color value is extracted.
[0102] Step 3013: When it is determined that the target image has not been compressed during network transmission, extract the color values of the pixel blocks in the target image.
[0103] In practice, users can send the pixelated target image to the server via the network using their terminal devices. Since the input layer of the watermark restoration model has multiple input ports, the server needs to preprocess the target image after receiving it, extracting image features as preprocessed data.
[0104] For target images that have not undergone application software compression, the color values (RGB values) of pixels within the pixelated region remain unchanged and can be extracted directly without further processing. The resulting time-series sequence is constructed from the color values of each column of pixels.
[0105] The color values of pixels within a pixelated region of a target image compressed by application software (e.g., compression software, instant messaging software, image forwarding software) may change, and the color value of each pixel may be different. To ensure that an accurate color value is extracted from each pixel, it is necessary to perform an averaging process on the color value of each pixel. The averaged color value is used as the color value of that pixel, and the color values of all pixels in the pixelated region are arranged into a time sequence.
[0106] The above method can distinguish between target images processed by the application software and those not processed by the application software. This way, even if the color values within the pixel grid are blurred or changed after the application software processing, it will not affect the accuracy of the extracted preprocessed data.
[0107] In the pixelated region of the target image after compression by application software, the color values of each pixel may change, and the edges of the pixel cells change more significantly, while the center of the pixel cells generally changes less. Therefore, the color value of the center region of the pixel cell is selected and averaged. The average result is used as the color value of the pixel cell. In this way, the color values of each pixel cell are integrated into a time series as preprocessed data.
[0108] In practice, users can send the pixelated target image to the server via the network using their terminal devices. Since the input layer of the watermark restoration model has multiple input ports, the server needs to preprocess the target image after receiving it, extracting image features as preprocessed data.
[0109] The target image is determined to be uncompressed by the application software. The color values of each column of pixels in the pixelated region of the target image are extracted as preprocessing data. Specifically, for target images that have not undergone application software compression, the color values (RGB values) of the pixels in the pixelated region remain unchanged and can be extracted directly without further processing. The obtained color values of each column of pixels are then used to construct a time-series sequence.
[0110] Step 302: Call the pre-trained watermark restoration model to process the color value to obtain the watermark restoration image corresponding to the target image; the watermark restoration model is trained and generated based on a set of watermark images synthesized from predefined watermark style information and background style information. The watermark restoration model takes the color value of the pixel block corresponding to the image as the input of the model and takes the image restored by pixelation watermark as the output.
[0111] In some embodiments, step 302 includes:
[0112] Step 3021: Obtain the number of candidates specified by the user.
[0113] Step 3022: Based on the number of candidates, call the watermark restoration model to process the color value to obtain the number of candidate watermark restoration images and their corresponding confidence scores.
[0114] In practice, users can set the number of output results of the watermark restoration model according to their needs (i.e., a predetermined number, for example, N). If the number of watermark text results obtained after the target image is processed by the watermark restoration model is greater than or equal to N, then the top N watermark text results are selected for output based on their confidence level (e.g., probability value) from largest to smallest. If the number of watermark text results is less than N, then the watermark text results obtained according to the sorting result are directly output.
[0115] The above-described embodiments enable accurate watermark text recognition processing of target images using watermark restoration models, effectively improving recognition accuracy.
[0116] Step 303: Remove blank and / or overlapping parts from the watermark restoration image corresponding to the target image using a heuristic search algorithm to obtain the watermark restoration result corresponding to the target image.
[0117] In practice, preprocessed data is input into the watermark restoration model. The model performs forward processing via a forward neural network to obtain a forward output; and backward processing via a backward neural network to obtain a backward output. The forward and backward outputs are then processed (e.g., averaging, combining, or weighting according to corresponding weights) to obtain one or more final watermarked text results. The watermark restoration model can determine the confidence level or probability value of these watermarked text results and output them after sorting them according to the confidence level or probability value. For example, the obtained watermarked text results might be "6999 - probability value 90%", "6899 - probability value 50%", and "6889 - probability value 46%".
[0118] In some embodiments, in step 303, the heuristic search algorithm (Beamsearch) in the output layer of the watermark restoration model is used to remove blank and / or overlapping parts in each watermark text result, and the output layer outputs each watermark text result after removal.
[0119] This allows the heuristic search algorithm to remove blank and / or overlapping parts from the watermarked text recognition results, resulting in an output with the same length as the original watermarked text, thus ensuring the watermarked text restoration effect.
[0120] In some embodiments, the watermark text recognition method of this embodiment further includes:
[0121] In response to determining multiple watermark restoration models, where each watermark restoration model corresponds to a target image with a pixel grid size.
[0122] Obtain the size of each pixel in the pixelated region of the target image, and select the target watermark restoration model from multiple watermark restoration models based on the size.
[0123] Extract the color values from the pixel blocks of the target image, and input the obtained color values into the target watermark restoration model for watermark recognition processing (the specific watermark recognition is as described above).
[0124] In practical implementation, following the training method described in the above embodiments, different watermark restoration models capable of handling different pixel grid sizes may be obtained. This requires first determining the size of the pixel grid in the target image, then selecting the target watermark restoration model corresponding to that size, and using this target watermark restoration model to perform watermark recognition processing according to the watermark recognition process described above. If the target image has multiple pixel grid sizes, the pixelated region is divided according to the pixel grid size. Based on the division results, multiple corresponding target watermark restoration models are selected for processing, and the watermark text results from each target watermark restoration model are integrated to output the final watermark text result.
[0125] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method.
[0126] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0127] Based on the same inventive concept, corresponding to the training method of the watermark restoration model in any of the above embodiments, this application also provides a training device for the watermark restoration model.
[0128] refer to Figure 4 The training device includes:
[0129] The watermark and background acquisition module is configured to acquire watermark style information and background style information; the watermark style information is used to indicate the style of the watermark character content; the background style information is used to indicate the style of the background image content.
[0130] The watermark image generation module is configured to generate a watermark image set based on the watermark style information and the background style information, wherein the watermark image set includes multiple images with visible watermarks.
[0131] The pixelation processing module is configured to perform pixelation processing on each watermark image in the watermark image set and extract the pixel values in the pixel blocks as training samples, and use the clear watermark corresponding to the watermark image as the sample label, and combine each training sample and its corresponding sample label to generate a training dataset.
[0132] The training module is configured to construct a bidirectional recurrent neural network model and call the training dataset to train the bidirectional recurrent neural network model to obtain a bidirectional trained neural network model that meets the training termination condition as a watermark restoration model. The watermark restoration model is used to restore the visible watermark characters in the image.
[0133] In some embodiments, the watermark and background acquisition module is further configured to:
[0134] Displays a watermark style information configuration interface and a background style information configuration interface; the watermark style information configuration interface is used to configure at least one of the following: watermark content, watermark color, watermark font, watermark size, and watermark tilt; the background style information interface is used to configure background color, dark watermark, or texture; receives watermark style information configured by the user through the watermark style information configuration interface, and receives background style information configured by the user through the background style information configuration interface.
[0135] In some embodiments, the watermark image generation module is further configured to:
[0136] The watermark adjustment strategy is invoked to randomly adjust the watermark images in the watermark image set to obtain the adjusted watermark image set. The watermark adjustment strategy is used to randomly extract watermark images and randomly adjust at least one dimension of watermark font size, watermark transparency, watermark tilt and watermark position.
[0137] In some embodiments, the training module is further configured to:
[0138] Construct a bidirectional recurrent neural network model; call the training dataset and train the bidirectional recurrent neural network model iteratively using a connection-time classification algorithm to obtain a model that meets the training termination condition, which is then used as the watermark restoration model.
[0139] In some embodiments, the pixelation processing module is configured to:
[0140] A preset pixelation strategy is invoked to perform pixelation processing on each watermark image in the watermark image set. The pixelation strategy is used to indicate at least one of the following: the size of the pixel grid, the shape and size of the pixelated area, and the position coordinates of the pixelated area.
[0141] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.
[0142] The apparatus of the above embodiments is used to implement the corresponding training method in any of the foregoing embodiments, and has the beneficial effects of the corresponding training method embodiments, which will not be repeated here.
[0143] Based on the same inventive concept, corresponding to the watermark text recognition method of any of the above embodiments, this application also provides a watermark text recognition device.
[0144] refer to Figure 5 The watermark text recognition device includes:
[0145] The color value extraction module is configured to acquire a target image and extract color values from pixel blocks in the target image.
[0146] The text restoration module is configured to call a pre-trained watermark restoration model to process the color value to obtain the watermark restoration image corresponding to the target image; the watermark restoration model is trained and generated based on a set of watermark images synthesized from predefined watermark style information and background style information, and the watermark restoration model takes the color value of the pixel block corresponding to the image as the input of the model and takes the image restored by pixelation watermark as the output.
[0147] The filtering module is configured to remove blank and / or overlapping portions in the watermark restoration image corresponding to the target image using a heuristic search algorithm to obtain the watermark restoration result corresponding to the target image.
[0148] In some embodiments, the text restoration module is further configured to:
[0149] Obtain the number of candidates specified by the user; based on the number of candidates, call the watermark restoration model to process the color value to obtain the number of candidate watermark restoration images and their corresponding confidence scores.
[0150] In some embodiments, the color value extraction module is further configured to:
[0151] Acquire the target image, and determine whether the target image has been compressed during network transmission based on its source;
[0152] When it is determined that the target image has been compressed during network transmission, mean filtering is performed on the central part of the pixel block in the target image and then the color value is extracted.
[0153] When it is determined that the target image has not been compressed during network transmission, the color values of the pixel blocks in the target image are extracted.
[0154] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.
[0155] The apparatus described above is used to implement the corresponding watermark text recognition method in any of the foregoing embodiments, and has the beneficial effects of the corresponding watermark text recognition method embodiments, which will not be repeated here.
[0156] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any of the above embodiments.
[0157] Figure 6This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 610, a memory 620, an input / output interface 630, a communication interface 640, and a bus 650. The processor 610, memory 620, input / output interface 630, and communication interface 640 are interconnected internally via the bus 650.
[0158] The processor 610 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0159] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.
[0160] The input / output interface 630 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.
[0161] The communication interface 640 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (e.g., USB, Ethernet cable) or wireless means (e.g., mobile network, Wi-Fi, Bluetooth).
[0162] Bus 650 includes a pathway for transmitting information between various components of the device, such as processor 610, memory 620, input / output interface 630, and communication interface 640.
[0163] It should be noted that although the above-described device only shows the processor 610, memory 620, input / output interface 630, communication interface 640, and bus 650, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.
[0164] The electronic devices described above are used to implement the corresponding resource allocation method or text correction method based on the container cluster management system in any of the foregoing embodiments, and have the beneficial effects of the corresponding resource allocation method or text correction method based on the container cluster management system, which will not be repeated here.
[0165] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the methods of any of the above embodiments.
[0166] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0167] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods of any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0168] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the above embodiments of this application, which are not provided in detail for the sake of brevity.
[0169] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0170] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.
[0171] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.
Claims
1. A training method for a watermark restoration model, characterized in that, include: Obtain watermark style information and background style information; the watermark style information is used to indicate the style of the watermark character content. The background style information is used to indicate the style of the background image content; A watermark image set is generated by combining the watermark style information and the background style information, and the watermark image set includes multiple images with visible watermarks; Each watermark image in the watermark image set is pixelated, and the pixel values in the pixel blocks are extracted as training samples. The corresponding watermark is used as the sample label. The training samples and their corresponding sample labels are combined to generate a training dataset. The process of extracting pixel values from the pixel blocks includes: extracting at least one pixel cell from the pixelated region of the pixelated watermark image; arranging the at least one pixel cell in a matrix according to its position in the pixelated watermark image to obtain a pixel cell matrix; extracting the color value of each column of the pixel cell matrix as a time sequence; normalizing the time sequence; and using the normalized time sequence as a training sample. A bidirectional recurrent neural network model is constructed, and the training dataset is used to train the bidirectional recurrent neural network model to obtain a bidirectional recurrent neural network model that meets the training termination condition, which is used as a watermark restoration model. The watermark restoration model is used to restore the visible watermark characters in the image.
2. The method according to claim 1, characterized in that, The acquisition of watermark style information and background style information includes: The system displays a watermark style information configuration interface and a background style information configuration interface; the watermark style information configuration interface is used to configure at least one of the following: watermark content, watermark color, watermark font, watermark size, and watermark tilt; the background style information configuration interface is used to configure the background color, dark watermark, or texture. Receive watermark style information configured by the user through the watermark style information configuration interface, and receive background style information configured by the user through the background style information configuration interface.
3. The method according to claim 1, characterized in that, Before compressing each watermarked image in the watermarked image set to obtain the corresponding compressed watermarked image, the method further includes: The watermark adjustment strategy is invoked to randomly adjust the watermark images in the watermark image set to obtain the adjusted watermark image set. The watermark adjustment strategy is used to randomly extract watermark images and randomly adjust at least one dimension of watermark font size, watermark transparency, watermark tilt and watermark position.
4. The method according to claim 1, characterized in that, A bidirectional recurrent neural network model is constructed, and the training dataset is used to train the bidirectional recurrent neural network model to obtain a bidirectional recurrent neural network model that meets the training termination condition. This model serves as the watermark restoration model, which is used to restore visible watermark characters in an image, including: Construct a bidirectional recurrent neural network model; The bidirectional recurrent neural network model is trained iteratively using the training dataset and a connection-time classification algorithm to obtain a model that meets the training termination condition, which is then used as the watermark restoration model.
5. The method according to claim 1, characterized in that, Each watermark image in the watermark image set is pixelated, including: A preset pixelation strategy is invoked to perform pixelation processing on each watermark image in the watermark image set. The pixelation strategy is used to indicate at least one of the following: the size of the pixel grid, the shape and size of the pixelated area, and the position coordinates of the pixelated area.
6. A watermark restoration method, characterized in that, include: Acquire the target image and extract the color values from the pixel blocks of the target image; The color value is processed by calling a pre-trained watermark restoration model to obtain the watermark text content corresponding to the target image; The watermark restoration model is trained and generated based on a predefined training dataset. The model takes the color values of the pixel blocks corresponding to the target image as input and the watermark text content corresponding to the target image as output. The predefined training dataset is generated through the following steps: synthesizing a watermark image set based on predefined watermark style information and background style information; pixelating each watermark image in the set and extracting the pixel values from the pixel blocks as training samples; using the corresponding watermark as a sample label; and combining the training samples and their corresponding sample labels to generate a training dataset. The process of extracting pixel values from the pixel blocks includes: extracting at least one pixel cell from the pixelated region of the pixelated watermark image; arranging the at least one pixel cell in a matrix according to its position in the pixelated watermark image to obtain a pixel cell matrix; extracting the color values of each column of pixel cells in the pixel cell matrix as a time series; normalizing the time series; and using the normalized time series as training samples. The watermark restoration result of the target image is obtained by removing blank and / or overlapping parts in the watermark text content corresponding to the target image through a heuristic search algorithm.
7. The method according to claim 6, characterized in that, The process of calling a pre-trained watermark restoration model to process the color values to obtain the watermark text content corresponding to the target image includes: Get the number of candidates specified by the user; Based on the number of candidates, the watermark restoration model is invoked to process the color value to obtain the number of candidate watermark text contents and their respective confidence levels.
8. The method according to claim 6, characterized in that, The step of acquiring the target image and extracting the color values from the pixel blocks of the target image includes: Acquire the target image, and determine whether the target image has been compressed during network transmission based on its source; When it is determined that the target image has been compressed during network transmission, mean filtering is performed on the central part of the pixel block in the target image and then the color value is extracted. When it is determined that the target image has not been compressed during network transmission, the color values of the pixel blocks in the target image are extracted.
9. A training device for a watermark restoration model, characterized in that, include: The watermark and background acquisition module is configured to acquire watermark style information and background style information; the watermark style information is used to indicate the style of the watermark character content; the background style information is used to indicate the style of the background image content. The watermark image generation module is configured to generate a watermark image set based on the watermark style information and the background style information, wherein the watermark image set includes multiple images with visible watermarks. The training dataset construction module is configured to perform pixelation processing on each watermark image in the watermark image set, extract pixel values from pixel blocks as training samples, and use the corresponding watermark as sample labels. The training samples and their corresponding sample labels are then combined to generate a training dataset. The process of extracting pixel values from pixel blocks includes: extracting at least one pixel cell from the pixelated region of the pixelated watermark image; arranging the at least one pixel cell in a matrix according to its position in the pixelated watermark image to obtain a pixel cell matrix; extracting the color values of each column of pixel cells in the pixel cell matrix as a time series; normalizing the time series; and using the normalized time series as a training sample. The training module is configured to construct a bidirectional recurrent neural network model and call the training dataset to train the bidirectional recurrent neural network model to obtain a bidirectional recurrent neural network model that meets the training termination condition as a watermark restoration model. The watermark restoration model is used to restore the visible watermark characters in the image.
10. A watermarked text recognition device, characterized in that, include: The color value extraction module is configured to acquire a target image and extract color values from pixel blocks in the target image. The text restoration module is configured to call a pre-trained watermark restoration model to process the color value to obtain the watermark text content corresponding to the target image; The watermark restoration model is trained and generated based on a predefined training dataset. The model takes the color values of the pixel blocks corresponding to the target image as input and the watermark text content corresponding to the target image as output. The predefined training dataset is generated through the following steps: synthesizing a watermark image set based on predefined watermark style information and background style information; pixelating each watermark image in the set and extracting the pixel values from the pixel blocks as training samples; using the corresponding watermark as a sample label; and combining the training samples and their corresponding sample labels to generate a training dataset. The process of extracting pixel values from the pixel blocks includes: extracting at least one pixel cell from the pixelated region of the pixelated watermark image; arranging the at least one pixel cell in a matrix according to its position in the pixelated watermark image to obtain a pixel cell matrix; extracting the color values of each column of pixel cells in the pixel cell matrix as a time series; normalizing the time series; and using the normalized time series as training samples. The filtering module is configured to remove blank and / or overlapping parts in the watermark text content corresponding to the target image through a heuristic search algorithm, so as to obtain the watermark restoration result corresponding to the target image.
11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the program to implement the method as described in any one of claims 1 to 8.
12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, Computer instructions are used to cause a computer to perform the method as claimed in any one of claims 1 to 8.