Image processing method, device, storage medium and computer equipment

Abstract

The present application provides an image processing method, apparatus, storage medium and computer equipment. After obtaining an image to be processed, at least one enhanced image is obtained by performing image enhancement processing on it. In this way, the obtained enhanced image and the image to be processed are used as model inputs and input into a watermark processing model for processing. The watermark image in the image to be processed can be accurately and efficiently obtained by utilizing the high contrast between the watermark area in the enhanced image and the image content itself, and the watermark image can be repaired as needed to obtain a high-precision repaired image. This solves the technical problem in traditional watermark removal methods that the watermark area cannot be reliably identified due to the low contrast between the watermark area and the surrounding image content, thereby resulting in low accuracy of the repaired image obtained after repairing the watermark area.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology and specifically to an image processing method, device, storage medium and computer equipment. Background Art

[0002] Nowadays, as people's awareness of copyright protection is gradually increasing, many images posted on the Internet are set with watermarks, that is, identifiers composed of several Chinese characters and / or letters, so that the source of the image can be determined by detecting the watermark on the image.

[0003] In practical applications, when it is necessary to remove the watermark information in an image to obtain a watermark-free repaired image, the prior art uses an autoencoder to process the image with watermark information to obtain a repaired image after removing the watermark and repairing the corresponding area.

[0004] However, due to the ever-changing image content, if the watermark area is slightly different from the surrounding image content, the watermark area cannot be reliably identified, resulting in the restoration image obtained after repairing the watermark area is not accurate enough. Summary of the invention

[0005] In view of this, in order to solve the technical problem that in the traditional watermark removal method, the watermark area cannot be reliably identified due to the low contrast between the watermark area and the surrounding image content, and thus the accuracy of the repaired image obtained after repairing the watermark area is low, the application proposes an image processing method, the method comprising:

[0006] Get the image to be processed;

[0007] Performing enhancement processing on the image to be processed to obtain at least one enhanced image;

[0008] Inputting the image to be processed and the at least one enhanced image into a watermark processing model for processing, and outputting a restored image of the image to be processed and / or a watermark image contained in the image to be processed;

[0009] The repaired image refers to an image after the watermark image contained in the image to be processed is repaired.

[0010] In some embodiments, the step of performing enhancement processing on the image to be processed to obtain at least one enhanced image includes:

[0011] Obtaining the pixel value of each pixel of the image to be processed;

[0012] Compare each acquired pixel value with at least one enhancement threshold to obtain a comparison result corresponding to the enhancement threshold;

[0013] The comparison result is used to adjust the pixel value of the corresponding pixel point to obtain an enhanced image corresponding to the enhancement threshold.

[0014] In some embodiments, the training process of the watermark processing model includes:

[0015] Obtain multiple original images, specific characters and watermark icons;

[0016] Using the specific characters and the watermark icon, the plurality of original images are processed to generate a test image and a training image corresponding to each original image, wherein the difference between the test image and the training image is whether the watermark icon is present;

[0017] Performing enhancement processing on the training images respectively to obtain at least one training enhanced image corresponding to the training images;

[0018] Each of the training images and at least one training enhancement image corresponding to the training image constitutes a training sample image group;

[0019] The training sample image group is input into the autoencoder network for learning and training until the output repair image and the corresponding test image meet the first training termination condition, and the output watermark image and the corresponding specific characters and the watermark icon meet the second training termination condition, thereby obtaining a watermark processing model.

[0020] In some embodiments, the processing of the plurality of original images using the specific characters and the watermark icon to generate a test image and a training image corresponding to each original image includes:

[0021] Adding the specific characters to the multiple original images respectively to obtain multiple test images;

[0022] The specific characters and the watermark icon constitute watermark information;

[0023] The watermark information is added to the multiple original images to obtain a corresponding multiple training images.

[0024] The present application also proposes an image processing device, the device comprising:

[0025] An image acquisition module, used for acquiring an image to be processed;

[0026] An image enhancement module, used for performing enhancement processing on the image to be processed to obtain at least one enhanced image;

[0027] An image processing module, used for inputting the image to be processed and the at least one enhanced image into a watermark processing model for processing, and outputting a restored image of the image to be processed and / or a watermark image contained in the image to be processed;

[0028] The repaired image refers to an image after the watermark image contained in the image to be processed is repaired.

[0029] The present application also proposes a storage medium on which a computer program is stored. The computer program is executed by a processor to implement each step of the above-mentioned image processing method.

[0030] The present application also proposes a computer device, the computer device comprising:

[0031] Communication interface;

[0032] A memory, used for storing a program for implementing the above-mentioned image processing method;

[0033] The processor is used to load and execute the program stored in the memory to implement the steps of the above-mentioned image processing method.

[0034] Based on the above technical scheme, the present application provides an image processing method, device, storage medium and computer equipment. After obtaining the image to be processed, at least one enhanced image is obtained by performing image enhancement processing on it. In this way, the obtained enhanced image and the image to be processed are used as model inputs and input into the watermark processing model for processing. The high contrast between the watermark area in the enhanced image and the image content itself can be utilized to accurately and efficiently obtain the watermark image in the image to be processed, and the watermark image can be repaired as needed to obtain a high-precision repaired image. This solves the technical problem in the traditional watermark removal method that the watermark area cannot be reliably identified due to the low contrast between the watermark area and the surrounding image content, thereby resulting in low accuracy of the repaired image obtained after repairing the watermark area. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0036] Figure 1 A schematic diagram of the hardware structure of an optional example of a computer device for implementing the image processing method proposed in the present application is shown;

[0037] Figure 2 A schematic flow chart of an optional example of the image processing method proposed in the present application is shown;

[0038] Figure 3A schematic diagram showing a flow chart of an optional scenario example of the image processing method proposed in the present application;

[0039] Figure 4 A schematic diagram showing another optional example of the image processing method proposed in the present application;

[0040] Figure 5 A schematic diagram showing another optional example of the image processing method proposed in the present application;

[0041] Figure 6 A schematic structural diagram of an optional example of an image processing device proposed in the present application is shown;

[0042] Figure 7 A schematic diagram showing another optional example structure of an image processing device proposed in the present application is shown;

[0043] Figure 8 A schematic diagram showing another optional example structure of an image processing device proposed in the present application is shown;

[0044] Fig. 9 An optional structural diagram of the distributed system proposed in an embodiment of the present application applied to a blockchain system is shown. DETAILED DESCRIPTION

[0045] In response to the technical problem of low accuracy of existing watermark detection and removal raised by the background technology, with the development of artificial intelligence (AI) in computer vision technology (CV) and machine learning / deep learning, this application hopes to use machine learning / deep learning, an artificial intelligence technology, to train sample data and obtain a more reliable watermark processing model.

[0046] Among them, computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify targets, trace traces, measure targets, and perform further graphics processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. Computer vision technology usually includes image processing, image recognition, image retrieval, video processing and other technologies. This application can use this computer vision technology to detect watermark information in the image to be processed, and remove the watermark in the image to be processed to obtain a repaired image. The specific implementation process can refer to the description of the corresponding part below.

[0047] In order to improve the problems existing in the existing watermark removal methods, this application proposes a method that can more reliably and accurately identify the watermark area in the original image, so that after repairing it, the obtained repaired image can be guaranteed to be sufficiently accurate. However, in the existing watermark detection method, a target detection algorithm is usually used to detect watermarks on the input image. If there are Chinese characters, letters and other characters in the content of the input image itself, it often misleads the watermark detection, and the detected watermark area cannot be detected, which in turn affects the accuracy of the repaired image.

[0048] In order to improve the above problems, through the study of the technical problems existing in the existing watermark detection and removal methods, and targeting the problem of low contrast between the watermark area and its surrounding images, this application proposes to highlight the watermark itself by constructing a variety of enhanced images, that is, to increase the contrast between the watermark area and its surrounding images, so as to improve the reliability of watermark removal and thus improve the accuracy of the resulting restored image.

[0049] Furthermore, in order to solve the problem that some original images contain characters, which makes it impossible to reliably identify watermark information, research and development has found that the watermark information in the original image contains not only characters, but also watermark icons, and the watermark icon is used to identify the application platform of the image source. Even if the image itself contains characters, it will not contain watermark icons. Therefore, this application can also distinguish whether the characters belong to the image itself or the watermark information by identifying whether there is a watermark icon in front of the characters, so as to improve the accuracy of watermark detection.

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] It should be noted that, for the convenience of description, only the parts related to the relevant invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0052] It should be understood that the "system", "device", "unit" and / or "module" used in this application is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0053] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

[0054] Among them, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two. The following terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0055] In addition, flow charts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, the various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0056] Reference Figure 1 , is a schematic diagram of an optional example of a computer device for implementing the image processing method provided by the present application. In practical applications, the computer device may be a server, or a terminal with image processing capabilities, such as a smart phone, a tablet computer, an ultra-mobile personal computer (UMPC), a netbook, a desktop computer, etc. The present application does not limit the product type of the computer device. Figure 1 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0057] like Figure 1 As shown, the computer device proposed in this embodiment may include: a communication interface 11, a memory 12 and a processor 13, wherein:

[0058] The number of each of the communication interface 11, the memory 12, and the processor 13 can be at least one, and the communication interface 11, the memory 12, and the processor 13 can be connected to a communication bus, and data can be exchanged with each other through the communication bus. The specific implementation process can be determined according to actual application requirements and is not described in detail in this application.

[0059] The communication interface 11 can be an interface of a communication module suitable for a wireless network or a wired network, such as an interface of a GSM module, a WIFI module, a Bluetooth module, a wireless radio frequency model, a 5G module or other communication module, which can realize data interaction with other devices and receive information such as images sent by other devices; of course, the communication interface 11 can also include interfaces such as USB interfaces, serial / parallel ports, etc., which are used to realize data interaction between internal components of a computer device, which can be determined according to the product type of the computer device, and this application will not describe them in detail one by one.

[0060] The memory 12 can be used to store programs for implementing the image processing method proposed in the present application; the processor 13 can be used to load and execute the programs stored in the memory 12 to implement the various steps of the image processing method proposed in the embodiments of the present application. The specific implementation process can refer to but is not limited to the description of the corresponding parts of the embodiments below.

[0061] In the embodiment of the present application, the memory 12 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device. The processor 13 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, etc.

[0062] In one possible implementation, the memory 12 may include a program storage area and a data storage area. The program storage area may store an operating system, an application program required for at least one function (such as an image display function, an image recognition function), a program for implementing the image processing method proposed in the present application, etc.; the data storage area may store data generated during the use of the computer device, such as an acquired image to be processed, an enhanced image obtained after processing, watermark information, a repaired image, etc.

[0063] It should be understood that Figure 1 The structure of the computer device shown does not constitute a limitation on the computer device in the embodiment of the present application. In practical applications, the computer device may include Figure 1More or fewer components shown, or a combination of certain components, can be determined in conjunction with the product type and functional requirements of the computer device, and this application does not list them one by one here.

[0064] Reference Figure 2 , is a flowchart of an optional example of the image processing method proposed in the present application, the method can be applied to a computer device, the type and structure of the computer device can refer to but not limited to the computer device described in the above embodiment, such as Figure 2 As shown, the image processing method proposed in this embodiment may include but is not limited to the following steps:

[0065] Step S11, obtaining an image to be processed;

[0066] In this embodiment, the image to be processed may be an image containing watermark information. This application does not limit the source of the image to be processed. It may be uploaded by a user to a computer device, or crawled by a computer device from a third-party application platform, etc. It can be determined according to actual scenario requirements.

[0067] In some embodiments, in addition to watermark information, the image to be processed may also contain characters in its image content, such as at least one word, and / or at least one letter, and / or at least one symbol. The present application does not limit the content and quantity of the characters, which may be a single character or a string of multiple characters. Of course, the image content itself may not contain characters, but only images, and the present application does not limit this.

[0068] Step S12, performing enhancement processing on the image to be processed to obtain at least one enhanced image;

[0069] It should be noted that the present application does not limit the image enhancement processing method. In computer vision technology, image enhancement processing can improve the visual effect of the image, purposefully emphasize the overall or local characteristics of the image, make the originally unclear image clear or emphasize certain features of interest, expand the differences between the features of different objects in the image, suppress features of no interest, thereby improving image quality, enriching the amount of information, enhancing image interpretation and recognition effects, and meeting the needs of certain special analyses.

[0070] Based on this, in this embodiment, by performing enhancement processing on the image to be processed, the watermark information in the watermark area of ​​the image to be processed can be made clearer, and the image content itself can be made unclear, thereby increasing the contrast between the watermark information and the image content itself, and also increasing the contrast between the watermark information and the surrounding image content. Especially when the watermark information in the image to be processed is close in color to the surrounding image content and the contrast is low, this image enhancement technology is used to improve the contrast, which is helpful for subsequent reliable identification of the watermark information in the image to be processed and improves the accuracy of the restored image.

[0071] In the present application, refer to Figure 3 As shown in the schematic diagram of the scene flow, after the computer device obtains the image to be processed, Figure 3 Take the example that the image to be processed also contains characters. In some embodiments, the image to be processed may not contain characters. The processing process of such an image to be processed is similar and will not be described in detail in this application. In order to improve the accuracy of image processing, this embodiment can enhance the image to be processed according to at least one contrast ratio to obtain an enhanced image with a corresponding contrast ratio, such as Figure 3 As shown, the contrast value between the watermark information and the content of the image to be processed in each enhanced image is different. Figure 3 The contrast of enhanced image 2 is greater than that of enhanced image 1.

[0072] It should be noted that the present application does not limit the contrast and number of the enhanced images obtained, and the methods for acquiring different enhanced images can be the same or different. That is to say, the present application can adopt one image enhancement method to enhance the image to be processed to obtain one or more enhanced images, or can adopt different image enhancement methods to enhance the image to be processed to obtain multiple enhanced images. The present application does not limit the specific implementation method of step S12.

[0073] Step S13: input the image to be processed and the at least one enhanced image into a watermark processing model for processing, and output a restored image of the image to be processed and / or a watermark image contained in the image to be processed.

[0074] Following the above description, after obtaining the image to be processed and at least one corresponding enhanced image, the present application Figure 3 As shown, these images can be used as model input images and input into a pre-trained watermark processing model for processing, and the desired watermark image, i.e., watermark information, and / or the desired repaired image can be output. The repaired image is an image obtained by removing the watermark information in the image to be processed and repairing the area where the watermark was removed.

[0075] It can be seen that in some embodiments, the present application utilizes a watermark processing model to simultaneously output the watermark image and the repaired image of the image to be processed. Compared with the traditional concept of separately constructing a watermark detection model and a watermark removal model, processing the image to be processed respectively, and outputting the corresponding watermark image or repaired image, the embodiments of the present application simplify the image processing steps and improve the image processing efficiency.

[0076] The watermark processing model can be obtained by training sample data using an autoencoder network, and the sample data includes a sample training image and a training enhanced image obtained after the sample training image is enhanced. The present application does not limit the specific training method of the watermark processing model. It should be understood that in the training process of the watermark processing model, in order to improve the accuracy of the obtained watermark image, the loss function of the watermark image based on the loss function of the restored image based on the loss function of the restored image based on the loss function of the restored image may be different or the same, and the present application does not limit the type of the loss function.

[0077] The autoencoder network can be a neural network with 3 layers or more than 3 layers. It is a type of unsupervised learning network that can automatically learn features from unlabeled data. It is a neural network that aims to reconstruct input information. It can give better feature descriptions than the original data and has strong feature learning capabilities. In deep learning, the features generated by the autoencoder network are often used to replace the original data to achieve better results. This application does not limit the type and operation principle of the autoencoder network used for model training, and can be flexibly selected according to actual needs.

[0078] To sum up, in this embodiment, after obtaining the image to be processed, it is not directly input into the watermark processing model for processing, but first the image to be processed is subjected to image enhancement processing. After obtaining at least one enhanced image, the obtained enhanced image and the image to be processed are used as model inputs, and then input into the watermark processing model for processing, so that during the image processing process of the watermark processing model, the high contrast characteristic between the watermark area in the enhanced image and the image content itself can be utilized to more accurately and efficiently obtain the watermark image in the image to be processed, and the area where the watermark image is located in the image to be processed is repaired to obtain the repaired image. Compared with the prior art of directly processing the image to be processed using an autoencoder, the accuracy of the watermark image and the repaired image is greatly improved, and the image processing efficiency is improved.

[0079] Reference Figure 4, which is a flowchart of another optional example of the image processing method proposed in the present application. This embodiment may be an optional implementation method for image enhancement in the image processing method described in the above embodiment. For other processing steps of image processing, reference may be made to the description of the corresponding parts of the above embodiment. This embodiment will not be repeated here, and mainly describes the detailed implementation process of the above step S12. Figure 4 As shown, the implementation method may include:

[0080] Step S21, obtaining the pixel value of each pixel of the image to be processed;

[0081] Step S22, comparing each acquired pixel value with at least one enhancement threshold, to obtain a comparison result corresponding to the enhancement threshold;

[0082] The embodiment of the present application does not limit the specific value of each enhancement threshold, which can be determined based on the pixel value of each pixel point of the image to be processed.

[0083] Step S23, using the comparison result, adjusting the pixel value of the corresponding pixel point to obtain an enhanced image corresponding to the enhancement threshold.

[0084] In combination with the above description, this embodiment constructs an enhanced image in order to highlight the watermark itself in the image so as to improve the reliability of watermark removal. However, under normal circumstances, the brightness of the watermark area is different from the brightness of the image content itself. This embodiment can enhance the image to be processed based on this difference.

[0085] Specifically, this embodiment can use the pixel value of the pixel point to indicate the brightness value of the image to be processed. After determining an enhancement threshold T, for each pixel point of the image to be processed, if its pixel value is less than T, it is generally considered that the pixel point belongs to the pixel point of the image content to be processed itself, rather than the pixel point of the watermark information. In order to highlight the watermark information, this embodiment can reduce the pixel value of the pixel point, such as adjusting the pixel value of the pixel point whose pixel value is less than T to 0, but is not limited to this.

[0086] If the pixel value of a pixel in the image to be processed is greater than T, the pixel value of the pixel can be adjusted to a value between 0 and 256. The specific pixel value to be adjusted can be determined based on actual conditions, and this application does not limit it. However, it should be understood that after the above-mentioned pixel value adjustment, the difference between the pixel values ​​of pixel points with pixel values ​​greater than T and pixel points with pixel values ​​less than T will increase, thereby increasing the contrast between the watermark information and the image content.

[0087] According to the enhancement processing described above, this application selects different enhancement thresholds T for processing, and can obtain enhanced images with different contrasts. The specific implementation process is not described in detail in this application.

[0088] In summary, this embodiment utilizes the brightness difference between the watermark area and the image content area in the image to be processed, and enhances the brightness difference by adjusting the pixel values ​​of the pixels in each area, that is, improving the contrast, thereby obtaining an enhanced image with different contrasts, to assist in the extraction and removal of the watermark information in the image to be processed, thereby improving the image processing efficiency and accuracy.

[0089] In some embodiments, in order to achieve enhanced processing of the image to be processed and obtain an enhanced image, the present application may also use a histogram equalization method and / or a histogram matching method to enhance the image to be processed and obtain at least one enhanced image.

[0090] Among them, histogram equalization is a method of adjusting contrast using image histogram in the field of image processing. It is usually used to increase the global contrast of many images, especially when the contrast of the useful data of the image is quite close. In this way, the brightness can be better distributed on the histogram. This can be used to enhance the local contrast without affecting the overall contrast. The histogram is a graphical representation of the data distribution. It is a two-dimensional statistical chart. Its two coordinates are the statistical sample and the measurement of a certain attribute corresponding to the sample.

[0091] Based on this, the present application can obtain the histogram information of the image to be processed, and use the histogram information to obtain a transformation function, thereby using the transformation function to process the image to be processed to increase the contrast of the image to be processed. The present application does not elaborate on the specific method of how to use the histogram equalization method to achieve image enhancement processing.

[0092] Histogram matching, also known as histogram specification, refers to an image enhancement method that converts the histogram of an image into a histogram of a specified shape. Specifically, it can match the histogram of a certain image or a certain area to another image so that the tones of the two images remain consistent. It can also match between single-band image histograms, or simultaneously match multi-band images. Before comparing two images, their histograms are usually consistent. This application does not elaborate on how to use the histogram matching method to achieve the enhancement process of the image to be processed.

[0093] It should be noted that the present application does not limit the implementation method of image enhancement processing, and is not limited to the several implementation methods listed in the present application. The present embodiment can flexibly select the image enhancement method according to the actual situation to achieve enhanced processing of the image to be processed, and the present application does not describe them in detail one by one.

[0094] Reference Figure 5, which is a flowchart of another optional example of the image processing method proposed in this application. This embodiment can be an optional training implementation method of the above-mentioned watermark processing model, such as Figure 5 As shown, the training process may include:

[0095] Step S31, obtaining a plurality of original images, specific characters and watermark icons;

[0096] In this embodiment, the original image can be any natural image, which can be considered as a pure image without characters and watermark icons. This application does not limit the source of the original image.

[0097] A specific character can be a single character or a string composed of at least one word, at least one letter and / or at least one symbol. The present application does not limit the content of the specific character. In actual applications, according to actual needs, the present application can select one or more groups of specific characters to construct test images and training images. For each group of characters, a corresponding watermark icon can be configured.

[0098] Among them, the watermark icon can be a unique identifier of different application platforms, such as the logo of the application platform, so that the source of the image can be determined by identifying the watermark icon. This application does not limit the content of the watermark icon.

[0099] Step S32, adding specific characters to the multiple original images respectively to obtain multiple test images;

[0100] Step S33, forming watermark information from specific characters and watermark icons, and adding the watermark information to multiple original images to obtain corresponding multiple training images;

[0101] In order to reliably distinguish the characters in the watermark image and the characters in the image content itself, this embodiment can use the pre-set watermark icon and specific characters to construct a watermark image similar to that in the real scene, that is, the watermark icon can be followed by one or more characters as the identification ID of the image. At the same time, a test image with characters in the image content is constructed so that in the model training process, it is learned that the characters in the watermark image must rely on the watermark icon, so that the obtained model only recognizes the characters with the watermark icon in front and ignores the characters without the watermark icon in front.

[0102] Based on this, for each original image, on the one hand, the embodiment of the present application can add the above-mentioned specific characters and use the obtained image as a test image. On the other hand, specific characters and watermark icons can be added to the original image and the obtained image can be used as a training image. It can be seen that the test image and the training image obtained by processing the same original image contain the same characters. The difference is that the test image only contains characters but not watermark icons, while the training image contains both characters and watermark icons. The training image can refer to the above Figure 3 The image to be processed is shown in the figure. The test image can refer to the above Figure 3 The repaired image is shown.

[0103] It can be seen that the present application can utilize pre-defined specific characters and watermark icons to process multiple original images, generate test images and training images corresponding to each original image, and the difference between the test image and the training image corresponding to the same original image is whether they have a watermark icon.

[0104] In a possible implementation, the process of constructing the watermark information may be as follows: configuring the watermark icon to be adjacent to the specific character, and the watermark icon to be located in front of the specific character in the reading order, to obtain the corresponding watermark information, i.e., the format of "watermark icon + specific character".

[0105] Step S34, performing enhancement processing on the training images respectively to obtain at least one training enhanced image corresponding to the training images;

[0106] Regarding the image enhancement processing process of step S34, reference may be made to the description of the corresponding part of the above embodiment, and the embodiment of the present application will not be repeated here.

[0107] Step S35, each training image and at least one training enhancement image corresponding to the training image form a training sample image group;

[0108] In this embodiment, after each training image is enhanced using different enhancement thresholds T, the obtained training enhanced image can be put together with the corresponding training image to form an image group, namely, a training sample image group. This application does not limit the number of enhanced images contained in each training sample image group, which can be Figure 3 The two enhanced images shown may also include three or even more enhanced images. The specific implementation process is similar and will not be described in detail in this application.

[0109] Step S36, input the training sample image group into the autoencoder network for learning and training, until the output repaired image and the corresponding test image meet the first training termination condition, and the output watermark image and the corresponding watermark information meet the second training termination condition, and obtain the watermark processing model.

[0110] In combination with the above analysis, in some embodiments, the autoencoder network can be a fully convolutional neural network, which can first extract features of the input image and then restore the desired target image from the extracted features. This application does not describe in detail the specific operation process of the fully convolutional neural network.

[0111] In an embodiment of the present application, the repaired image output by the model is an original image with characters superimposed on it and without a watermark icon added, such as the above-mentioned test image. Therefore, the above-mentioned first training termination condition may be that the similarity between the repaired image output by the model and the above-mentioned corresponding test image is higher than the first similarity threshold, and it can be considered that the first training termination condition is met.

[0112] Similarly, since the autoencoder network used in the present application to implement watermark detection and watermark removal is shared, under normal circumstances, the watermark image output by the model should be the image of the watermark information in the input training image. Therefore, the above-mentioned second training termination condition can be that the similarity between the watermark image output by the model and the corresponding watermark information is higher than the second similarity threshold, and it can be considered that the second training termination condition is met.

[0113] It should be noted that the present application does not limit the specific values ​​of the first similarity threshold and the second similarity threshold, and does not limit the contents of the first training termination condition and the second training termination condition.

[0114] In some embodiments, the present application can use a loss function to implement model training. Specifically, this embodiment inputs a training sample image group into an autoencoder network for learning and training until the loss values ​​of the loss function of the repaired image and the loss function of the watermarked image are each less than a threshold. Specifically, it can be said that the loss values ​​of the loss function of the repaired image and the loss function of the watermarked image tend to be stable, and the training can be stopped. The model finally obtained by training is recorded as a watermark processing model.

[0115] In a possible implementation, the loss function of the above-mentioned repaired image may be an L1 loss function between the repaired image and the test image; and the loss function of the watermarked image may be a cross entropy classification loss function between the watermarked image output pixel by pixel and the watermark information.

[0116] Among them, the L1 loss function is the L1 norm loss function, also known as the minimum absolute deviation (LAD) and the minimum absolute error (LAE). In general, it minimizes the sum of the absolute differences between the target value and the estimated value. This application does not elaborate on the application of the L1 loss function in the watermark processing model training process.

[0117] Cross entropy is mainly used to measure the difference information between two probability distributions. Cross entropy is used as a loss function in neural networks (machine learning). If p represents the distribution of true labels, and q is the predicted label distribution of the trained model, the cross entropy loss function can measure the similarity between p and q. The embodiment of the present application uses cross entropy as a loss function because the learning rate can be controlled by the output error. Therefore, the use of the sigmoid function when the gradient is descended avoids the problem of reduced learning rate of the mean square error loss function. This application does not elaborate on the application of the cross entropy loss function in the training process of the watermark processing model.

[0118] To sum up, in the process of training the watermark processing model, this embodiment will construct a test image containing characters and a training image containing the characters and a watermark icon for the same original image, so that the trained model will only recognize characters with watermark icons during watermark detection and ignore characters without watermark icons in front of them, thereby avoiding the adverse effects of characters in the image content itself on the watermark detection results and reducing the accuracy of watermark detection.

[0119] In addition to the training images obtained above, for the training sample data of the input model, the present application can also perform enhancement processing on them to obtain corresponding enhanced images, so that the training images and their corresponding enhanced images are used as training sample image groups and input into the autoencoder network for learning and training, so as to utilize the high contrast between the watermark area in the enhanced image and the surrounding image content to reliably identify the repaired image that does not contain watermark information, thereby improving the reliability and training efficiency of the watermark processing model obtained through training.

[0120] In some embodiments, after obtaining multiple training images through the above step S33, they can be directly used as model training samples and directly input into the autoencoder network for learning and training. The model training process can refer to the description of the corresponding part of the above embodiment. The watermark processing model obtained by such training can only recognize characters with watermark icons and ignore characters without watermark icons in front, thereby avoiding the adverse effects of characters in the image content itself on the watermark detection results, and reducing the accuracy of watermark detection and watermark removal. Therefore, in practical applications, after obtaining the image to be processed, the image to be processed can be directly input into the watermark processing model for processing, so that accurate and reliable watermark images and / or repaired images can be obtained to meet the image processing requirements. The specific implementation process can refer to the description of the corresponding part of the above embodiment, and this application will not be described separately.

[0121] It can be seen that in practical applications, the present application can flexibly select the watermark processing model trained according to the above different training samples according to the specific situation of the watermark area of ​​the image to be processed, realize watermark detection and / or watermark removal, and meet application requirements.

[0122] In some embodiments, if the content of the image to be processed that actually needs to be processed does not contain any characters, then when training the watermark processing model, the original image can be used as a test image, and the image after the watermark information is added to the original image can be used as a training image. After that, the model training is performed in the manner described above to obtain a watermark processing model suitable for the image to be processed in this scenario, thereby improving the image processing efficiency. Of course, the watermark processing model trained in the above application can still be applied to the image processing scenario of the image to be processed that does not carry characters in the above application. The image processing process in this scenario is similar to the description process of the corresponding part of the above embodiment, and this application will not repeat it.

[0123] Reference Figure 6 , is a schematic diagram of an optional example of the structure of the image processing device proposed in this application, which can be applied to computer devices, such as Figure 6 As shown, the device may include:

[0124] An image acquisition module 21 is used to acquire an image to be processed;

[0125] An image enhancement module 22, configured to perform enhancement processing on the image to be processed to obtain at least one enhanced image;

[0126] In some embodiments, Figure 7 As shown, the above-mentioned image enhancement module 22 may include:

[0127] The pixel value acquisition unit 221 is used to acquire the pixel value of each pixel point of the image to be processed;

[0128] A comparison unit 222, configured to compare each acquired pixel value with at least one enhancement threshold, to obtain a comparison result corresponding to the enhancement threshold;

[0129] The pixel value adjustment unit 223 is used to adjust the pixel value of the corresponding pixel point by using the comparison result to obtain the enhanced image corresponding to the enhancement threshold.

[0130] In some further embodiments, the image enhancement module 22 may further include:

[0131] The first enhancement processing unit is used to perform enhancement processing on the image to be processed by adopting a histogram equalization method to obtain at least one enhanced image.

[0132] The second enhancement processing unit is used to perform enhancement processing on the image to be processed by adopting a histogram matching method to obtain at least one enhanced image.

[0133] It should be noted that the present application does not limit the above-mentioned image enhancement processing method. Different image enhancement methods can be implemented using corresponding functional modules, and the present application does not describe them in detail one by one.

[0134] The image processing module 23 is used to input the image to be processed and the at least one enhanced image into the watermark processing model for processing, and output a restored image of the image to be processed and / or a watermark image contained in the image to be processed;

[0135] The repaired image refers to an image after the watermark image contained in the image to be processed is repaired.

[0136] In some embodiments, in order to realize the construction of the watermark processing model, such as Figure 8 As shown, the image processing device proposed in this application may also include:

[0137] The original data acquisition module 24 is used to acquire multiple original images, specific characters and watermark icons;

[0138] A sample image generating module 25, configured to process the plurality of original images using the specific characters and the watermark icon, and generate a test image and a training image corresponding to each original image, wherein the difference between the test image and the training image is whether the watermark icon is present;

[0139] In a possible implementation, the sample image generation module 25 may include:

[0140] A test image obtaining unit, used for adding the specific characters to the multiple original images respectively to obtain multiple test images;

[0141] A watermark information forming unit, configured to form watermark information from the specific characters and the watermark icon;

[0142] The training image obtaining unit is used to add the watermark information to the multiple original images to obtain a corresponding multiple training images.

[0143] An enhancement processing module 26, configured to perform enhancement processing on the training images respectively to obtain at least one training enhanced image corresponding to the training images;

[0144] A training sample image group construction module 27, configured to form a training sample image group from each of the training images and at least one training enhancement image corresponding to the training image;

[0145] The model training module 28 is used to input the training sample image group into the autoencoder network for learning and training until the output repair image and the corresponding test image meet the first training termination condition, and the output watermark image and the corresponding specific characters and the watermark icon meet the second training termination condition, thereby obtaining a watermark processing model.

[0146] In a possible implementation, the model training module 28 may include:

[0147] A learning and training unit, used for inputting the training sample image group into the autoencoder network for learning and training, until the loss value of the loss function of the repaired image and the loss function of the watermarked image are respectively less than a threshold value;

[0148] The loss function of the repaired image is the L1 loss function between the repaired image and the test image; the loss function of the watermarked image is the cross entropy classification loss function between the watermarked image and the watermark information output pixel by pixel.

[0149] It should be noted that the various modules, units, etc. in the above-mentioned device embodiments can be stored in the memory as program modules, and the processor executes the above-mentioned program modules stored in the memory to implement the corresponding functions. For the functions implemented by each program module and its combination, as well as the technical effects achieved, please refer to the description of the corresponding parts of the above-mentioned method embodiments, which will not be repeated in this embodiment.

[0150] An embodiment of the present application further provides a storage medium on which a computer program is stored. The computer program is executed by a processor to implement the steps of the above-mentioned image processing method. The implementation process of the image processing method can refer to the description of the above-mentioned method embodiment.

[0151] Refer to Figure 2 The embodiment of the present application also proposes a computer device. Regarding the composition structure and functions of the computer device, please refer to the description of the corresponding parts of the above embodiment, and this application will not repeat them.

[0152] In practical applications, the computer device proposed in this application may be a node in a distributed system, which may be formed by a client and multiple nodes (any form of computer device in the access network, such as a server, a user terminal) connected through network communication. This application does not limit the type of the distributed system and its composition structure.

[0153] Taking the distributed system as the blockchain system as an example, refer to Fig. 9The distributed system proposed in the embodiment of the present application is applied to an optional structural diagram of a blockchain system, and the distributed system 100 can be formed by multiple nodes 200 and clients 300. In the distributed system, any machine such as a server or a terminal can join and become a node, and the node includes a hardware layer, an intermediate layer, an operating system layer, and an application layer.

[0154] like Fig. 9 In the example, routing can be a basic function of a node, used to support communication between nodes; the application is deployed in the blockchain, and can implement specific services according to actual business needs, record data related to the implementation of the function to form record data, carry a digital signature in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system, so that other nodes can add the record data to the temporary block when the source and integrity of the record data are successfully verified. In this embodiment, the application can support the image processing method described in the above embodiment.

[0155] The blockchain includes a series of blocks that are connected to each other in the order of their generation. It is essentially a decentralized database, a series of data blocks generated by cryptographic methods. Each data block contains relevant information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block to improve the security of data storage. This application does not elaborate on the specific structure and application of the blockchain.

[0156] based on Fig. 9 In the distributed system shown, when a user needs to perform watermark detection and watermark removal on an image, the image can be sent to a computer device at any node in the distributed system for processing through the client. The specific processing process can refer to the description of the corresponding part of the above method embodiment, and this application will not repeat it. The node can feed back the obtained repaired image and / or watermark image to the client output to meet the user's needs.

[0157] It should be noted that the computer device proposed in this application may not belong to a node in a distributed system. Even if it belongs to a node in a distributed system, the distributed system is not limited to the blockchain system. This application can be determined based on actual needs.

[0158] In addition, the various embodiments in this specification are described in a progressive or parallel manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the devices and computer equipment disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0159] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0160] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0161] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the core idea or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: The method comprises: Get the image to be processed; Performing enhancement processing on the image to be processed to obtain at least one enhanced image; Inputting the image to be processed and the at least one enhanced image into a watermark processing model for processing, and outputting a restored image of the image to be processed and / or a watermark image contained in the image to be processed; The repaired image refers to an image after the watermark image contained in the image to be processed is repaired; The training process of the watermark processing model includes: Using specific characters and watermark icons, multiple original images are processed to generate a test image and a training image corresponding to each original image, wherein the difference between the test image and the training image is whether the watermark icon is present; The training sample image group is input into the autoencoder network for learning and training until the output repair image and the corresponding test image meet the first training termination condition, and the output watermark image and the corresponding specific character and the watermark icon meet the second training termination condition, thereby obtaining a watermark processing model; a training sample image group consists of a training image and at least one training enhancement image corresponding to the training image.

2. The method according to claim 1, characterized in that The step of performing enhancement processing on the image to be processed to obtain at least one enhanced image comprises: Obtaining the pixel value of each pixel of the image to be processed; Compare each acquired pixel value with at least one enhancement threshold to obtain a comparison result corresponding to the enhancement threshold; The comparison result is used to adjust the pixel value of the corresponding pixel point to obtain an enhanced image corresponding to the enhancement threshold.

3. The method according to claim 1, characterized in that The step of performing enhancement processing on the image to be processed to obtain at least one enhanced image comprises: The image to be processed is enhanced by adopting a histogram equalization method and / or a histogram matching method to obtain at least one enhanced image.

4. The method according to any one of claims 1 to 3, characterized in that: The training process of the watermark processing model also includes: Before the multiple original images are processed by using the specific characters and watermark icons to generate a test image and a training image corresponding to each original image, multiple original images, specific characters and watermark icons are obtained; After the specific characters and watermark icons are used to process multiple original images to generate a test image and a training image corresponding to each original image, the training images are enhanced to obtain at least one training enhanced image corresponding to the training images.

5. The method according to any one of claims 1 to 3, characterized in that: The method of processing a plurality of original images by using specific characters and watermark icons to generate a test image and a training image corresponding to each original image includes: Adding the specific characters to the multiple original images respectively to obtain multiple test images; The specific characters and the watermark icon constitute watermark information; The watermark information is added to the multiple original images to obtain a corresponding multiple training images.

6. The method according to claim 5, characterized in that The watermark information formed by the specific characters and the watermark icon includes: The watermark icon is configured to be adjacent to the specific character, and the watermark icon is located in front of the specific character in the reading order to obtain corresponding watermark information, and the specific character includes at least one word, and / or at least one letter, and / or at least one symbol.

7. The method according to any one of claims 1 to 3, characterized in that: The step of inputting the training sample image group into the autoencoder network for learning and training until the output repaired image and the corresponding test image meet the first training termination condition, and the output watermark image and the corresponding specific character and the watermark icon meet the second training termination condition includes: Input the training sample image group into the autoencoder network for learning and training until the loss value of the restoration image loss function and the watermark image loss function is less than the threshold value; The loss function of the repaired image is the L1 loss function between the repaired image and the test image; the loss function of the watermarked image is the cross entropy classification loss function between the watermarked image and the watermark information output pixel by pixel.

8. An image processing device, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be processed; An image enhancement module, used for performing enhancement processing on the image to be processed to obtain at least one enhanced image; An image processing module, used for inputting the image to be processed and the at least one enhanced image into a watermark processing model for processing, and outputting a restored image of the image to be processed and / or a watermark image contained in the image to be processed; The repaired image refers to an image after the watermark image contained in the image to be processed is repaired; A sample image generation module, used to process multiple original images using specific characters and watermark icons, and generate a test image and a training image corresponding to each original image, wherein the difference between the test image and the training image is whether the watermark icon is present; The model training module is used to input the training sample image group into the autoencoder network for learning and training until the output repair image and the corresponding test image meet the first training termination condition, and the output watermark image and the corresponding specific character and the watermark icon meet the second training termination condition, so as to obtain the watermark processing model; a training sample image group is composed of a training image and at least one training enhancement image corresponding to the training image.

9. The device according to claim 8, characterized in that The image enhancement module comprises: A pixel value acquisition unit, used to acquire the pixel value of each pixel point of the image to be processed; A comparison unit, used to compare each acquired pixel value with at least one enhancement threshold, to obtain a comparison result corresponding to the enhancement threshold; The pixel value adjustment unit is used to adjust the pixel value of the corresponding pixel point by using the comparison result to obtain the enhanced image corresponding to the enhancement threshold.

10. The device according to claim 8, characterized in that The image enhancement module comprises: A first enhancement processing unit, configured to perform enhancement processing on the image to be processed by adopting a histogram equalization method to obtain at least one enhanced image; The second enhancement processing unit is used to perform enhancement processing on the image to be processed by adopting a histogram matching method to obtain at least one enhanced image.

11. The device according to any one of claims 8 to 10, characterized in that: The device also includes: The original data acquisition module is used to acquire multiple original images, specific characters and watermark icons before processing multiple original images using specific characters and watermark icons to generate test images and training images corresponding to each original image; The enhancement processing module is used to perform enhancement processing on the training images respectively after processing multiple original images using specific characters and watermark icons to generate test images and training images corresponding to each original image, so as to obtain at least one training enhanced image corresponding to the training images.

12. The device according to any one of claims 8 to 10, characterized in that: The sample image generation module comprises: A test image obtaining unit, used for adding the specific characters to the multiple original images respectively to obtain multiple test images; A watermark information forming unit, configured to form watermark information from the specific characters and the watermark icon; The training image obtaining unit is used to add the watermark information to the multiple original images to obtain a corresponding multiple training images.

13. The device according to claim 12, characterized in that The watermark information forming unit is specifically used for: The watermark icon is configured to be adjacent to the specific character, and the watermark icon is located in front of the specific character in the reading order to obtain corresponding watermark information, and the specific character includes at least one word, and / or at least one letter, and / or at least one symbol.

14. The device according to any one of claims 8 to 10, characterized in that: The model training module includes: A learning and training unit, used for inputting the training sample image group into the autoencoder network for learning and training, until the loss value of the loss function of the repaired image and the loss function of the watermarked image are respectively less than a threshold value; The loss function of the repaired image is the L1 loss function between the repaired image and the test image; the loss function of the watermarked image is the cross entropy classification loss function between the watermarked image and the watermark information output pixel by pixel.

15. A storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement each step of the image processing method according to any one of claims 1 to 7.

16. A computer device, characterized in that: The computer device comprises: Communication interface; A memory, used to store a program for implementing the image processing method according to any one of claims 1 to 7; A processor is used to load and execute the program stored in the memory to implement each step of the image processing method according to any one of claims 1 to 7.

Images