Watermark embedding method and device, electronic equipment and storage medium

Abstract

The invention discloses a watermark embedding method and device, a watermark recognition method and device, electronic equipment and a storage medium. The embodiment of the invention relates to the technical fields of artificial intelligence, cloud technology and the like. The watermark embedding method comprises the following steps: acquiring a frequency domain coefficient sequence of a target image block in a to-be-processed image under a single-color channel; selecting a target frequency domain coefficient sequence from the frequency domain coefficient sequences; determining a reference image block under the single-color channel according to the target frequency domain coefficient sequence and sub-watermark information corresponding to a target image block in the watermark information; and obtaining a watermark image according to the target image block and the reference image block corresponding to the target image block under the single-color channel. In the application, the target frequency domain coefficient sequence in the preset position range in the to-be-processed image is adjusted to obtain the watermark image, the adjustment amount and the adjustment amplitude of the frequency domain coefficient of the to-be-processed image are relatively low, the influence of watermark information on the to-be-processed image is reduced, and the image quality of the obtained watermark image is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a watermark embedding method, device, electronic device and storage medium. Background Art

[0002] Due to the rapid development of Internet technology, more and more images and videos are facing the risk of being pirated and leaked, which will cause significant losses to the copyright holders. Therefore, it is necessary to strengthen the copyright protection of images and videos and the traceability after leakage. Among them, watermarking is an important part of image and video copyright protection.

[0003] At present, the watermark encryption of the image can be completed by converting the image from the spatial domain to the frequency domain and implanting information in the frequency domain. However, when the watermark is embedded using the above method, the embedded watermark information has a great impact on the quality of the original image, resulting in poor image quality after the watermark information is embedded. Summary of the invention

[0004] In view of this, the embodiments of the present application propose a watermark embedding method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present application provides a watermark embedding method, the method comprising: obtaining a frequency domain coefficient sequence of a target image block in an image to be processed under a single color channel; selecting a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel; adjusting the frequency domain coefficients in the target frequency domain coefficient sequence according to a size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and sub-watermark information corresponding to the target image block in the watermark information to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence; determining a reference image block corresponding to the target image block under a single color channel according to the adjusted frequency domain coefficient sequence; and fusing the target image block in the image to be processed and the reference image block corresponding to the target image block under a single color channel to obtain a watermark image corresponding to the image to be processed.

[0006] In a second aspect, an embodiment of the present application provides a watermark recognition method, the method comprising: obtaining a predicted frequency domain coefficient sequence of a predicted image block including a watermark to be recognized in a watermark image to be recognized under a single color channel; selecting a target predicted frequency domain coefficient sequence located in a preset position range from the predicted frequency domain coefficient sequence; determining the predicted watermark information of the predicted image block under a single color channel based on the size relationship between each frequency domain coefficient in the target predicted frequency domain coefficient sequence; determining the watermark recognition result of the watermark to be recognized based on the predicted watermark information of the predicted image block under multiple single color channels.

[0007] In a third aspect, an embodiment of the present application provides a watermark embedding device, which includes: a first acquisition module, used to acquire a frequency domain coefficient sequence of a target image block in an image to be processed under a single color channel; a first selection module, used to select a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel; an adjustment module, used to adjust the frequency domain coefficients in the target frequency domain coefficient sequence according to the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, so as to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence; a first determination module, used to determine a reference image block corresponding to the target image block under a single color channel according to the adjusted frequency domain coefficient sequence; and a fusion module, used to fuse the target image block in the image to be processed and the reference image block corresponding to the target image block under a single color channel, so as to obtain a watermark image corresponding to the image to be processed.

[0008] Optionally, the adjusted frequency domain coefficient sequence includes adjusted frequency domain coefficients corresponding to each frequency domain coefficient in the target frequency domain coefficient sequence; the adjustment module is also used to traverse the frequency domain coefficients in the target frequency domain coefficient sequence according to a preset step size; for the first frequency domain coefficient traversed earlier and the second frequency domain coefficient traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information meet the adjustment conditions, determine the adjustment coefficient according to the first frequency domain coefficient and the second frequency domain coefficient; according to the adjustment coefficient, adjust the first frequency domain coefficient and the second frequency domain coefficient respectively to obtain the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient and the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0009] Optionally, the adjustment module is further used to determine the adjustment coefficient according to the frequency domain coefficient difference between the first frequency domain coefficient and the second frequency domain coefficient.

[0010] Optionally, the adjustment module is also used to determine the adjustment sub-coefficient based on the sum of the first frequency domain coefficient and the second frequency domain coefficient and the adjustment coefficient; adjust the first frequency domain coefficient according to the first frequency domain coefficient and the adjustment sub-coefficient to obtain the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient; adjust the second frequency domain coefficient according to the second frequency domain coefficient and the adjustment sub-coefficient to obtain the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0011] Optionally, the adjustment module is also used to use the first frequency domain coefficient as the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient, and use the second frequency domain coefficient as the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient, for the first frequency domain coefficient traversed earlier and the second frequency domain coefficient traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information do not meet the adjustment conditions.

[0012] Optionally, the fusion module is further used to fuse the reference image blocks corresponding to the target image block under multiple single color channels to obtain a fused reference image block corresponding to the target image block; determine the template image block corresponding to the target image block according to the difference between the target image block and the fused reference image block corresponding to the target image block; and perform weighted summation on the target image block and the template image block corresponding to the target image block based on a preset intensity parameter to obtain a watermark image corresponding to the image to be processed.

[0013] Optionally, the fusion module is further used to determine the difference between the target image block and the fused reference image block corresponding to the target image block as an intermediate image block corresponding to the target image block; and normalize the intermediate image block corresponding to the target image block to obtain a template image block corresponding to the target image block.

[0014] Optionally, the first determination module is also used to replace the target frequency domain coefficient sequence in the frequency domain coefficient sequence with the adjusted frequency domain coefficient sequence to obtain a replaced frequency domain coefficient sequence; determine the adjusted frequency domain map based on the replaced frequency domain coefficient sequence; transform the adjusted frequency domain map to the spatial domain to obtain a reference image block corresponding to the target image block under a single color channel.

[0015] In a fourth aspect, an embodiment of the present application provides a watermark recognition device, which includes: a second acquisition module, used to obtain a predicted frequency domain coefficient sequence of a predicted image block including a watermark to be recognized in a watermark image to be recognized under a single color channel; a second selection module, used to select a target predicted frequency domain coefficient sequence located in a preset position range from the predicted frequency domain coefficient sequence; a second determination module, used to determine the predicted watermark information of the predicted image block under a single color channel according to the size relationship between each frequency domain coefficient in the target predicted frequency domain coefficient sequence; and a result determination module, used to determine the watermark recognition result of the watermark to be recognized according to the predicted watermark information of the predicted image block under multiple single color channels.

[0016] Optionally, the second determination module is further used to traverse the predicted frequency domain coefficients in the target predicted frequency domain coefficient sequence according to a preset step size; for the first predicted frequency domain coefficient traversed earlier and the second predicted frequency domain coefficient traversed later in two adjacent traversals, determine the initial predicted watermark information corresponding to the two adjacent traversals according to the size relationship between the first predicted frequency domain coefficient and the second predicted frequency domain coefficient; determine the proportion of each initial predicted watermark information based on multiple initial predicted watermark information determined by multiple traversals; determine the predicted watermark information of the predicted image block under a single color channel based on the proportion of each initial predicted watermark information.

[0017] Optionally, the second acquisition module is also used to input the watermark image to be identified into the target detection model to obtain the predicted position information output by the target detection model and the predicted probability of the predicted position information; filter the target predicted position information whose predicted probability reaches the probability threshold in the predicted position information; obtain the area in the image to be identified that corresponds to the target predicted position information as the predicted image block; and determine the predicted frequency domain coefficient sequence of the predicted image block under a single color channel.

[0018] Optionally, the device also includes a training module, which is used to obtain a sample image including sample watermark information, sample position information corresponding to the sample watermark information, and a sample label corresponding to the sample position information; input the sample image into the initial detection model to obtain the sample prediction position information output by the initial detection model and the sample prediction probability of the sample prediction position; determine a first loss value based on the sample prediction position information and the sample position information; determine a second loss value based on the sample prediction probability and the sample label; and train the initial model based on the first loss value and the second loss value to obtain a target detection model.

[0019] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory; one or more computer programs are stored in the memory and configured to be executed by the processor to implement the above method.

[0020] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, wherein the above method is executed when the computer program is run by a processor.

[0021] In a seventh aspect, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the above-mentioned method.

[0022] The embodiments of the present application provide a watermark embedding method, device, electronic device and storage medium. In the present application, for a target image block in an image to be processed, a target frequency domain coefficient sequence located in a preset position range is selected from the frequency domain coefficient sequence of the target image block under a single color channel, and then based on the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, the frequency domain coefficients in the target frequency domain coefficient sequence are adjusted to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence. In this way, it is ensured that the adjusted frequency domain coefficient sequence carries the sub-watermark information corresponding to the target image block, and further ensures that the reference image block obtained based on the adjusted frequency domain coefficient sequence carries the corresponding sub-watermark information, thereby realizing the addition of the sub-watermark information to the target image block, and further obtaining a watermark image corresponding to the image to be processed. Moreover, in the present application, the target frequency domain coefficient sequence within a preset position range in the processed image is adjusted to obtain the watermark image, rather than adjusting all the frequency domain coefficients corresponding to the processed image. The adjustment amount of the frequency domain coefficients of the processed image is low and the adjustment amplitude is low, which reduces the influence of the watermark information embedded in the watermark image on the original image to be processed, thereby improving the image quality of the obtained watermark image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram showing an application scenario proposed in an embodiment of the present application;

[0025] Figure 2 A flow chart of a watermark embedding method proposed in one embodiment of the present application is shown;

[0026] Figure 3 A schematic diagram of an image to be processed in an embodiment of the present application is shown;

[0027] Figure 4 A schematic diagram showing a scanning process of a zigzag scanning method in an embodiment of the present application is shown;

[0028] Figure 5 A flow chart of a watermark embedding method proposed in another embodiment of the present application is shown;

[0029] Figure 6 A schematic diagram showing another image to be processed in an embodiment of the present application is shown;

[0030] Figure 7Shows Figure 6 Schematic diagram of the corresponding watermark image;

[0031] Figure 8 A flow chart of a watermark recognition method proposed in one embodiment of the present application is shown;

[0032] Fig. 9 A schematic diagram showing another image to be processed in an embodiment of the present application is shown;

[0033] Fig.10 A schematic diagram of a sample image in an embodiment of the present application is shown;

[0034] Fig.11 A schematic diagram showing a prediction result output by a target detection model in an embodiment of the present application is shown;

[0035] Fig.12 Shows Fig.11 Schematic diagram of the corresponding predicted image block;

[0036] Fig.13 A block diagram of a watermark embedding device proposed in one embodiment of the present application is shown;

[0037] Fig.14 A block diagram of an identification embedding device proposed in one embodiment of the present application is shown;

[0038] Fig.15 A structural block diagram of an electronic device for executing a watermark embedding method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. According to the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0042] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0043] The present application discloses a watermark embedding method, device, electronic device and storage medium, which involve artificial intelligence and cloud technology, etc.

[0044] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0045] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0046] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0047] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0048] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0049] like Figure 1 As shown, the application scenario applicable to the embodiment of the present application includes a terminal 20 and a server 10, and the terminal 20 and the server 10 are connected through a wired network or a wireless network. The terminal 20 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart home appliance, a vehicle terminal, an aircraft, a wearable device terminal, a virtual reality device, and other terminal devices that can display pages, or run other applications that can call page display applications (such as instant messaging applications, shopping applications, search applications, game applications, forum applications, map traffic applications, etc.).

[0050] The server 10 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 10 may be used to provide services for applications running on the terminal 20.

[0051] The terminal 20 can obtain the image to be processed and send the image to be processed to the server 10. The server 10 obtains the frequency domain coefficient sequence of the target image block in the image to be processed under the single color channel, and then the server 10 determines the reference image block corresponding to the target image block under the single color channel based on the frequency domain coefficient sequence of the target image block under the single color channel. The server 10 continues to fuse the target image block in the image to be processed and the reference image block corresponding to the target image block under the single color channel to obtain a watermark image corresponding to the image to be processed, and finally the server 10 returns the watermark image to the terminal.

[0052] The terminal 20 can also obtain a watermark image to be identified, and then send the watermark image to be identified to the server 10. The server 10 determines the predicted image block in the watermark image to be identified through the target detection model obtained through training, and determines the watermark recognition result of the watermark to be identified based on the predicted frequency domain coefficient sequence of the predicted image block under a single color channel. The server 10 then returns the watermark to be identified to the terminal 20.

[0053] As another implementation method, the terminal 20 can obtain the image to be processed, and then obtain the frequency domain coefficient sequence of the target image block in the image to be processed under a single color channel, and determine the reference image block corresponding to the target image block under the single color channel based on the frequency domain coefficient sequence of the target image block under the single color channel. Then, the terminal 20 fuses the target image block in the image to be processed and the reference image block corresponding to the target image block under the single color channel to obtain a watermark image corresponding to the image to be processed.

[0054] It can be understood that the terminal 20 can also obtain the watermark image to be identified, and continue to determine the predicted image block in the watermark image to be identified through the target detection model sent by the server. After that, the terminal 20 determines the watermark recognition result of the watermark to be identified based on the predicted frequency domain coefficient sequence of the predicted image block under a single color channel.

[0055] Among them, the server 10 can train the initial detection model according to the training samples to obtain the target detection model, and then the server 10 can store the target detection model in the distributed cloud storage system, and the terminal 20 obtains the target detection model from the distributed cloud storage system, and obtains the predicted image block through the target detection model.

[0056] For the convenience of description, in the following embodiments, watermark embedding performed by an electronic device is taken as an example for description.

[0057] See also Figure 2 , Figure 2 A flowchart of a watermark embedding method proposed in one embodiment of the present application is shown. The method can be applied to an electronic device, which can be Figure 1 The terminal 20 or the server 10 in the embodiment of the present invention comprises:

[0058] S110, obtaining a frequency domain coefficient sequence of a target image block in the image to be processed under a single color channel.

[0059] In this embodiment, the image to be processed may refer to an image to which watermark information is to be added, for example, the image to be processed may refer to a color image taken of a cat or a color image taken of a building. The pixel value of each pixel in the image to be processed may include sub-pixel values ​​under multiple single color channels, such as a red channel, a green channel, and a blue channel.

[0060] In one possible implementation, each video frame in the video to be processed can be obtained as an image to be processed, a corresponding watermark image can be determined for each video frame, and a video with watermark information added can be obtained based on the watermark image corresponding to each video frame, thereby achieving the addition of watermark information to the video to be processed.

[0061] In another possible implementation, some video frames can be selected from all video frames of the video to be processed as images to be processed, corresponding watermark images can be determined for the selected video frames, and based on the watermark images corresponding to the selected video frames and unselected video frames, a video with watermark information added can be obtained, in which some video frames of the video with watermark information have watermark information.

[0062] The target image block refers to the image block to be watermarked in the image to be processed. The watermarks added to different target image blocks in the image to be processed may be the same or different. At least one target image block may be selected in the image to be processed, and the sizes of the target image blocks may be the same. For example, 8×8 image blocks of the same size may be selected, and the size of each image block is 64×64.

[0063] The size information of the target image block may be determined according to the size of the image to be processed, and the target image block may be determined in the image to be processed according to the size information of the target image block. When the target image block includes multiple blocks, the multiple target image blocks may be irregularly or regularly distributed in the image to be processed. Figure 3 As shown, the image blocks to be processed include 16 target image blocks of the same size (each dotted box selected area is a target image block), and the 16 image blocks are distributed in a regular 4×4 pattern (the distance between adjacent target image blocks in each row is the same, and the distance between adjacent target image blocks in each column is the same) in the image to be processed.

[0064] For each target image block (an image block is also called a block), the target image block can be subjected to color channel stripping processing to obtain image blocks under each single color channel (a single color channel can be a red channel, a green channel or a blue channel), and the image block of the target image block under each single color channel is subjected to discrete cosine transform (Discrete Cosine Transform, abbreviated as DCT) to obtain a frequency domain image of the target image block under each single color channel. Afterwards, the frequency domain image of the target image block under each single color channel is scanned to obtain a frequency domain coefficient sequence of the target image block under each single color channel.

[0065] Among them, the frequency domain image of the target image block in each single color channel can be scanned by a zigzag scanning method (zigzag scanning method) to achieve one-dimensional expansion of the frequency domain image of the target image block in each single color channel to obtain a frequency domain coefficient sequence of the target image block in each single color channel.

[0066] The scanning process of the zigzag scanning method can be as follows Figure 4 As shown, in Figure 4In the figure, the frequency domain diagram includes 63 frequency domain coefficients (one square represents one frequency domain coefficient). Starting from the upper left end of the frequency domain diagram, each frequency domain coefficient is scanned in the order of the arrows, and the frequency domain coefficients are arranged in the scanning order to obtain the frequency domain coefficient sequence corresponding to the frequency domain diagram.

[0067] For example, for any target image block a1 in the image A to be processed, the target image block a1 is subjected to color channel stripping processing to obtain an image block b1 under the red channel, an image block b2 under the green channel, and an image block b3 under the blue channel. The image block b1 under the red channel is subjected to discrete cosine transform to obtain a frequency domain image c1, and then the frequency domain image c1 is scanned to obtain a frequency domain coefficient sequence d1 of the target image block a1 under the red channel. Similarly, the image block b2 under the green channel is subjected to discrete cosine transform to obtain a frequency domain image c2, and then the frequency domain image c1 is scanned to obtain a frequency domain coefficient sequence d2 of the target image block a1 under the green channel. Similarly, the image block b3 under the blue channel is subjected to discrete cosine transform to obtain a frequency domain image c3, and then the frequency domain image c3 is scanned to obtain a frequency domain coefficient sequence d3 of the target image block a1 under the green channel.

[0068] S120 , selecting a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel.

[0069] The preset position range may refer to the position range corresponding to the intermediate frequency frequency domain coefficient. Zigzag scanning is scanning from the upper left corner of the frequency domain diagram to the lower right corner in sequence, so the frequency domain coefficients scanned first will be lower and the frequency domain coefficients scanned later will be higher. Therefore, the preset position range in this application may refer to the frequency domain coefficients between the 533rd position and the 789th position in the frequency domain coefficient sequence, and the preset position range corresponds to 257 (789-532) frequency domain coefficients.

[0070] In this embodiment, only the frequency domain coefficients of the intermediate frequency domain are modified, so that the modification amplitude of the frequency domain coefficients is small, so that the added watermark information has little effect on the image quality of the original image to be processed, thereby improving the image quality of the watermarked image.

[0071] Frequency domain coefficients located in a preset position range can be selected from the frequency domain coefficient sequence, and the selected frequency domain coefficients can be arranged from small to large according to the position number of the frequency domain coefficient in the frequency domain coefficient sequence (the position number refers to the rank, for example, the position number 20 refers to the frequency domain coefficient ranked 20th) to obtain the target frequency domain coefficient sequence.

[0072] For the frequency domain coefficient sequence of the target image block under a single color channel, determine the target frequency domain coefficient sequence of each frequency domain coefficient sequence under each single color channel. For example, for the frequency domain coefficient sequence d1 of the target image block a1 under the red channel, select the target frequency domain coefficient sequence e1 located in a preset position range from the frequency domain coefficient sequence d1, for the frequency domain coefficient sequence d2 of the target image block a1 under the green channel, select the target frequency domain coefficient sequence e2 located in a preset position range from the frequency domain coefficient sequence d1, and for the frequency domain coefficient sequence d1 of the target image block a1 under the blue channel, select the target frequency domain coefficient sequence e1 located in a preset position range from the frequency domain coefficient sequence d1.

[0073] S130, adjusting the frequency domain coefficients in the target frequency domain coefficient sequence according to the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence.

[0074] The watermark information indicates the watermark to be embedded in the image to be processed. Since the watermark is added to the target image block in the image to be processed, the watermark information includes the sub-watermark information corresponding to each target image block. The watermark information is in the form of a watermark vector. The dimension of the watermark vector corresponds to the number of target image blocks. A value in the watermark vector is the sub-watermark information corresponding to a target image block. The sub-watermark information can be 1 or 0, for example.

[0075] A plurality of target image blocks may be determined in the image to be processed, and respective sub-watermark information may be set for each target image block, and the sub-watermark information of each target image block may be combined to obtain the watermark information of the image block to be processed.

[0076] Exemplarily, for different publishers, different watermark information can be obtained by setting different sub-watermark information for each target image block, so that the watermark information added by different publishers is different, thereby achieving the distinction between publishers through watermark information.

[0077] For example, the image to be processed includes 64 target image blocks, and the sub-watermark information corresponding to each target image block can be 1 or 0. Different publishers can select a sub-watermark information for each target image block according to their needs, thereby obtaining a 64-bit watermark information, and establishing an information mapping table between the publisher and the watermark information, so as to identify the publisher through the 64-bit watermark information. After decoding the image with the watermark information, the decoded watermark information is obtained, and the publisher can be determined through the decoded watermark information and the established information mapping table.

[0078] For example, Figure 3As shown, when the image blocks to be processed include 16 target image blocks of the same size, if it is determined that the sub-watermark information corresponding to the four target image blocks in the first row is all 1, the sub-watermark information corresponding to the four target image blocks in the second row is all 0, the sub-watermark information corresponding to the four target image blocks in the third row is all 0, and the sub-watermark information corresponding to the four target image blocks in the fourth row is all 1, then the watermark information can be {1,1,1,1,0,0,0,0,0,0,0,0,1,1,1,1}.

[0079] The target image block corresponds to the target frequency domain coefficient sequence under multiple single color channels respectively, and the adjusted frequency domain coefficient sequence of the target image block under each single color channel is determined according to the target frequency domain coefficient sequence under each single color channel and the sub-watermark information of the target image block. The adjusted frequency domain coefficient sequence includes the adjusted frequency domain coefficients corresponding to each frequency domain coefficient in the target frequency domain coefficient sequence, and the arrangement order of each adjusted frequency domain coefficient in the adjusted frequency domain coefficient sequence corresponds to the arrangement order of each frequency domain coefficient in the target frequency domain coefficient sequence. For example, the adjusted frequency domain coefficient corresponding to the mth frequency domain coefficient in the frequency domain coefficient sequence is arranged at the mth position in the adjusted frequency domain coefficient sequence, and m is a non-zero natural number.

[0080] The adjustment parameters can be determined according to the size relationship between the frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block, and the frequency domain coefficients in the target frequency domain coefficient sequence are summed, subtracted or multiplied with the adjustment parameters to obtain the adjusted frequency domain coefficients corresponding to each of the frequency domain coefficients in the target frequency domain coefficient sequence.

[0081] Exemplarily, for any two adjacent frequency domain coefficients in the target frequency domain coefficient sequence, if the first frequency domain coefficient of the two frequency domain coefficients is smaller than the second frequency domain coefficient and the sub-watermark information is 1, or, the first frequency domain coefficient of the two frequency domain coefficients is greater than the second frequency domain coefficient and the sub-watermark information is 0, the two frequency domain coefficients are operated to obtain adjustment parameters, and the two frequency domain coefficients are respectively operated with the adjustment parameters to obtain adjusted frequency domain coefficients corresponding to the two frequency domain coefficients; if the first frequency domain coefficient of the two frequency domain coefficients is smaller than the second frequency domain coefficient and the sub-watermark information is 0, or, the first frequency domain coefficient of the two frequency domain coefficients is greater than the second frequency domain coefficient and the sub-watermark information is 1, the two frequency domain coefficients themselves are used as the adjusted frequency domain coefficients corresponding to each of the two frequency domain coefficients.

[0082] S140 . Determine a reference image block corresponding to the target image block in a single color channel according to the adjusted frequency domain coefficient sequence.

[0083] According to the adjusted frequency domain coefficient sequence of the target image block in each single color channel, the reference image block of the target image block in each single color channel is determined. For example, according to the target frequency domain coefficient sequence e2 of the target image block a1 in the green channel, the adjusted frequency domain coefficient sequence f2 of the target image block a1 in the green channel is determined, and according to the adjusted frequency domain coefficient sequence f2, the reference image block of the target image block a1 in the green channel is determined.

[0084] In this embodiment, the target frequency domain coefficient sequence in the frequency domain coefficient sequence may be replaced by the adjusted frequency domain coefficient sequence to obtain a replaced frequency domain coefficient sequence; an adjusted frequency domain graph may be determined based on the replaced frequency domain coefficient sequence; the adjusted frequency domain graph may be transformed into the spatial domain to obtain a reference image block corresponding to the target image block under a single color channel.

[0085] For the frequency domain coefficient sequence of the target image block in each single color channel, the adjusted frequency domain coefficient sequence in the single color channel is used to replace the target frequency domain coefficient sequence in the single color channel to obtain the replaced frequency domain coefficient sequence of the target image block in each single color channel, and then the replaced frequency domain coefficient sequence in each single color channel is restored to a frequency domain image according to zigzag scanning to obtain the adjusted frequency domain image in each single color channel, and finally the adjusted frequency domain image in each single color channel is transformed into the spatial domain through IDCT (Inverse Discrete Cosine Transform) to obtain the reference image block in each single color channel.

[0086] For example, the target frequency domain coefficient sequence e1 in the frequency domain coefficient sequence d1 of the target image block a1 under the red channel is replaced with the adjusted frequency domain coefficient sequence f1 of the target image block a1 under the red channel to obtain a replacement frequency domain coefficient sequence g1 of the target image block a1 under the red channel, the replacement frequency domain coefficient sequence g1 is restored to obtain an adjusted frequency domain image of the target image block a1 under the red channel, and the adjusted frequency domain image of the target image block a1 under the red channel is subjected to IDCT transformation to obtain a reference image block of the target image block a1 under the red channel.

[0087] It can be understood that the frequency domain coefficients corresponding to other positions in the replacement frequency domain coefficient sequence except the adjustment frequency domain coefficient sequence are the same as the frequency domain coefficients corresponding to other positions in the frequency domain coefficient sequence of the target image block under a single color channel except the target frequency domain coefficient sequence. That is, it is equivalent to adjusting the frequency domain coefficients in the target frequency domain coefficient sequence in the frequency domain coefficient sequence of the target image block under a single color channel to obtain the replacement frequency domain coefficient sequence, without adjusting other frequency domain coefficients in the frequency domain coefficient sequence of the target image block under a single color channel.

[0088] S150, fusing a target image block in the image to be processed with a reference image block corresponding to the target image block in a single color channel to obtain a watermark image corresponding to the image to be processed.

[0089] After obtaining the reference image block corresponding to the target image block under a single color channel, the reference image blocks corresponding to the target image block under multiple single color channels can be fused first to obtain a fused reference image block corresponding to the target image block, and then the watermark image corresponding to the image to be processed can be obtained based on the target image block and the fused reference image block.

[0090] The watermark image may be obtained by superimposing the fused reference image block corresponding to the target image block with the target image block in the image to be processed. Superposition may refer to summing the pixel values ​​of the corresponding pixels in the fused reference image block and the target image block in the image to be processed. For example, if the fused reference image block and the target image block are both 64×64 image blocks, then the pixel value of the first pixel in the first row of the fused reference image block is summed with the pixel value of the first pixel in the first row of the target image block, and the summed result is used as the pixel value of the pixel corresponding to the first pixel in the first row of the target image block in the watermark image.

[0091] In this embodiment, for the target image block in the image to be processed, a target frequency domain coefficient sequence located in a preset position range is selected from the frequency domain coefficient sequence of the target image block under a single color channel, and then based on the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, the frequency domain coefficients in the target frequency domain coefficient sequence are adjusted to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence, so that the adjusted frequency domain coefficient sequence is guaranteed to carry the sub-watermark information corresponding to the target image block, and further ensures that the reference image block obtained based on the adjusted frequency domain coefficient sequence carries the corresponding sub-watermark information, thereby realizing the addition of the sub-watermark information to the target image block, and further obtaining the watermark image corresponding to the image to be processed. Moreover, in this embodiment, only the target frequency domain coefficient sequence in the preset position range in the image to be processed is adjusted to obtain the watermark image, rather than adjusting all the frequency domain coefficients corresponding to the image to be processed. The adjustment amount and amplitude of the frequency domain coefficients of the image to be processed are relatively low, which reduces the influence of the watermark information embedded in the watermark image on the original image to be processed, thereby improving the image quality of the obtained watermark image.

[0092] See also Figure 5 , Figure 5 A flowchart of a watermark embedding method proposed in another embodiment of the present application is shown. The method can be applied to an electronic device, which can be Figure 1 The terminal 20 or the server 10 in the embodiment of the present invention comprises:

[0093] S210, obtaining a frequency domain coefficient sequence of a target image block in the image to be processed under a single color channel; and selecting a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel.

[0094] The description of S210 refers to the description of S110 to S120 above, which will not be repeated here.

[0095] S220 , traverse the frequency domain coefficients in the target frequency domain coefficient sequence according to a preset step size.

[0096] The preset step size can be a step size set according to demand, for example, the preset step size can be 2, 3, 4, etc. The target frequency domain coefficient sequence can be traversed according to the preset step size, and a frequency domain coefficient is selected each time. For example, for the frequency domain coefficient sequence d2 of the target image block a1 under the green channel, the target frequency domain coefficient sequence e2 selected according to the preset position range includes 257 frequency domain coefficient sequences. The target frequency domain coefficient sequence is traversed according to the preset step size of 2, and the first frequency domain coefficient, the third frequency domain coefficient, the fifth frequency domain coefficient, ..., the 257th frequency domain coefficient in the target frequency domain coefficient sequence are selected, and a total of 129 frequency domain coefficients are obtained by traversing.

[0097] S230. For the first frequency domain coefficient that is traversed earlier and the second frequency domain coefficient that is traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information meet the adjustment condition, determine the adjustment coefficient according to the first frequency domain coefficient and the second frequency domain coefficient.

[0098] The adjustment condition may include: the first frequency domain coefficient is smaller than the second frequency domain coefficient and the sub-watermark information is 1, or the first frequency domain coefficient is larger than the second frequency domain coefficient and the sub-watermark information is 0.

[0099] When the adjustment condition is met, the adjustment coefficient may be determined according to the first frequency domain coefficient and the second frequency domain coefficient, wherein the adjustment coefficient is determined according to the frequency domain coefficient difference between the first frequency domain coefficient and the second frequency domain coefficient.

[0100] Exemplarily, the frequency domain coefficient difference between the second frequency domain coefficient and the first frequency domain coefficient may be determined, and the difference is subtracted from the preset value to obtain the adjustment coefficient. The calculation process may be expressed as Formula 1, which is as follows:

[0101]

[0102] Wherein, s2 is the second frequency domain coefficient, s1 is the first frequency domain coefficient, k is a preset constant, k can be 20, is the adjustment factor.

[0103] S240. Adjust the first frequency domain coefficient and the second frequency domain coefficient respectively according to the adjustment coefficient to obtain an adjusted frequency domain coefficient corresponding to the first frequency domain coefficient and an adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0104] After obtaining the adjustment coefficient, the adjustment frequency domain coefficient corresponding to the first frequency domain coefficient and the second frequency domain coefficient can be respectively operated by the adjustment coefficient to obtain the adjustment frequency domain coefficient corresponding to the first frequency domain coefficient and the adjustment frequency domain coefficient corresponding to the second frequency domain coefficient, wherein the operation can include summation, difference and product.

[0105] As an implementation method, S240 includes: determining an adjustment sub-coefficient based on the sum of the first frequency domain coefficient and the second frequency domain coefficient and the adjustment coefficient; adjusting the first frequency domain coefficient based on the first frequency domain coefficient and the adjustment sub-coefficient to obtain an adjusted frequency domain coefficient corresponding to the first frequency domain coefficient; adjusting the second frequency domain coefficient based on the second frequency domain coefficient and the adjustment sub-coefficient to obtain an adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0106] The sum of the first frequency domain coefficient and the second frequency domain coefficient can be calculated, and the adjustment coefficient is compared with the sum to obtain the adjustment sub-coefficient. Then, for the first frequency domain coefficient, the product of the first frequency domain coefficient and the adjustment sub-coefficient is calculated, and the difference between the first frequency domain coefficient and the product result is used as the adjustment frequency domain coefficient corresponding to the first frequency domain coefficient. The calculation process can be expressed as Formula 2, which is as follows:

[0107]

[0108] Among them, s'1 is the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient, s1 is the first frequency domain coefficient, s2 is the second frequency domain coefficient, is the adjustment factor, is the adjustment factor.

[0109] For the second frequency domain coefficient, the product of the second frequency domain coefficient and the adjustment sub-coefficient is calculated, and the difference between the second frequency domain coefficient and the product result is used as the adjustment frequency domain coefficient corresponding to the second frequency domain coefficient. The calculation process can be expressed as Formula 3, which is as follows:

[0110]

[0111] Wherein, s'2 is the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0112] S250. For the first frequency domain coefficient traversed earlier and the second frequency domain coefficient traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information do not meet the adjustment conditions, the first frequency domain coefficient is used as the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient, and the second frequency domain coefficient is used as the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0113] That is to say, if the first frequency domain coefficient is smaller than the second frequency domain coefficient and the sub-watermark information is 0, or, if the first frequency domain coefficient is greater than the second frequency domain coefficient and the sub-watermark information is 1, the first frequency domain coefficient and the second frequency domain coefficient are not adjusted, and the adjusted frequency domain coefficients corresponding to the first frequency domain coefficient and the second frequency domain coefficient are themselves.

[0114] S260 . Determine a reference image block corresponding to the target image block in a single color channel according to the adjusted frequency domain coefficient sequence.

[0115] The description of S260 may refer to the description of S140 above, and will not be repeated here.

[0116] S270 , fusing reference image blocks corresponding to the target image block under multiple single color channels to obtain a fused reference image block corresponding to the target image block.

[0117] The reference image blocks corresponding to the target image block under multiple single color channels are fused to obtain a fused color image block corresponding to the target image block as a fused reference image block.

[0118] S280 . Determine a template image block corresponding to the target image block according to a difference between the target image block and a fused reference image block corresponding to the target image block.

[0119] A difference may be made between the target image block and the fused reference image block corresponding to the target image block, and the difference may be obtained as a template image block corresponding to the target image block.

[0120] As an implementation method, the difference between the target image block and the fused reference image block corresponding to the target image block can also be determined as the intermediate image block corresponding to the target image block; the intermediate image block corresponding to the target image block is normalized to obtain the template image block corresponding to the target image block. The normalization process can refer to normalizing the pixel value of each pixel point in the intermediate image within 0 to 255.

[0121] S290 , based on a preset intensity parameter, weighted sum is performed on the target image block and the template image block corresponding to the target image block to obtain a watermark image corresponding to the image to be processed.

[0122] The strength parameter may refer to the strength of the watermark information. The larger the strength parameter, the more obvious the added watermark information. The smaller the strength parameter, the less obvious the added watermark information. In this application, the corresponding strength parameter may be set according to requirements. This application does not limit this. For example, the strength parameter may be 0.03.

[0123] The difference between 1 and the intensity parameter can be used as the weight of the target image block, and the intensity parameter can be used as the weight of the template image block corresponding to the target image block. The target image block and the template image block corresponding to the target image block can be weighted and summed by the weight of the target image block and the weight of the template image block corresponding to the target image block to obtain the watermark image block corresponding to the target image block; then, according to the position information of the target image block in the image to be processed, the watermark image block corresponding to the target image block can be spliced ​​with the image block without watermark in the image to be processed to obtain the watermark image corresponding to the image to be processed. At this time, the watermark image block in the watermark image = target image block * (1-intensity parameter) + template image block * intensity parameter.

[0124] In the obtained watermark image, the position corresponding to each target image block is a watermark block. The watermark block refers to the watermark block embedded in the spatial domain image (that is, the image to be processed). A watermark block can be used as a 1-bit information carrier, that is, a watermark block is an image block with a sub-watermark information added.

[0125] like Figure 6 As shown, Figure 6 is the image to be processed, Figure 7 for Figure 6 The corresponding watermark image, where the intensity parameter set when adding watermark information to the watermark image is 0.2, Figure 7 The watermark image in includes 8×8 watermark blocks, where Figure 7 The area in the white dotted box is a watermark block.

[0126] contrast Figure 6 and Figure 7 It can be seen that when the strength parameter is 0.2, the added watermark information has a higher visibility. Figure 7 This is just an example, so a larger intensity parameter is set to facilitate the observation of watermark information. If the intensity parameter is set to be much smaller than 0.2, the watermark in the obtained watermark image will be more difficult to be recognized by the naked eye, so that the quality of the obtained watermark image is better. Therefore, in the actual application process, a smaller intensity parameter can be set to obtain a watermark image with higher quality.

[0127] In this embodiment, a plurality of frequency domain coefficients are first traversed by a preset step size, and the template image block of the target image block is determined by adjusting the traversed frequency domain coefficients and the sub-watermark information in the set watermark information, so that the template image block can be changed according to the different sub-watermark information, thereby achieving the effect of customizing the watermark according to the needs.

[0128] At the same time, watermark images with different visual effects can be obtained according to different intensity parameters, and then a watermark image with less obvious watermark information can be obtained through a lower intensity parameter, thereby avoiding the influence of the watermark information on the image quality of the processed image and making the watermark image have higher quality.

[0129] See also Figure 8 , Figure 8 A flowchart of a watermark recognition method proposed in an embodiment of the present application is shown. The method can be applied to an electronic device, which can be Figure 1 The terminal 20 or the server 10 in the embodiment of the present invention comprises:

[0130] S310: Obtain a predicted frequency domain coefficient sequence of a predicted image block including a watermark to be identified in the watermarked image to be identified under a single color channel.

[0131] The watermark image to be identified may refer to an image with a watermark to be identified, and the watermark image to be identified may be obtained according to the watermark embedding method as described above. The specific process is described in the above embodiment and will not be repeated here.

[0132] The predicted image block refers to an image block including the watermark to be identified in the watermark image to be identified. The watermark image to be identified may include one or more predicted image blocks, and the watermarks in different predicted image blocks may be different or the same.

[0133] The predicted image block including the watermark to be identified in the watermark image to be identified may be determined by manual observation, and the predicted image block may be one or more. For example, the watermark image to be identified includes 16 predicted image blocks.

[0134] For each predicted image block, the predicted image block can be stripped of color channels to obtain single-channel predicted image blocks under each single color channel, and the single-channel predicted image blocks under each single color channel are subjected to discrete cosine transform to obtain a predicted frequency domain map of the predicted image block under each single color channel, and then the predicted frequency domain map of the predicted image block under each single color channel is scanned to obtain a predicted frequency domain coefficient sequence of the predicted image block under each single color channel, wherein the predicted frequency domain coefficient sequence includes multiple predicted frequency domain coefficients. The predicted frequency domain map of the predicted image block under each single color channel can be scanned in a zigzag scanning order.

[0135] As an implementation method, the watermark image to be identified can also be input into the target detection model to obtain the predicted position information output by the target detection model and the predicted probability of the predicted position information; the target predicted position information whose predicted probability reaches the probability threshold is screened in the predicted position information; the area corresponding to the target predicted position information in the image to be identified is obtained as the predicted image block; and the predicted frequency domain coefficient sequence of the predicted image block under a single color channel is determined. The predicted probability of the predicted position information refers to the probability that the pixel area (or image block) indicated by the predicted position information in the watermark image to be identified is embedded with a watermark.

[0136] The predicted position information may include the width, height, and coordinates of the prediction box output by the target detection model. The position of the prediction box can be determined based on the coordinates of the prediction box, and the area selected by the prediction box can be determined based on the width and height of the prediction box.

[0137] The probability threshold can be a value set based on demand, for example, the probability threshold can be 0.5. The area corresponding to the predicted position information that reaches the probability threshold is more likely to be embedded with a watermark, and the area selected by the prediction box in the predicted position information that reaches the probability threshold in the image to be identified is determined as the predicted image block; the area corresponding to the predicted position information that does not reach the probability threshold is less likely to include watermark information, and the predicted position information that does not reach the probability threshold is discarded.

[0138] In this embodiment, the acquisition process of the target detection model may include: acquiring a sample image including sample watermark information, sample position information corresponding to the sample watermark information, and a sample label corresponding to the sample position information; the sample position information is used to indicate the area where the sample watermark information is located in the sample image, and the sample label is used to indicate that the area corresponding to the sample position information in the sample image includes the sample watermark information; inputting the sample image into the initial detection model to obtain the sample prediction position information output by the initial detection model and the sample prediction probability of the sample prediction position; the sample prediction position information is used to indicate the area where the initial detection model predicts that the sample watermark information is located in the sample image, and the sample prediction probability is the probability that the initial detection model predicts that the area corresponding to the sample position information in the sample image includes the sample watermark information; determining a first loss value based on the sample prediction position information and the sample position information; determining a second loss value based on the sample prediction probability and the sample label; training the initial model based on the first loss value and the second loss value to obtain the target detection model.

[0139] Among them, the initial model can be a Yolo detection series network, a CNN (Convolutional Neural Network) detection network, or a Transformer-based detection network, etc.

[0140] Training samples can be obtained. Sample watermark information is added to the sample images in the training samples, and sample position information (including the width and height of the added sample detection box and the coordinates of the sample detection box, the sample detection box is used to select the watermark information in the sample image) and sample labels corresponding to the sample position information are added to the sample images. The sample label can be 1, indicating that the area corresponding to the sample position information includes the sample watermark information.

[0141] The sample prediction position information may include the width and height of the sample prediction detection box and the coordinates of the sample prediction detection box, and the sample prediction detection box is used to select the watermark information in the sample image. The mean square error loss value may be determined by the coordinates of the sample prediction detection box in the sample prediction position information and the coordinates of the sample detection box in the sample position information as the first loss value.

[0142] If the sample watermark included in the sample image includes multiple sub-sample watermarks, each sub-sample watermark corresponds to a sample prediction detection frame and a sample detection frame, and the mean square error loss value of each sub-sample watermark is determined according to the coordinates of the sample prediction detection frame in the sample prediction position information corresponding to each sub-sample watermark and the coordinates of the sample detection frame in the sample position information, and the mean square error loss values ​​of each sub-sample watermark are summed as the first loss value.

[0143] If each batch of training samples includes multiple sample images, the sum of the first loss values ​​of the sample images in the batch of training samples is determined as the first loss value of the batch of training samples.

[0144] The cross entropy loss value can be determined by the sample prediction probability and the sample label as the second loss value. If the sample watermark included in the sample image includes multiple sub-sample watermarks, each sub-sample watermark corresponds to a sample prediction probability and a sample label. According to the sample prediction probability and sample label corresponding to each sub-sample watermark, the cross entropy loss value of each sub-sample watermark is determined, and the cross entropy loss values ​​of each sub-sample watermark are summed as the second loss value.

[0145] If each batch of training samples includes multiple sample images, the sum of the second loss values ​​of each sample image in the batch of training samples is determined as the second loss value of the batch of training samples.

[0146] For each sample image, the first loss value and the second loss value of the sample image are summed to obtain the final loss value of the sample image, and the final loss values ​​of each sample image in each batch of training samples are summed to obtain the final loss value of the training samples in the batch. The initial detection model is trained using the final loss values ​​of the training samples in each batch until all batches of training samples are traversed to obtain the target detection model.

[0147] Afterwards, a test sample may be obtained. The sample images in the test sample are in the same form as the sample images in the training sample, and will not be described in detail here.

[0148] The prediction effect of the target detection model can be tested by testing samples. If the prediction effect of the target detection model is good, the target detection model can be applied. If the prediction effect of the target detection model is poor, the above training process can be repeated for the target detection model until the prediction effect of the target detection model is good, and a target detection model with good prediction effect is obtained. The prediction effect of the target detection model can be evaluated by AUC (Area Under Curve, ROC (receiver operating characteristic curve) area) and F1 score.

[0149] S320 , selecting a target prediction frequency domain coefficient sequence located in a preset position range from the prediction frequency domain coefficient sequence.

[0150] The predicted frequency domain coefficients within a preset position range can be selected from the predicted frequency domain coefficient sequence, and the selected predicted frequency domain coefficients can be arranged from small to large according to the position number of the predicted frequency domain coefficient in the predicted frequency domain coefficient sequence (the position number refers to the rank, for example, the position number 20 refers to the frequency domain coefficient ranked 20th) to obtain the target predicted frequency domain coefficient sequence.

[0151] S330 , determining predicted watermark information of the predicted image block under a single color channel according to a size relationship between frequency domain coefficients in a target predicted frequency domain coefficient sequence.

[0152] A target prediction frequency domain coefficient sequence of the predicted image block in each single color channel is obtained, and the predicted watermark information of the predicted image block in each single color channel is determined according to the size relationship between each frequency domain coefficient in the target prediction frequency domain coefficient sequence of the predicted image block in each single color channel.

[0153] A prediction sub-watermark information can be determined according to the size relationship between any two adjacent prediction frequency domain coefficients in the target prediction frequency domain coefficient sequence, and the proportion of each prediction sub-watermark information in the target prediction frequency domain coefficient sequence is counted, and the prediction watermark information of the target prediction frequency domain coefficient sequence is determined according to the prediction sub-watermark information with a larger proportion.

[0154] For example, if the previous predicted frequency domain coefficient of any two adjacent predicted frequency domain coefficients in the target predicted frequency domain coefficient sequence is greater than the next predicted frequency domain coefficient, the predicted sub-watermark information is determined to be 1; if the previous predicted frequency domain coefficient of any two adjacent predicted frequency domain coefficients in the target predicted frequency domain coefficient sequence is less than the next predicted frequency domain coefficient, the predicted sub-watermark information is determined to be -1. Afterwards, the proportion of -1 and 1 determined by the target predicted frequency domain coefficient sequence is counted. If the proportion of 1 is larger, the watermark 1 corresponding to 1 is determined as the predicted watermark information. If it is determined that the proportion of -1 is larger, the watermark 0 corresponding to -1 is determined as the predicted watermark information.

[0155] As another implementation, the predicted frequency domain coefficients in the target predicted frequency domain coefficient sequence may be traversed according to a preset step size; for the first predicted frequency domain coefficient traversed earlier and the second predicted frequency domain coefficient traversed later in two adjacent traversals, the initial predicted watermark information corresponding to the two adjacent traversals is determined according to the size relationship between the first predicted frequency domain coefficient and the second predicted frequency domain coefficient; based on multiple initial predicted watermark information determined by multiple traversals, the proportion of each initial predicted watermark information is determined; based on the proportion of each initial predicted watermark information, the predicted watermark information of the predicted image block under a single color channel is determined.

[0156] The method of traversing the target prediction frequency domain coefficient sequence is similar to the method of traversing the above-mentioned target frequency domain coefficient sequence, and will not be repeated here.

[0157] In this embodiment, when the first predicted frequency domain coefficient is smaller than the second predicted frequency domain coefficient, the initial predicted watermark information corresponding to two adjacent traversals can be determined as a first value (for example, -1); when the first predicted frequency domain coefficient is larger than the second predicted frequency domain coefficient, the initial predicted watermark information corresponding to two adjacent traversals can be determined as a second value (for example, 1); after completing the traversal of the target predicted frequency domain coefficient sequence, the proportion of the first value and the second value is determined, and the watermark corresponding to the value with a larger proportion is used as the predicted watermark information of the predicted image block under a single color channel.

[0158] For example, the first value is -1, the second value is 1, the watermark corresponding to the first value is 0, and the watermark corresponding to the second value is 1. The target prediction frequency domain coefficient sequence of the predicted image block under the red channel includes 257 prediction frequency domain coefficients, which are traversed according to the preset step size 2 to obtain 129 prediction frequency domain coefficients, and 128 pairs of first prediction frequency domain coefficients and second prediction frequency domain coefficients for comparison are determined according to the 129 prediction frequency domain coefficients, and the size relationship between each pair of first prediction frequency domain coefficients and second prediction frequency domain coefficients for comparison is determined, and according to the size relationship between each pair of first prediction frequency domain coefficients and second prediction frequency domain coefficients for comparison, it is determined that the first value -1 appears 120 times and the second value 1 appears 28 times, and it is determined that the first value -1 accounts for a large proportion, and it is determined that the watermark 0 corresponding to the first value -1 is the predicted watermark information 0 of the predicted image block under the red channel.

[0159] S340 , determining a watermark recognition result of a watermark to be recognized according to the predicted watermark information of the predicted image block under multiple single color channels.

[0160] The watermark image to be identified may include at least one predicted image block. For each predicted image block, the predicted watermark information of the predicted image block under multiple single color channels can be determined, and the proportion of each predicted watermark information can be counted. The predicted watermark information with a larger proportion of the predicted watermark information is used as the watermark recognition result of the predicted image block. Finally, the watermark recognition results of each predicted image block are summarized to obtain the watermark recognition result of the watermark to be identified.

[0161] For example, the watermark image to be identified includes 9 predicted image blocks. For the first predicted image block r1, according to steps S310-S330, it is determined that the predicted watermark information of the predicted image block r1 under the red channel is 1, the predicted watermark information under the green channel is 1, and the predicted watermark information under the blue channel is 0. It is determined that the predicted watermark information of 1 accounts for a large proportion, and the watermark recognition result of the predicted image block r1 is determined to be 1. Then, for the other 8 predicted image blocks, it is determined that the watermark recognition result of the predicted image block r2 is 1, the watermark recognition result of the predicted image block r3 is 1, the watermark recognition result of the predicted image block r4 is 0, the watermark recognition result of the predicted image block r5 is 0, the watermark recognition result of the predicted image block r6 is 1, the watermark recognition result of the predicted image block r7 is 0, the watermark recognition result of the predicted image block r8 is 1, and the watermark recognition result of the predicted image block r9 is 0. The watermark recognition results of all the predicted image blocks are summarized to obtain the watermark recognition result of the watermark to be identified as {1,1,1,0,0,1,0,1,0}.

[0162] In this embodiment, the predicted watermark information of the predicted image block under a single color channel can be accurately determined through the size relationship between each frequency domain coefficient in the target predicted frequency domain coefficient sequence, so that the accuracy of the watermark recognition result of the watermark to be identified is higher based on the predicted watermark information of the predicted image block under each single color channel.

[0163] At the same time, by determining the predicted image block based on the target detection model based on deep learning, even if the watermark image to be identified is an image that has been compressed, rotated, cropped, etc., the predicted image block can be accurately located, so that the accuracy of the watermark recognition result of the watermark to be identified is high. And by predicting the image block through the target detection model based on deep learning, it is not necessary to traverse the entire area of ​​the watermark image to be identified, which reduces the computational complexity and improves the efficiency of watermark recognition.

[0164] In order to explain the technical solution of the present application more clearly, the transaction processing method of the present application is explained below in conjunction with an exemplary scenario. In this scenario, the watermark information to be added to the image to be processed x1 is {1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0}, that is, 16 target image blocks need to be determined in the image to be processed x1, and the watermark to be identified of the watermarked image to be processed x2 is {1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0}, that is, 16 predicted image blocks also need to be identified in the watermarked image to be processed x2.

[0165] 1. The processing process of the image x1 to be processed.

[0166] like Fig. 9 As shown, a 4×4 array of target image blocks is determined in the image to be processed x1 (an area selected by a dotted box is a target image block), each square is a target image block, and the target image blocks can be arranged in order from left to right in each row, that is, the sub-watermark information corresponding to the second target image block in the second row is the sixth sub-watermark information 0 in the watermark information.

[0167] For the first target image block y1 in the first row, a color channel stripping process is performed on the target image block y1 to obtain an image block y11 of the target image block y1 under the red channel, an image block y12 of the target image block y1 under the green channel, and an image block y13 of the target image block y1 under the blue channel.

[0168] Perform discrete cosine transform on the image block y11 to obtain the frequency domain map corresponding to the image block y11, and scan the frequency domain map corresponding to the image block y11 according to zigzag scanning to obtain the frequency domain coefficient sequence y111 corresponding to the image block y11, and then determine the target frequency domain coefficient sequence y1111 from the frequency domain coefficient sequence y111 according to the preset position range. After that, traverse the target frequency domain coefficient sequence y1111 according to the preset step size of 2, and determine the adjusted frequency domain coefficient sequence y11111 corresponding to the image block y11 according to the first frequency domain coefficient traversed earlier and the second frequency domain coefficient traversed later in two adjacent traversals and the sub-watermark information 1, and then use the adjusted frequency domain coefficient sequence y11111 to replace the target frequency domain coefficient sequence y1111 in the frequency domain coefficient sequence y111 to obtain the replaced frequency domain coefficient sequence y111111. Afterwards, the replaced frequency domain coefficient sequence y111111 can be restored to the adjusted frequency domain image y1111111, and the adjusted frequency domain image y1111111 is transformed into the spatial domain through inverse discrete cosine transform to obtain the reference image block under the red channel.

[0169] Similarly, image block y12 and image block y13 are processed to obtain reference image blocks corresponding to image block y12 and image block y13 respectively. Then, the reference image block corresponding to image block y11, the reference image block corresponding to image block y12, and the reference image block corresponding to image block y13 are fused to obtain a colored fused reference image block y10 corresponding to the target image block y1.

[0170] Afterwards, the target image block y1 may be subtracted from the fused reference image block y10, and the subtraction result may be normalized to obtain a template image block y100.

[0171] According to the above process, all the target image blocks in the image to be processed are traversed to obtain the template image blocks corresponding to each target image block: template image block y100 corresponding to target image block y1, template image block y200 corresponding to target image block y2, template image block y300 corresponding to target image block y4, template image block y400 corresponding to target image block y5, template image block y500 corresponding to target image block y6, template image block y600 corresponding to target image block y7, template image block y800 corresponding to target image block y9 Corresponding template image block y800, template image block y900 corresponding to target image block y9, template image block y1000 corresponding to target image block y10, template image block y1100 corresponding to target image block y1200 corresponding to target image block y13, template image block y1300 corresponding to target image block y1400 corresponding to target image block y15, template image block y1500 corresponding to target image block y16, and template image block y1600 corresponding to target image block y16.

[0172] Finally, each target image block and the template image block corresponding to each target image block are fused by a preset intensity parameter to obtain a watermark image.

[0173] 2. The processing process of the watermark image x2 to be identified.

[0174] 2.1 Training process of target detection model

[0175] You can get the MS COCO 2017 dataset, and get 118287 images from the MS COCO 2017 dataset to build training samples, and get 5000 images from the MS COCO 2017 dataset to build test samples.

[0176] like Fig.10 As shown, the sample position information of the sample image x3 in the training sample is represented by a sample detection box (a dotted box is a sample detection box), and the value 1 in the sample detection box is the sample watermark information in the area selected by the sample detection box of the sample image x3.

[0177] Acquire a sample image including sample watermark information, sample position information corresponding to the sample watermark information, and a sample label corresponding to the sample position information; input the sample image into an initial detection model to obtain sample prediction position information output by the initial detection model and a sample prediction probability of the sample prediction position; determine a mean square error loss value as a first loss value based on the sample prediction position information and the sample position information; determine a cross entropy loss value as a second loss value based on the sample prediction probability and the sample label; train the initial model based on the first loss value and the second loss value to obtain the target detection model.

[0178] After obtaining the target detection model, the target detection model is tested using sample images in the test sample. The test result shows that the prediction effect of the target detection model meets the requirements and the recall rate can basically be close to 1. It is determined that the target detection model can be used to determine the predicted image block.

[0179] 2.2 Recognition of the watermark image x2

[0180] The watermark image to be identified is input into the target detection model to obtain the detection result output by the target detection model. The detection result is as follows: Fig.11 As shown in the figure, the detection result includes 17 predicted position information (displayed in the form of prediction boxes, one dotted box is a prediction box) and the prediction probability of each of the 17 predicted position information (the prediction probability refers to the number in the prediction box), and the prediction probability refers to the probability that the predicted position information includes watermark information. The predicted probability greater than 0.5 is determined as the target predicted position information, and the image block in the target predicted position information is used as the predicted image block. In this way, 16 predicted image blocks are determined.

[0181] The results of determining 16 predicted image blocks are as follows Fig.12 As shown, for the first predicted image block z1 in the first row, the predicted image block z1 can be subjected to color channel stripping processing to obtain an image block z11 of the predicted image block z1 under the red channel, an image block z12 of the predicted image block z1 under the green channel, and an image block z13 of the predicted image block z1 under the blue channel.

[0182] Then, a discrete cosine transform is performed on the image block z11 of the predicted image block z1 under the red channel to obtain the frequency domain map corresponding to the image block z11, and the frequency domain map corresponding to the image block z11 is scanned according to the zigzag scan to obtain the predicted frequency domain coefficient sequence z111 corresponding to the image block z11, and then the target predicted frequency domain coefficient sequence z1111 is determined from the predicted frequency domain coefficient sequence z111 according to the preset position range. After that, the target predicted frequency domain coefficient sequence z1111 is traversed according to the preset step size of 2, and the initial predicted watermark information corresponding to the two adjacent traversals is determined according to the size relationship between the first predicted frequency domain coefficient traversed earlier and the second predicted frequency domain coefficient traversed later in the two adjacent traversals. After completing the traversal of the target predicted frequency domain coefficient sequence z1111, it is determined that the proportion of the initial predicted watermark information 1 is relatively large, and the label 1 corresponding to the initial predicted watermark information 1 is determined as the predicted watermark information of the predicted image block z1 under the red channel.

[0183] In this way, the predicted watermark information of the predicted image block z1 in the green channel is determined to be 1 and the predicted watermark information of the predicted image block z1 in the blue channel is determined to be 0. The predicted watermark information 1 accounts for a large proportion, and the watermark recognition result of the predicted image block z1 is determined to be 1.

[0184] According to the above process, all the predicted image blocks in the watermark image to be identified are traversed to obtain the watermark recognition results corresponding to each predicted image block: watermark recognition result 1 corresponding to predicted image block z1, watermark recognition result 1 corresponding to predicted image block z2, watermark recognition result 1 corresponding to predicted image block z3, watermark recognition result 1 corresponding to predicted image block z4, watermark recognition result 0 corresponding to predicted image block z5, watermark recognition result 0 corresponding to predicted image block z6, watermark recognition result 0 corresponding to predicted image block z7, watermark recognition result 0 corresponding to predicted image block z8, watermark recognition result 1 corresponding to predicted image block z9, watermark recognition result 1 corresponding to predicted image block z10, watermark recognition result 1 corresponding to predicted image block z11, watermark recognition result 1 corresponding to predicted image block z12, watermark recognition result 0 corresponding to predicted image block z13, watermark recognition result 0 corresponding to predicted image block z14, watermark recognition result 0 corresponding to predicted image block z15, and watermark recognition result z0 corresponding to predicted image block z16.

[0185] Finally, the watermark recognition results corresponding to all predicted image blocks are summarized, and the recognition result of the watermark to be identified is {1,1,1,1,0,0,0,0,1,1,1,1,0,0,0,0}.

[0186] In this scenario, a new digital watermarking technology (a technology that hides information in a carrier, in this scenario the carrier refers to the image to be processed) is proposed. The watermark information is embedded in the form of discrete cosine transform, and the watermark encryption scheme is located through a deep learning eye detection model. The advantages of good quality and robustness of the watermark image are retained, and the complexity is extremely small, so it can be applied to actual watermark encryption scenarios.

[0187] The watermark information is embedded by modifying the frequency domain interval difference in the frequency domain, and only the intermediate frequency part is modified, so that the embedded watermark information has little impact on the image quality; in the watermark information detection stage (the stage of identifying the watermark), the deep learning target detection model is used to locate the position of the frequency domain watermark, and then the frequency domain similarity calculation is used to decode the watermark block information. In the case of small image quality loss, the problem that the frequency domain watermark algorithm is not resistant to resize attacks is solved. At the same time, watermark embedding and recognition are corresponding, and the embedded frequency domain difference can be parsed, and then the embedded watermark information can be decoded according to the frequency domain difference, realizing accurate recognition of watermark information.

[0188] In addition, if traditional methods are used to traverse all watermark images to be identified during the watermark information detection stage, the computational complexity will soar and the watermark recognition efficiency will be reduced. In this scenario, a deep learning target detection model is used to locate the predicted image block where the watermark is located, which reduces the computational complexity and improves the watermark recognition efficiency.

[0189] See also Fig.13 , Fig.13 A block diagram of a watermark embedding device proposed in an embodiment of the present application is shown, and the device 1300 includes:

[0190] The first acquisition module 1310 is used to acquire a frequency domain coefficient sequence of a target image block in the image to be processed under a single color channel;

[0191] A first selection module 1320 is used to select a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel;

[0192] An adjustment module 1330 is used to adjust the frequency domain coefficients in the target frequency domain coefficient sequence according to the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, so as to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence;

[0193] A first determination module 1340 is used to determine a reference image block corresponding to the target image block in a single color channel according to the adjusted frequency domain coefficient sequence;

[0194] The fusion module 1350 is used to fuse the target image block in the image to be processed with the reference image block corresponding to the target image block under a single color channel to obtain a watermark image corresponding to the image to be processed.

[0195] Optionally, the adjusted frequency domain coefficient sequence includes adjusted frequency domain coefficients corresponding to each frequency domain coefficient in the target frequency domain coefficient sequence; the adjustment module 1330 is also used to traverse the frequency domain coefficients in the target frequency domain coefficient sequence according to a preset step size; for the first frequency domain coefficient traversed earlier and the second frequency domain coefficient traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information meet the adjustment conditions, determine the adjustment coefficient according to the first frequency domain coefficient and the second frequency domain coefficient; according to the adjustment coefficient, adjust the first frequency domain coefficient and the second frequency domain coefficient respectively to obtain the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient and the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0196] Optionally, the adjustment module 1330 is further configured to determine an adjustment coefficient according to a frequency domain coefficient difference between the first frequency domain coefficient and the second frequency domain coefficient.

[0197] Optionally, the adjustment module 1330 is also used to determine the adjustment sub-coefficient based on the sum of the first frequency domain coefficient and the second frequency domain coefficient and the adjustment coefficient; adjust the first frequency domain coefficient based on the first frequency domain coefficient and the adjustment sub-coefficient to obtain the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient; adjust the second frequency domain coefficient based on the second frequency domain coefficient and the adjustment sub-coefficient to obtain the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

[0198] Optionally, the adjustment module 1330 is also used to use the first frequency domain coefficient as the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient, and use the second frequency domain coefficient as the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient, for the first frequency domain coefficient traversed earlier and the second frequency domain coefficient traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information do not meet the adjustment conditions.

[0199] Optionally, the fusion module 1350 is further used to fuse the reference image blocks corresponding to the target image block under multiple single color channels to obtain a fused reference image block corresponding to the target image block; determine the template image block corresponding to the target image block according to the difference between the target image block and the fused reference image block corresponding to the target image block; and perform weighted summation on the target image block and the template image block corresponding to the target image block based on a preset intensity parameter to obtain a watermark image corresponding to the image to be processed.

[0200] Optionally, the fusion module 1350 is further used to determine the difference between the target image block and the fused reference image block corresponding to the target image block as an intermediate image block corresponding to the target image block; and normalize the intermediate image block corresponding to the target image block to obtain a template image block corresponding to the target image block.

[0201] Optionally, the first determination module 1310 is also used to replace the target frequency domain coefficient sequence in the frequency domain coefficient sequence with the adjusted frequency domain coefficient sequence to obtain a replaced frequency domain coefficient sequence; determine the adjusted frequency domain map based on the replaced frequency domain coefficient sequence; transform the adjusted frequency domain map into the spatial domain to obtain a reference image block corresponding to the target image block under a single color channel.

[0202] See also Fig.14 , Fig.14 A block diagram of a watermark recognition device proposed in an embodiment of the present application is shown, and the device 1400 includes:

[0203] The second acquisition module 1410 is used to acquire a prediction frequency domain coefficient sequence of a prediction image block including a watermark to be identified in the watermark image to be identified under a single color channel;

[0204] The second selection module 1420 is used to select a target prediction frequency domain coefficient sequence located in a preset position range from the prediction frequency domain coefficient sequence;

[0205] The second determination module 1430 is used to determine the predicted watermark information of the predicted image block under a single color channel according to the size relationship between the frequency domain coefficients in the target predicted frequency domain coefficient sequence;

[0206] The result determination module 1440 is used to determine the watermark recognition result of the watermark to be recognized according to the predicted watermark information of the predicted image block under multiple single color channels.

[0207] Optionally, the second determination module 1430 is further used to traverse the predicted frequency domain coefficients in the target predicted frequency domain coefficient sequence according to a preset step size; for the first predicted frequency domain coefficient traversed earlier and the second predicted frequency domain coefficient traversed later in two adjacent traversals, determine the initial predicted watermark information corresponding to the two adjacent traversals according to the size relationship between the first predicted frequency domain coefficient and the second predicted frequency domain coefficient; determine the proportion of each initial predicted watermark information based on multiple initial predicted watermark information determined by multiple traversals; determine the predicted watermark information of the predicted image block under a single color channel based on the proportion of each initial predicted watermark information.

[0208] Optionally, the second acquisition module 1410 is also used to input the watermark image to be identified into the target detection model to obtain the predicted position information output by the target detection model and the predicted probability of the predicted position information; filter the target predicted position information whose predicted probability reaches the probability threshold in the predicted position information; obtain the area in the image to be identified that corresponds to the target predicted position information as the predicted image block; and determine the predicted frequency domain coefficient sequence of the predicted image block under a single color channel.

[0209] Optionally, the device also includes a training module, which is used to obtain a sample image including sample watermark information, sample position information corresponding to the sample watermark information, and a sample label corresponding to the sample position information; input the sample image into the initial detection model to obtain the sample prediction position information output by the initial detection model and the sample prediction probability of the sample prediction position; determine a first loss value based on the sample prediction position information and the sample position information; determine a second loss value based on the sample prediction probability and the sample label; and train the initial model based on the first loss value and the second loss value to obtain a target detection model.

[0210] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0211] Fig.15 The structural block diagram of an electronic device for executing the watermark embedding method according to an embodiment of the present application is shown. The electronic device may be Figure 1 The terminal 20 or the server 10, etc., need to be explained that, Fig.15 The computer system 1500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0212] like Fig.15 As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1502 or the program loaded from the storage part 1508 to the random access memory (RAM) 1503, such as executing the method in the above embodiment. In RAM 1503, various programs and data required for system operation are also stored. CPU 1501, ROM 1502 and RAM 1503 are connected to each other through bus 1504. Input / output (I / O) interface 1505 is also connected to bus 1504.

[0213] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, etc.; an output section 1507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed so that a computer program read therefrom is installed into the storage section 1508 as needed.

[0214] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1509, and / or installed from a removable medium 1511. When the computer program is executed by a central processing unit (CPU) 1501, various functions defined in the system of the present application are executed.

[0215] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0216] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0217] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0218] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device. The above computer-readable storage medium carries computer-readable instructions, and when the computer-readable storage instructions are executed by a processor, the method in any of the above embodiments is implemented.

[0219] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method in any of the above embodiments.

[0220] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0221] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable an electronic device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0222] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. It should be understood that the present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A watermark embedding method, characterized in that: The method comprises: Obtaining a frequency domain coefficient sequence of a target image block in a to-be-processed image under a single color channel; Selecting a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel; According to the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, the frequency domain coefficients in the target frequency domain coefficient sequence are adjusted to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence; Determining a reference image block corresponding to the target image block under a single color channel according to the adjusted frequency domain coefficient sequence; The target image block in the image to be processed and the reference image block corresponding to the target image block under a single color channel are fused to obtain a watermark image corresponding to the image to be processed.

2. The method according to claim 1, characterized in that The adjusted frequency domain coefficient sequence includes the adjusted frequency domain coefficients corresponding to each frequency domain coefficient in the target frequency domain coefficient sequence; The step of adjusting the frequency domain coefficients in the target frequency domain coefficient sequence according to the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence includes: Traversing the frequency domain coefficients in the target frequency domain coefficient sequence according to a preset step size; For a first frequency domain coefficient that is traversed earlier and a second frequency domain coefficient that is traversed later in two adjacent traversals, if a magnitude relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information meet an adjustment condition, determining an adjustment coefficient according to the first frequency domain coefficient and the second frequency domain coefficient; The first frequency domain coefficient and the second frequency domain coefficient are adjusted respectively according to the adjustment coefficient to obtain an adjusted frequency domain coefficient corresponding to the first frequency domain coefficient and an adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

3. The method according to claim 2, characterized in that The determining the adjustment coefficient according to the first frequency domain coefficient and the second frequency domain coefficient includes: An adjustment coefficient is determined according to a frequency domain coefficient difference between the first frequency domain coefficient and the second frequency domain coefficient.

4. The method according to claim 2, characterized in that: The adjusting the first frequency domain coefficient and the second frequency domain coefficient respectively according to the adjustment coefficient to obtain the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient and the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient includes: Determine an adjustment sub-coefficient according to the sum of the first frequency domain coefficient and the second frequency domain coefficient and the adjustment coefficient; Adjusting the first frequency domain coefficient according to the first frequency domain coefficient and the adjustment sub-coefficient to obtain an adjusted frequency domain coefficient corresponding to the first frequency domain coefficient; The second frequency domain coefficient is adjusted according to the second frequency domain coefficient and the adjustment sub-coefficient to obtain an adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

5. The method according to claim 2, characterized in that: After traversing the frequency domain coefficients in the target frequency domain coefficient sequence according to the preset step length, the method further includes: For the first frequency domain coefficient that is traversed first and the second frequency domain coefficient that is traversed later in two adjacent traversals, if the size relationship between the first frequency domain coefficient and the second frequency domain coefficient and the sub-watermark information do not meet the adjustment condition, the first frequency domain coefficient is used as the adjusted frequency domain coefficient corresponding to the first frequency domain coefficient, and the second frequency domain coefficient is used as the adjusted frequency domain coefficient corresponding to the second frequency domain coefficient.

6. The method according to claim 1, characterized in that The step of fusing the target image block in the image to be processed with the reference image block corresponding to the target image block under a single color channel to obtain a watermark image corresponding to the image to be processed includes: Fusing reference image blocks corresponding to the target image block under multiple single color channels to obtain a fused reference image block corresponding to the target image block; Determining a template image block corresponding to the target image block according to a difference between the target image block and a fused reference image block corresponding to the target image block; Based on a preset intensity parameter, a weighted sum is performed on the target image block and a template image block corresponding to the target image block to obtain a watermark image corresponding to the image to be processed.

7. The method according to claim 6, characterized in that The step of determining the template image block corresponding to the target image block according to the difference between the target image block and the fused reference image block corresponding to the target image block comprises: Determine a difference between the target image block and a fused reference image block corresponding to the target image block as an intermediate image block corresponding to the target image block; Normalization processing is performed on the intermediate image block corresponding to the target image block to obtain a template image block corresponding to the target image block.

8. The method according to claim 1, characterized in that The step of determining, according to the adjusted frequency domain coefficient sequence, a reference image block corresponding to the target image block under a single color channel comprises: Replacing a target frequency domain coefficient sequence in the frequency domain coefficient sequence with the adjusted frequency domain coefficient sequence to obtain a replacement frequency domain coefficient sequence; Determining an adjusted frequency domain map according to the replaced frequency domain coefficient sequence; The adjusted frequency domain image is transformed into the spatial domain to obtain a reference image block corresponding to the target image block under a single color channel.

9. A watermark recognition method, characterized in that: The method comprises: Acquire a prediction frequency domain coefficient sequence of a prediction image block including a watermark to be identified in the watermark image to be identified under a single color channel; Selecting a target prediction frequency domain coefficient sequence located in a preset position range from the prediction frequency domain coefficient sequence; Determining predicted watermark information of the predicted image block under a single color channel according to a size relationship between each frequency domain coefficient in the target predicted frequency domain coefficient sequence; A watermark recognition result of the watermark to be recognized is determined according to the predicted watermark information of the predicted image block under multiple single color channels.

10. The method according to claim 9, characterized in that The step of determining the predicted watermark information of the predicted image block under a single color channel according to the size relationship between the frequency domain coefficients in the target predicted frequency domain coefficient sequence includes: Traversing the predicted frequency domain coefficients in the target predicted frequency domain coefficient sequence according to a preset step size; For the first predicted frequency domain coefficient that is traversed earlier and the second predicted frequency domain coefficient that is traversed later in two adjacent traversals, determining initial predicted watermark information corresponding to the two adjacent traversals according to a magnitude relationship between the first predicted frequency domain coefficient and the second predicted frequency domain coefficient; Determine the proportion of each of the initial predicted watermark information according to the multiple initial predicted watermark information determined by the multiple traversals; According to the proportion of each of the initial predicted watermark information, the predicted watermark information of the predicted image block in a single color channel is determined.

11. The method according to claim 9, characterized in that The step of obtaining a predicted frequency domain coefficient sequence of a predicted image block including a watermark to be identified in the watermark image to be identified under a single color channel comprises: Inputting the watermark image to be identified into a target detection model to obtain predicted position information output by the target detection model and a predicted probability of the predicted position information; Filtering the target predicted position information whose predicted probability reaches a probability threshold from the predicted position information; Acquire a region in the image to be identified that corresponds to the target predicted position information as the predicted image block; Determine a prediction frequency domain coefficient sequence of the prediction image block under a single color channel.

12. The method according to claim 11, characterized in that Before inputting the to-be-identified watermark image into a target detection model to obtain the predicted position information output by the target detection model and the predicted probability of the predicted position information, the method further includes: Acquire a sample image including sample watermark information, sample position information corresponding to the sample watermark information, and a sample label corresponding to the sample position information; Inputting the sample image into an initial detection model to obtain sample prediction position information output by the initial detection model and sample prediction probability of the sample prediction position; Determining a first loss value according to the sample prediction position information and the sample position information; Determining a second loss value according to the sample prediction probability and the sample label; The initial model is trained according to the first loss value and the second loss value to obtain the target detection model.

13. A watermark embedding device, characterized in that: The device comprises: The first acquisition module is used to acquire a frequency domain coefficient sequence of a target image block in the image to be processed under a single color channel; A first selection module is used to select a target frequency domain coefficient sequence located in a preset position range from the frequency domain coefficient sequence of the target image block under a single color channel; An adjustment module, configured to adjust the frequency domain coefficients in the target frequency domain coefficient sequence according to the size relationship between different frequency domain coefficients in the target frequency domain coefficient sequence and the sub-watermark information corresponding to the target image block in the watermark information, so as to obtain an adjusted frequency domain coefficient sequence corresponding to the target frequency domain coefficient sequence; A first determination module, configured to determine a reference image block corresponding to the target image block under a single color channel according to the adjusted frequency domain coefficient sequence; The fusion module is used to fuse the target image block in the image to be processed with the reference image block corresponding to the target image block under a single color channel to obtain a watermark image corresponding to the image to be processed.

14. A watermark recognition device, characterized in that: The device comprises: The second acquisition module is used to acquire a prediction frequency domain coefficient sequence of a prediction image block including a watermark to be identified in the watermark image to be identified under a single color channel; A second selection module is used to select a target prediction frequency domain coefficient sequence located in a preset position range from the prediction frequency domain coefficient sequence; A second determination module is used to determine the predicted watermark information of the predicted image block under a single color channel according to the size relationship between the frequency domain coefficients in the target predicted frequency domain coefficient sequence; The result determination module is used to determine the watermark recognition result of the watermark to be recognized according to the predicted watermark information of the predicted image block under multiple single color channels.

15. An electronic device, characterized in that: include: one or more processors; Memory; One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, the one or more computer programs being configured to perform the method according to any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be called by a processor to execute the method according to any one of claims 1 to 12.

Images