Watermarking methods, devices, electronic devices and readable storage media

Abstract

This application discloses a watermarking method, apparatus, electronic device, and readable storage medium, belonging to the field of image processing technology. The watermarking method of this application includes: acquiring a target video segment, the target video segment comprising multiple frames of images; determining a target watermark pre-addition position in the multiple frames of images; determining color feature information of a target watermark pre-addition area, the target watermark pre-addition area being a watermark pre-addition area in the multiple frames of images corresponding to the target watermark pre-addition position; generating a target watermark identifier based on the color feature information; and adding the target watermark identifier to the target watermark pre-addition position in the multiple frames of images. Thus, watermark identifiers can be dynamically added to videos, thereby improving the protective effect of watermark identifiers.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a watermarking method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] In existing technologies, when adding watermarks to videos, the watermark is usually placed in a fixed location, such as the upper right corner. In this case, the watermark can be easily removed by cropping, failing to protect video copyright. Therefore, the existing methods of adding watermarks offer poor protection. Summary of the Invention

[0003] The purpose of this application is to provide a watermarking method, apparatus, electronic device, and readable storage medium to solve the problem that watermarks added in the existing way have poor protective effects.

[0004] To solve the above-mentioned technical problems, this application is implemented as follows:

[0005] Firstly, a watermark addition method is provided, including:

[0006] Acquire a target video segment, which includes multiple frames of images;

[0007] Determine the target watermark pre-addition position in the multi-frame images;

[0008] Determine the color feature information of the target watermark pre-addition area, wherein the target watermark pre-addition area is the watermark pre-addition area in the multi-frame image corresponding to the target watermark pre-addition position;

[0009] Based on the color feature information, a target watermark identifier is generated;

[0010] The target watermark identifier is added to the target watermark pre-addition position in the multi-frame images respectively.

[0011] Secondly, a watermark adding device is provided, comprising:

[0012] The acquisition module is used to acquire a target video segment, which includes multiple frames of images;

[0013] The first determining module is used to determine the target watermark pre-addition position in the multi-frame images;

[0014] The second determining module is used to determine the color feature information of the target watermark pre-addition area, wherein the target watermark pre-addition area is the watermark pre-addition area in the multi-frame image corresponding to the target watermark pre-addition position;

[0015] The generation module is used to generate a target watermark identifier based on the color feature information;

[0016] An adding module is used to add the target watermark identifier to the target watermark pre-addition position in the multi-frame images respectively.

[0017] Thirdly, an electronic device is provided, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0018] Fourthly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0019] In this embodiment, after acquiring the target video segment, the pre-position of the target watermark in the multiple frames of images included in the target video segment can be determined, and the color feature information of the target watermark pre-position area can be determined. The target watermark pre-position area is the watermark pre-position area in the multiple frames of images corresponding to the target watermark pre-position. Based on the color feature information, a target watermark identifier is generated, and the target watermark identifier is added to the target watermark pre-position in the multiple frames of images respectively. Therefore, the watermark addition position can be randomly changed based on the video image content, thereby dynamically adding watermark identifiers in the corresponding video and improving the protection effect of the watermark identifier. Attached Figure Description

[0020] Figure 1 This is a flowchart of a watermark addition method provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the connection between adjacent target boxes in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the region division in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the flipping process in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of a watermark adding device provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0028] The watermarking method, apparatus, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0029] Please see Figure 1 , Figure 1 This is a flowchart illustrating a watermarking method provided in an embodiment of this application. The method is applied to an electronic device, which may be a video server or a client. Figure 1 As shown, the method includes the following steps:

[0030] Step 11: Obtain the target video segment, which includes multiple frames of images.

[0031] Step 12: Determine the target watermark pre-addition position in the multi-frame images. The target watermark pre-addition position is the same in the multi-frame images, which can be understood as the watermark pre-addition position of the target video segment.

[0032] Step 13: Determine the color feature information of the target watermark pre-addition area, which is the watermark pre-addition area in the multi-frame image corresponding to the target watermark pre-addition position. The target watermark pre-addition area is the same in the multi-frame image and can be understood as the watermark pre-addition area of ​​the target video segment.

[0033] Step 14: Generate the target watermark identifier based on the color feature information.

[0034] Step 15: Add the target watermark identifier to the target watermark pre-addition position in the multi-frame images respectively.

[0035] In some embodiments, the target video segment is a video segment of a preset time T. This preset time T can be pre-set based on actual needs, such as 4 seconds, 5 seconds, etc., and is not limited thereto. If the video frame rate is F, then the target video segment includes T*F frames.

[0036] In some embodiments, the size of the target watermark pre-addition area can be determined based on the size of the watermark identifier to be added, such as being slightly larger than the size of the watermark identifier, for example, by enlarging the size of the watermark identifier to be added by 1.2 times to obtain the target watermark pre-addition area.

[0037] In some embodiments, when generating the target watermark identifier, the obtained color feature information can be used to perform targeted personalized processing on the initial watermark identifier, making the generated target watermark identifier less likely to be recognized by deep learning-based adversarial neural networks and the like.

[0038] Here, by adding the target watermark identifier to the pre-added area of ​​the target watermark in multiple frames of images, the target watermark identifier can be added to the target video segment, thus completing the watermark identifier generation in the target video segment. The target video segment can be understood as the basic video unit for adding the watermark identifier, and also the time unit for watermark position changes. It can be selected as any video segment of the corresponding video. For example, assuming the duration of the target video segment is 4 seconds, it means that within this 4-second time period, the watermark position remains unchanged at a specific location. Then, when entering the next video segment, the watermark position will change or disappear once.

[0039] The watermarking method of this application embodiment, after acquiring the target video segment, can determine the target watermark pre-addition position in the multiple frames of images included in the target video segment, and determine the color feature information of the target watermark pre-addition area. The target watermark pre-addition area is the watermark pre-addition area in the multiple frames of images corresponding to the target watermark pre-addition position. Based on the color feature information, a target watermark identifier is generated, and the target watermark identifier is added to the target watermark pre-addition position in the multiple frames of images respectively. Therefore, the watermark addition position can be randomly changed based on the video image content, thereby achieving dynamic addition of the watermark identifier. This prevents the watermark identifier from being removed by simple cropping, improving the protection effect of the watermark identifier.

[0040] Furthermore, by generating a target watermark identifier to be added to the target video segment based on the color feature information of the target watermark pre-addition area, the watermark identifier can be intelligently color-changed to ensure that the watermark identifier is integrated into the target video frame as much as possible while maintaining visual distinguishability. This avoids being discriminated and cracked by deep learning-based adversarial neural networks and ensures the security of the watermark identifier.

[0041] In this embodiment of the application, the target watermark pre-addition position in the multi-frame images can be determined based on the content recognition of the multi-frame images in the target video segment, so as to randomly change the watermark addition position.

[0042] Optionally, the process of determining the target watermark pre-addition position in multiple frames of images in step 12 above may include: first, detecting the target object in each frame of the multiple frames; then, determining the target watermark pre-addition position in the multiple frames based on the detected target object in each frame, wherein the target watermark pre-addition position does not overlap with the position of the detected target object. In this way, the target watermark pre-addition position can be determined based on the content detection of the video image, allowing the watermark addition position to change randomly based on the video image content, thus dynamically adding a watermark identifier to the corresponding video, and the watermark addition position does not affect the video content or the viewing experience.

[0043] In some embodiments, convolutional neural network (CNN) deep learning models can be used to detect target objects in images, including but not limited to face recognition and object recognition. For example, the target objects to be detected can include common objects in daily life such as cars, cats, dogs, and bicycles, and the breadth of object types can be expanded.

[0044] Optionally, the process of determining the target watermark pre-position in the multi-frame images may include: First, generating N target boxes in each frame image based on the target objects detected in each frame image; wherein, the target boxes correspond one-to-one with the detected target objects, that is, the number of target boxes generated corresponds to the number of detected target objects, and the number of target boxes in different frames image can be the same or different, the target boxes enclose the corresponding target objects, and N is an integer greater than or equal to 0; Second, arranging the N target boxes in each frame image in descending order of area; Then, for the arranged N target boxes, starting from the target box with the largest area, sequentially constructing lines between adjacent target boxes until there is an intersection point between the newly constructed line and the previously constructed line located outside the N target boxes; Then, determining the watermark candidate position in each frame image based on the position of the intersection point, wherein the watermark candidate position is located outside the N target boxes, that is, the watermark candidate position is not located within the N target boxes; Finally, determining the target watermark pre-position in the multi-frame images based on the watermark candidate position in each frame image. This allows for a simple and quick determination of the intersection points located outside of N target boxes for each frame of the image, thus ensuring that the watermark candidate positions are outside the target boxes and guaranteeing that the watermark markers added later do not affect the content expression of the corresponding video frame image.

[0045] Optionally, if the aforementioned adjacent target boxes include a first target box and a second target box, the lines connecting the constructed adjacent target boxes may include at least one of the following:

[0046] The line connecting the top left point of the first target box and the bottom right point of the second target box;

[0047] The line connecting the upper right point of the first bounding box and the lower left point of the second bounding box.

[0048] For example, when constructing lines connecting adjacent target boxes, one can first construct the line connecting the top-left and bottom-right points of adjacent target boxes. If an intersection point outside the target box is obtained, the process stops. If no intersection point outside the target box is obtained, the line connecting the top-right and bottom-left points of adjacent target boxes is constructed sequentially until an intersection point outside the target box is obtained. Alternatively, one can first construct the line connecting the top-right and bottom-left points of adjacent target boxes. If an intersection point outside the target box is obtained in either construction process, the process stops. If no intersection point outside the target box is obtained in either process, it can be determined that there is no watermark candidate location in the corresponding frame image.

[0049] Optionally, when determining the candidate watermark locations in each frame of the image based on the determined intersection points, any of the following may be included:

[0050] When the intersection point contains one intersection point, the position of the one intersection point is determined as the watermark candidate position in each frame of the image;

[0051] When the intersection point contains multiple intersection points, the position of the first intersection point is determined as the watermark candidate position in each frame image, and the first intersection point is any one of the multiple intersection points.

[0052] Understandably, ideally, a watermark candidate location should be determined in each frame of the image. However, in extreme cases, if the above construction process fails to obtain an intersection point outside the region where the target box is located, or if the number of detected target objects is small (i.e., the number of generated target boxes is small, such as 1 or 0), insufficient to form a line to obtain an intersection point outside the region where the target box is located, then it can be determined that there is no watermark candidate location in the corresponding frame of the image, or that any location can be used as a watermark candidate location.

[0053] For example, suppose the total number of faces and objects detected in a certain frame of an image is N, i.e., N bounding boxes are generated. These N bounding boxes are arranged in descending order of area, denoted as N1, N2, ..., Ni, Ni+1, ..., Nn, where area Ni > Ni+1, and n ∈ (1, N). Then, when determining the candidate watermark location in this frame of the image, we can: starting from N1, construct lines connecting adjacent bounding boxes sequentially. Here, "adjacent" refers to the adjacent bounding boxes arranged by area, not their position. This connection is between the upper left point of bounding box Ni and the lower right point of bounding box Ni+1. Figure 2 As shown by the solid line, it is determined whether the newly constructed connection intersects with the existing connection outside the region of the target box. If an intersection exists outside the region of the target box, the process ends, and the candidate watermark position is determined based on the position of this intersection. However, if no intersection outside the region of the target box is found after connecting all adjacent target boxes, the connection between adjacent target boxes can be reconstructed sequentially starting from N1. This connection is the connection between the upper right point of target box Ni and the lower left point of target box Ni+1. Figure 2 As shown by the dashed line, it determines whether the newly constructed line intersects with an existing line outside the target box region. If an intersection exists outside the target box region, the process ends. In extreme cases, if neither of the two construction processes yields an intersection outside the target box region, or if the number of detected faces and objects is small (i.e., the number of generated target boxes is small, such as 1 or 0), insufficient to form a line to obtain an intersection outside the target box region, then it can be determined that there is no watermark candidate location in the corresponding frame image, or that any location can be used as a watermark candidate location.

[0054] It should be noted that the above method for determining the candidate watermark position is only an example. Other methods can also be used to determine the candidate watermark position in each frame of the image, and this application does not limit this.

[0055] Optionally, the process of determining the target watermark pre-addition position based on the watermark candidate positions in each frame of the image may include: First, determining multiple watermark candidate positions based on the watermark candidate positions in each frame of the image; for example, if there are T*F frames of images, and each frame of the image has one watermark candidate position, then T*F watermark candidate positions can be determined; then, the center position corresponding to the multiple watermark candidate positions is determined as the target watermark pre-addition position in the multiple frames of images, that is, determined as the target watermark pre-addition position of the target video segment. For example, the multiple watermark candidate positions can be regarded as a set of scattered points, and then the concentration point of these scattered points can be determined in the manner of concentric circles, and the position of the concentration point can be determined as the target watermark pre-addition position of the target video segment.

[0056] In this embodiment of the application, in order to make the watermark addition position appear in the corresponding video in a specific way that may be discontinuous in time and randomly jump in position, so that video content thieves cannot accurately capture the watermark addition position, after determining the target watermark pre-addition position, the following operations can be performed:

[0057] Determine whether the target watermark pre-addition position is the same as the watermark pre-addition position in the first video segment, wherein the first video segment and the target video segment belong to the same video and include at least one video segment located before the target video segment;

[0058] When the target watermark pre-addition position is different from the watermark pre-addition position in each video segment included in the first video segment, the target watermark pre-addition position is determined to be valid; wherein, when the target watermark pre-addition position is the same as the watermark pre-addition position in any video segment included in the first video segment, the target watermark pre-addition position is determined to be invalid, and no watermark is added to the target video segment.

[0059] In other words, for a given video, a watermark can be added by comparing the pre-added watermark positions in multiple adjacent video segments. Specifically, if the pre-added watermark position of a video segment is different from the pre-added watermark positions in at least one of the preceding video segments, then the pre-added watermark position of that video segment is valid, and a watermark is added to that video segment. Conversely, if the pre-added watermark position of that video segment is the same as the pre-added watermark position in any of the preceding video segments, then the pre-added watermark position of that video segment is invalid, and no watermark is added to that video segment.

[0060] Optionally, the process of determining whether the target watermark pre-addition position is the same as the watermark pre-addition position in the first video segment may include:

[0061] Determine whether the target identifier in the area where the target watermark pre-addition position is located exists in a first list; wherein, the first list includes M identifiers, the M identifiers are the identifiers of the areas where the M watermark pre-addition positions are located, and the identifiers of different areas are different, the M watermark pre-addition positions are the watermark pre-addition positions in the M video segments included in the first video segment, and M is an integer greater than 0. When the target identifier does not exist in the first list, it is determined that the target watermark pre-addition position is not the same as the watermark pre-addition positions in the M video segments included in the first video segment, and the target watermark pre-addition position is determined to be valid. If the number of identifiers included in the first list has not reached the maximum value, the target identifier is added to the first list, or if the number of identifiers included in the first list has reached the maximum value, the target identifier is added to the first list, and the identifier added first in the first list is deleted, so as to dynamically update the identifiers in the first list; or, when the target identifier exists in the first list, it is determined that the target watermark pre-addition position is the same as the watermark pre-addition position in a certain video segment included in the first video segment, and the target watermark pre-addition position is determined to be invalid, and the target identifier is deleted from the first list.

[0062] In this way, by comparing with the identifiers in the first list, and by deleting and / or adding identifiers to the first list, the target watermark pre-addition position of the target video segment can be different from the watermark pre-addition position of at least one adjacent and previous video segment. As a result, the watermark addition position in the corresponding video may be discontinuous in time and randomly jump in position, making it impossible for video content thieves to accurately capture the watermark addition position.

[0063] In some embodiments, the first list may be a queue of storage elements created by the terminal program, such as storing up to 3 identifiers for dynamically determining the watermark pre-addition position of the video segment.

[0064] In some embodiments, if the width of the video image is W and the height is H, then the possible range of the watermark is [0.05*W, 0.95*W]*[0.05*H, 0.95*H], which is 90% of the width and height. If the watermark is rectangular with width and height w and h respectively, maintaining its horizontal and vertical orientation, and using the top-left corner LTP (LeftTopPoint) as the relative point, then the coordinate range of LTP is w∈[0.05*W, 0.95*Ww], h∈[0.05*H, 0.95*Hh]. Figure 3 As shown, the top-left corner of the video frame can be taken as the origin, and the dashed box inside represents the possible area of ​​the watermark. Taking the watermark located at the top-right corner of the area as an example, the coordinates of the LTP point at the top-left corner of the watermark are explained. Furthermore, as... Figure 3As shown, the video image can be divided into four regions by drawing vertical lines along both the horizontal and vertical directions. Then, the video frame size is scaled by a certain ratio (e.g., 30%) and placed in the center to form a fifth region. This fifth region is then numbered, for example, from one to five; these numbers serve as the identifiers for the corresponding regions. If the target watermark pre-addition location is in region one, the corresponding identifier is one; if it's in region two, the identifier is two; if it's in region three, the identifier is three; if it's in region four, the identifier is four; and if it's in region five, the identifier is five. It should be noted that this embodiment uses a five-region division of the video image as an example, but this application is not limited to this; it can also be divided into six, seven, etc., based on actual needs. The more regions divided, the more selectable watermark jump positions are available.

[0065] In this embodiment, the color feature information of the target watermark pre-addition area can be obtained by analyzing the color difference of the target watermark pre-addition area, such as by using deep learning image processing technology to decode the target watermark pre-addition area. The aforementioned determination of the color feature information of the target watermark pre-addition area may include:

[0066] For each frame of a multi-frame image in a target video segment, the target watermark pre-addition area in each frame is divided into S sub-regions, and the color value of each sub-region in the S sub-regions is calculated in turn to obtain S color values. That is, S color values ​​are obtained for each frame image, where S is an integer greater than 1.

[0067] Based on the S color values ​​in each frame of the image, the color feature information of the target watermark pre-addition area in the multi-frame image is determined, that is, the color feature information is related to the S color values ​​in each frame of the image.

[0068] In some embodiments, the color values ​​mentioned above can be selected as LAB color values. The LAB color space is based on human eye perception of color and can represent all colors that the human eye can perceive. L represents lightness, A represents red-green difference, and B represents blue-yellow difference. The brightness L* of a color indicates black when L* is 0 and white when L* is 100. The position a* between red and green indicates green when negative and red when positive. The position b* between yellow and blue indicates blue when negative and yellow when positive.

[0069] Optionally, when the determined color feature information is related to S color values ​​of each frame in the multi-frame images, and the S color values ​​are obtained by sequentially calculating the color values ​​of S sub-regions divided into the target watermark pre-addition area in each frame image, the process of generating the target watermark identifier based on the color feature information can include: first, dividing the initial watermark identifier into S sub-regions; then, using the S color values ​​in each frame image, setting the colors of the S sub-regions in the initial watermark identifier respectively to obtain the target watermark identifier. This ensures that the generated target watermark identifier is integrated into the video frame content as much as possible while maintaining visual distinguishability, thereby preventing the watermark identifier from being discriminated against and cracked by deep learning-based adversarial neural networks, and ensuring the security of the watermark identifier.

[0070] Furthermore, when using the S color values ​​in each frame of the image to obtain the target watermark identifier, the color values ​​of the corresponding sub-regions of the target watermark pre-addition area in the multi-frame image can be averaged according to the S color values ​​in each frame of the image to obtain S average color values; then, the colors of the S sub-regions in the initial watermark identifier are set using the S average color values ​​to obtain the target watermark identifier. For example, if the target video segment includes P frames (this is just an example), and the target watermark pre-applied area in each frame is divided into three sub-regions: sub-region 1, sub-region 2, and sub-region 3, then after obtaining the three color values ​​in each frame, the color values ​​of sub-region 1 in the P-frame image can be averaged to obtain average color value 1; the color values ​​of sub-region 2 in the P-frame image can be averaged to obtain average color value 2; and the color values ​​of sub-region 3 in the P-frame image can be averaged to obtain average color value 3. These average color values ​​1, 2, and 3 are then used as the color feature information for the target watermark pre-applied area of ​​the target video segment; that is, average color value 1 is used as the color feature information for sub-region 1, average color value 2 for sub-region 2, and average color value 3 for sub-region 3. Afterward, the average color values ​​1, 2, and 3 can be used to set the colors of the three sub-regions in the initial watermark identifier to obtain the target watermark identifier.

[0071] In other embodiments, a target frame image can be selected from the multi-frame images based on a preset algorithm, and then S color values ​​from the target frame image can be determined as the color feature information of the target watermark pre-addition area in the multi-frame images. It should be noted that this preset algorithm can be determined based on actual needs and is not limited thereto.

[0072] In other embodiments, after calculating the color value of each sub-region in the S sub-regions sequentially, when the difference between the color value of the first sub-region and the color value of the second sub-region is less than a preset threshold, that is, when the relative perceptual difference between the colors of the first and second sub-regions is small, the first and second sub-regions are considered adjacent sub-regions among the S sub-regions. The average of the color values ​​of the first and second sub-regions can be determined as the color value of the first and second sub-regions, simplifying subsequent watermark processing. The preset threshold can be determined based on actual needs.

[0073] Optionally, if the S sub-regions in the target watermark pre-addition area are obtained by dividing the target watermark pre-addition area into m*n units, and the S sub-regions in the initial watermark identifier are obtained by dividing the initial watermark identifier into n*m units; where n is an integer greater than 1, then when using the obtained S average color values ​​to set the colors of the S sub-regions in the initial watermark identifier, the first matrix can be first flipped horizontally, vertically, and diagonally to obtain the second matrix; the first matrix is ​​an m*n matrix, and the elements in the first matrix are the S average color values; then, using the S average color values ​​in the second matrix, the colors of the corresponding sub-regions in the initial watermark identifier are set to obtain the target watermark identifier. Taking a 2*4 matrix as an example, the process of flipping the first matrix horizontally, vertically, and diagonally can be as follows: Figure 4 As shown.

[0074] In this way, by performing horizontal, vertical and diagonal flips on the first matrix, the color feature information of S sub-regions in the target watermark pre-addition area can be transformed. Then, the transformed color feature information is used to generate the target watermark identifier. On the one hand, this prevents the target watermark identifier from being too similar in color to the video frame content, thus losing the watermark effect. On the other hand, it can maintain the similarity of the colors of the content around the video frame of the target watermark identifier, preventing it from being detected by the adversarial network.

[0075] For example, when decoding the color feature information of the target watermark pre-added area, a single frame image can be used as the research object to obtain the target watermark pre-added area RI of this frame image. For example, if the size of the watermark is magnified by 1.2 times, and the image size of the target watermark pre-added area RI is Rw*Rh, then from the magnification ratio, Rw = 1.2*w, where w is the width of the watermark. Similarly, the size of Rh can be determined. Then, the target watermark pre-added area RI can be vertically and horizontally divided into m*n sub-regions, where m and n can be equal or unequal, but all values ​​must be greater than 1. Then, the LAB color value is calculated for each of the m*n sub-regions. Finally, for the LAB color values ​​of the m*n sub-regions, the relative perceptual difference in color between two adjacent sub-regions in the LAB color space (horizontal and vertical directions, diagonal directions are not calculated) can be calculated. For example, Euclidean distance in three-dimensional space can be used, with the formula: When the relative perceptual difference in color between adjacent sub-regions is small, their average value can be taken to represent the LAB color value of that region, thus obtaining the LAB color value of each sub-region. Due to the continuous nature of video frames, the LAB color value of the target watermark pre-applied area can be calculated for each frame image within a video segment. Then, the average LAB color value within the video segment can be obtained, and the color feature information of the target watermark pre-applied area of ​​the video segment can be decoded.

[0076] For example, when generating the target watermark identifier, the color (RGB) of each area of ​​the target watermark identifier can be set according to the obtained LAB color value. The corresponding color conversion (i.e., from LAB to RGB) formula is as follows:

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] Wherein, Xn, Yn, and Zn are the reference white point CIEXYZ tri-color stimulus values. When calculating, the corresponding values ​​can be: Xn = 96.4221, Yn = 100.0000, Zn = 82.5221.

[0084] Since the universally accepted color gamut space is used, whether it is RGB or LAB, the transformation between the two color gamuts follows the universally accepted color gamut space definition. Therefore, this application directly provides the conversion formula and will not explain it again.

[0085] It should be noted that the watermarking method provided in this application embodiment can be executed by a watermarking device or a control module within that watermarking device for executing the watermarking method. This application embodiment uses a watermarking device executing the watermarking method as an example to illustrate the watermarking device provided in this application embodiment.

[0086] Please see Figure 5 , Figure 5 This is a schematic diagram of a watermark adding device provided in an embodiment of this application. The device is applied to an electronic device, which may be a video server or a client. Figure 5 As shown, the watermark adding device 50 includes:

[0087] Acquisition module 51 is used to acquire a target video segment, wherein the target video segment includes multiple frames of images;

[0088] The first determining module 52 is used to determine the target watermark pre-addition position in the multi-frame images;

[0089] The second determining module 53 is used to determine the color feature information of the target watermark pre-addition area, wherein the target watermark pre-addition area is the watermark pre-addition area in the multi-frame image corresponding to the target watermark pre-addition position;

[0090] The generation module 54 is used to generate a target watermark identifier based on the color feature information;

[0091] The adding module 55 is used to add the target watermark identifier to the target watermark pre-addition position in the multi-frame images respectively.

[0092] Optionally, the first determining module 52 includes:

[0093] The detection unit is used to detect the target object in each frame of the multi-frame image;

[0094] The first determining unit is used to determine the target watermark pre-addition position in the multi-frame image based on the target object detected in each frame image, wherein the target watermark pre-addition position does not overlap with the position of the detected target object.

[0095] Optionally, the first determining unit is specifically configured to: generate N target boxes in each frame image based on the target objects detected in each frame image; wherein, each target box corresponds one-to-one with the detected target object, the target box surrounds the corresponding target object, and N is an integer greater than or equal to 0; arrange the N target boxes in each frame image in descending order of area; for the arranged N target boxes, starting from the target box with the largest area, sequentially construct lines between adjacent target boxes until there is an intersection point between the newly constructed line and the previously constructed line located outside the N target boxes; determine the watermark candidate position in each frame image based on the position of the intersection point, the watermark candidate position being located outside the N target boxes; and determine the target watermark pre-addition position in the multi-frame image based on the watermark candidate position in each frame image.

[0096] Optionally, if the adjacent target boxes include a first target box and a second target box, the line connecting the adjacent target boxes includes at least one of the following:

[0097] The line connecting the top left point of the first target box and the bottom right point of the second target box;

[0098] The line connecting the upper right point of the first target box and the lower left point of the second target box.

[0099] Optionally, the first determining unit is further configured to perform any of the following:

[0100] When the intersection point contains one intersection point, the position of the one intersection point is determined as the watermark candidate position in each frame of the image;

[0101] When the intersection point contains multiple intersection points, the position of the first intersection point is determined as the watermark candidate position in each frame image, and the first intersection point is any one of the multiple intersection points.

[0102] Optionally, the second determining unit is specifically used to: determine multiple watermark candidate positions based on the watermark candidate positions in each frame of the image; and determine the center position corresponding to the multiple watermark candidate positions as the target watermark pre-addition position in the multiple frames of the image.

[0103] Optionally, the watermark adding device 50 also includes:

[0104] The judgment module is used to determine whether the target watermark pre-addition position is the same as the watermark pre-addition position in the first video segment, wherein the first video segment and the target video segment belong to the same video and include at least one video segment located before the target video segment.

[0105] The third determining module is used to determine that the target watermark pre-addition position is valid when the target watermark pre-addition position is different from the watermark pre-addition position in each video segment included in the first video segment; wherein, when the target watermark pre-addition position is the same as the watermark pre-addition position in any video segment included in the first video segment, the target watermark pre-addition position is determined to be invalid, and no watermark is added to the target video segment.

[0106] Optionally, the determination module is specifically used to: determine whether the target identifier in the area where the target watermark is pre-added is located exists in the first list;

[0107] The first list includes M identifiers, where each of the M identifiers represents an area containing a pre-added watermark, and each of the M pre-added watermark locations represents a pre-added watermark location in one of the M video segments contained in the first video segment. M is an integer greater than 0. When the target identifier does not exist in the first list, it is determined that the target watermark location is different from all the pre-added watermark locations in the M video segments. If the number of identifiers in the first list has not reached its maximum value, the target identifier is added to the first list; or if the number of identifiers in the first list reaches its maximum value, the target identifier is added to the first list, and the first identifier added to the first list is deleted. Alternatively, when the target identifier exists in the first list, it is determined that the target watermark location is the same as a pre-added watermark location in one of the M video segments, and the target identifier is deleted from the first list.

[0108] Optionally, the color feature information is related to S color values ​​of each frame in the multi-frame images, and the S color values ​​are obtained by sequentially calculating the color values ​​of S sub-regions divided into the target watermark pre-addition area in each frame image; where S is an integer greater than 1.

[0109] The generation module 54 is specifically used to: divide the initial watermark identifier into S sub-regions; and use the S color values ​​in each frame image to set the colors of the S sub-regions in the initial watermark identifier to obtain the target watermark identifier.

[0110] Optionally, the generation module 54 includes:

[0111] The calculation unit is used to average the color values ​​of the corresponding sub-regions of the target watermark pre-addition area in the multi-frame images according to the S color values ​​in each frame image, and obtain S average color values.

[0112] The generation unit is used to set the colors of the S sub-regions in the initial watermark identifier using the S average color values, and obtain the target watermark identifier.

[0113] Optionally, the S sub-regions in the target watermark pre-addition area are obtained by dividing the target watermark pre-addition area into m*n sub-regions, and the S sub-regions in the initial watermark identifier are obtained by dividing the initial watermark identifier into n*m sub-regions; where m is an integer greater than 1 and n is an integer greater than 1; the generation unit is specifically used for:

[0114] The first matrix is ​​flipped horizontally, vertically, and diagonally to obtain the second matrix; the first matrix is ​​an m*n matrix, and the elements in the first matrix are the S average color values;

[0115] Using the S average color values ​​in the second matrix, the colors of the corresponding sub-regions in the initial watermark identifier are set respectively to obtain the target watermark identifier.

[0116] The watermark adding device 50 of this application embodiment can achieve the above-mentioned... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0117] Optional, such as Figure 6 As shown, this application embodiment also provides an electronic device 60, including a processor 61, a memory 62, and a program or instructions stored in the memory 62 and executable on the processor 61. When the program or instructions are executed by the processor 61, they implement the various processes of the above-described watermarking method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0118] This application also provides a readable storage medium storing a program or instructions that, when executed by a processor, can achieve the above-described functions. Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0119] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0120] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0121] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a service classification device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0123] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for adding a watermark, characterized in that, include: Acquire a target video segment, which includes multiple frames of images; Determine the target watermark pre-addition position in the multi-frame images; The target watermark pre-addition position does not overlap with the position of the target object in each frame of the multi-frame image; Determine the color feature information of the target watermark pre-addition area, wherein the target watermark pre-addition area is the watermark pre-addition area in the multi-frame image corresponding to the target watermark pre-addition position; Based on the color feature information, a target watermark identifier is generated; The target watermark identifier is added to the target watermark pre-addition position in the multi-frame images respectively; The step of determining the target watermark pre-addition position in the multi-frame images includes: Detect the target object in each frame of the multi-frame image; Based on the target objects detected in each frame of the image, N target boxes are generated in each frame of the image; wherein, each target box corresponds one-to-one with the detected target object, the target box surrounds the corresponding target object, and N is an integer greater than or equal to 0; The N target boxes in each frame of the image are sorted in descending order of their area. For the N target boxes arranged, starting from the target box with the largest area, the lines between adjacent target boxes are constructed sequentially until there is an intersection point between the newly constructed line and the previously constructed line that is outside the N target boxes. Based on the location of the intersection point, the watermark candidate position in each frame image is determined, and the watermark candidate position is located outside the N target boxes; Based on the watermark candidate positions in each frame of the image, the target watermark pre-addition position in the multi-frame image is determined.

2. The method according to claim 1, characterized in that, If the adjacent target boxes include a first target box and a second target box, the line connecting the adjacent target boxes includes at least one of the following: The line connecting the top left point of the first target box and the bottom right point of the second target box; The line connecting the upper right point of the first target box and the lower left point of the second target box.

3. The method according to claim 1, characterized in that, The step of determining the watermark candidate position in each frame image based on the position of the intersection point includes any one of the following: When the intersection point contains one intersection point, the position of the one intersection point is determined as the watermark candidate position in each frame of the image; When the intersection point contains multiple intersection points, the position of the first intersection point is determined as the watermark candidate position in each frame image, and the first intersection point is any one of the multiple intersection points.

4. The method according to claim 1, characterized in that, The step of determining the target watermark pre-addition position in the multi-frame images based on the watermark candidate positions in each frame image includes: Based on the watermark candidate positions in each frame of the image, multiple watermark candidate positions are determined; The center position corresponding to the multiple watermark candidate positions is determined as the target watermark pre-addition position in the multiple frames of images.

5. The method according to claim 1, characterized in that, After determining the target watermark pre-addition position in the multi-frame images, the method further includes: Determine whether the target watermark pre-addition position is the same as the watermark pre-addition position in the first video segment, wherein the first video segment and the target video segment belong to the same video and include at least one video segment located before the target video segment; When the target watermark pre-addition position is different from the watermark pre-addition position in each video segment contained in the first video segment, the target watermark pre-addition position is determined to be valid. Specifically, if the target watermark pre-addition position is the same as the watermark pre-addition position in any video segment included in the first video segment, the target watermark pre-addition position is determined to be invalid, and no watermark identifier is added to the target video segment.

6. The method according to claim 1, characterized in that, The color feature information is related to S color values ​​of each frame in the multi-frame images. The S color values ​​are obtained by sequentially calculating the color values ​​of S sub-regions in each frame image to divide the target watermark pre-addition area. S is an integer greater than 1; The step of generating a target watermark identifier based on the color feature information includes: Divide the initial watermark into S sub-regions; Using the S color values ​​in each frame of the image, the colors of the S sub-regions in the initial watermark identifier are set respectively to obtain the target watermark identifier.

7. The method according to claim 6, characterized in that, The step of using S color values ​​from each frame of the image to set the colors of S sub-regions in the initial watermark identifier to obtain the target watermark identifier includes: Based on the S color values ​​in each frame of the image, the color values ​​of the corresponding sub-regions of the target watermark pre-addition area in the multi-frame images are averaged to obtain S average color values. Using the S average color values, the colors of the S sub-regions in the initial watermark are set respectively to obtain the target watermark.

8. The method according to claim 7, characterized in that, The S sub-regions in the target watermark pre-addition area are obtained by dividing the target watermark pre-addition area into m*n sub-regions, and the S sub-regions in the initial watermark identifier are obtained by dividing the initial watermark identifier into n*m sub-regions; where m is an integer greater than 1 and n is an integer greater than 1. The step of using the S average color values ​​to set the colors of the S sub-regions in the initial watermark identifier to obtain the target watermark identifier includes: The first matrix is ​​flipped horizontally, vertically, and diagonally to obtain the second matrix; the first matrix is ​​an m*n matrix, and the elements in the first matrix are the S average color values; Using the S average color values ​​in the second matrix, the colors of the corresponding sub-regions in the initial watermark identifier are set respectively to obtain the target watermark identifier.

9. A watermark adding device, characterized in that, include: The acquisition module is used to acquire a target video segment, which includes multiple frames of images; The first determining module is used to determine the target watermark pre-addition position in the multi-frame images; The target watermark pre-addition position does not overlap with the position of the target object in each frame of the multi-frame image; The second determining module is used to determine the color feature information of the target watermark pre-addition area, wherein the target watermark pre-addition area is the watermark pre-addition area in the multi-frame image corresponding to the target watermark pre-addition position; The generation module is used to generate a target watermark identifier based on the color feature information; An adding module is used to add the target watermark identifier to the target watermark pre-addition position in the multi-frame images respectively; The first determining module includes: The detection unit is used to detect the target object in each frame of the multi-frame image; The first determining unit is configured to generate N target boxes in each frame of the image based on the target objects detected in each frame; wherein each target box corresponds one-to-one with a detected target object, the target box surrounds the corresponding target object, and N is an integer greater than or equal to 0; the N target boxes in each frame of the image are arranged in descending order of area; for the arranged N target boxes, starting from the target box with the largest area, lines are sequentially constructed between adjacent target boxes until there is an intersection point between the newly constructed line and the previously constructed line located outside the N target boxes; based on the position of the intersection point, a watermark candidate position is determined in each frame of the image, the watermark candidate position being located outside the N target boxes; based on the watermark candidate position in each frame of the image, a target watermark pre-addition position is determined in the multi-frame image.

10. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the watermarking method as described in any one of claims 1 to 8.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the watermarking method as described in any one of claims 1 to 8.

Images