Watermark superposition method, device, equipment and medium

Abstract

The embodiment of the invention discloses a watermark superposition method and device, equipment and a medium. The method comprises the following steps: acquiring a visual field image acquired by an image acquisition device, and determining at least one candidate static region in the visual field image; determining at least one target static region from the candidate static regions according to watermark parameters of a watermark to be superposed, and taking the target static region as a watermark superposition region; determining a coding block division mode according to the position information of the superposed watermark region, so as to obtain a block video stream corresponding to the original video stream according to the coding block division mode; and obtaining a target part image from the block video stream based on the position information of the watermark overlapping area, and performing watermark overlapping on the target part image. According to the technical scheme, the watermark information can be superposed under the condition that the performance of the server is not greatly reduced, the resource consumption during watermark superposition is effectively reduced, the watermark superposition cost is reduced, and the watermark superposition efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a watermark superposition method, device, equipment and medium. Background Art

[0002] In security monitoring systems, watermark overlay can be used to identify the source and time information of images or videos, ensuring the reliability and traceability of images and videos for security monitoring, crime investigation and legal evidence. Watermark overlay is increasingly used in a wide range of scenarios. In some scenarios, watermarks need to be overlaid on video servers, and the videos with watermarks overlaid can be downloaded and watched by users.

[0003] In the related art, watermark information is superimposed after decoding all video frames (I frames and P frames), and then the video frames with superimposed watermarks are encoded. Among them, I frames are key frames and P frames are difference frames. However, this solution has high performance consumption, especially in the scene of high resolution and high frame rate, decoding all frames consumes more hardware resources, which seriously reduces the performance of the video server. Summary of the invention

[0004] The present invention provides a watermark superposition method, device, equipment and medium, which can superimpose watermark information without significantly reducing server performance, effectively reduce resource consumption of superimposed watermarks, reduce watermark superposition costs, and improve watermark superposition efficiency.

[0005] According to one aspect of the present invention, a watermark superposition method is provided, the method comprising:

[0006] Acquire a field of view image acquired by an image acquisition device, and determine at least one candidate static area in the field of view image;

[0007] Determine at least one target static area from the candidate static areas according to the watermark parameters of the watermark to be superimposed, and use the target static area as the superimposed watermark area;

[0008] Determine a coding block division method according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division method;

[0009] A target partial image is obtained from the block video stream based on the position information of the superimposed watermark area, and a watermark is superimposed on the target partial image.

[0010] According to another aspect of the present invention, there is provided a watermark superimposing device, comprising:

[0011] A candidate static region determination module, used to acquire a field of view image acquired by an image acquisition device, and determine at least one candidate static region in the field of view image;

[0012] A superimposed watermark region determining module, configured to determine at least one target static region from the candidate static regions according to the watermark parameters of the watermark to be superimposed, and use the target static region as the superimposed watermark region;

[0013] A coding block division mode determination module, used to determine the coding block division mode according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division mode;

[0014] The watermark superimposition module is used to obtain a target partial image from the block video stream based on the position information of the superimposed watermark area, and superimpose the watermark on the target partial image.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the watermark superimposition method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the watermark superimposition method described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention obtains the field of view image acquired by the image acquisition device, and determines at least one candidate static area in the field of view image; determines at least one target static area from the candidate static area according to the watermark parameters of the watermark to be superimposed, and uses the target static area as the superimposed watermark area; determines the coding block division method according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division method; obtains the target partial image from the block video stream based on the position information of the superimposed watermark area, and performs watermark superimposition on the target partial image. This technical solution can superimpose watermark information without significantly reducing the performance of the server, effectively reduces the resource consumption when superimposing watermarks, reduces the cost of watermark superimposition, and improves the efficiency of watermark superimposition.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a watermark superimposition method provided according to Embodiment 1 of the present invention;

[0024] Figure 2A This is a schematic diagram of Tile partitioning provided according to the first embodiment of the present invention;

[0025] Figure 2B This is a Slice block diagram provided according to the first embodiment of the present invention;

[0026] Figure 3 is a flow chart of a watermark superposition method provided according to Embodiment 2 of the present invention;

[0027] Figure 4 is a schematic diagram of a watermark superposition method provided according to Embodiment 2 of the present invention;

[0028] Figure 5 is a schematic diagram of watermark superimposition signaling control provided according to Embodiment 2 of the present invention;

[0029] Figure 6 is a schematic structural diagram of a watermark superimposition device provided according to Embodiment 3 of the present invention;

[0030] Figure 7 It is a structural schematic diagram of an electronic device for implementing a watermark superimposition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 This is a flow chart of a watermark superposition method provided in the first embodiment of the present invention. This embodiment is applicable to the case of low-cost watermark superposition on video streams. The method can be executed by a watermark superposition device. The watermark superposition device can be implemented in the form of hardware and / or software. The watermark superposition device can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes:

[0035] S110, acquiring a field of view image acquired by an image acquisition device, and determining at least one candidate static region in the field of view image.

[0036] This technical solution can be executed by a watermark overlay server and is suitable for video streams captured by an image acquisition device (such as a camera) under a fixed field of view. In this case, the detection area of ​​the image acquisition device is fixed and there are usually static objects in the detection area, such as buildings, vegetation, etc.

[0037] In this embodiment, a field of view image captured by an image acquisition device (such as a camera) is first obtained, and at least one candidate static area in the field of view image is determined. Among them, a static area may refer to an area where the image content is fixed, such as a building area, a vegetation area, etc. A candidate static area may refer to any static area in the field of view image. Exemplarily, an image capture request can be sent to the image acquisition device through a signaling control module in a watermark overlay server. After obtaining the captured field of view image, the area where static objects such as buildings and vegetation are located in the field of view image can be identified as a candidate static area through an AI recognition algorithm.

[0038] Exemplarily, the pixel coordinates of the upper left corner of the candidate static area are represented as (X, Y), and the width and height of the candidate static area are represented as (W, H). Assuming that the final encoding is performed according to N*N macroblocks or N*N CTUs, the identified candidate static areas need to be regularized to ensure that X and Y of the candidate static areas are both integer multiples of N, and that the coordinates of the lower right corner of the candidate static area (X+H) are also integer multiples of N. In addition, it is also necessary to ensure that there is no overlapping between the candidate static areas. Among them, CTU (coding tree units) is a tree-shaped coding unit, which is the basic coding unit in H.265.

[0039] S120, determining at least one target static region from the candidate static regions according to the watermark parameters of the watermark to be superimposed, and using the target static region as the superimposed watermark region.

[0040] In this embodiment, after determining at least one candidate static area in the field of view image, at least one target static area can be determined from the candidate static area according to the watermark parameters of the watermark to be superimposed, and the target static area can be used as the superimposed watermark area. The watermark parameters may include the watermark size and the number of watermarks. The watermark size can be described in terms of pixel size, which can be a value or a range. For example, the watermark size can be 20 pixels or 8-56 pixels. Optionally, determining at least one target static area from the candidate static area according to the watermark parameters of the watermark to be superimposed includes: determining the superimposed area condition according to the watermark size and the number of watermarks to be superimposed; determining the static area to be judged in turn according to the area priority of the candidate static area, and determining at least one target static area according to the judgment result of the static area to be judged and the superimposed area condition; wherein the static area to be judged includes at least one candidate static area.

[0041] Exemplarily, the number of watermarks and the watermark size are represented as MOUNT and FONTSIZE, respectively, and the width and height of the candidate static area are represented as (W, H). The superimposed area condition can be set as follows: 1. The area width is greater than the watermark size, which can be represented as: W>FONTSIZE; 2. The area height is greater than the watermark size, which can be represented as: H>FONTSIZE; 3. The sum of the total number of pixels in the area is greater than the sum of the pixels required by the font, which can be represented as: W*H>MOUNT*FONTSIZE*FONTSIZE, where W and H are both integer multiples of the watermark size. It should be noted that if the superimposed watermark area includes multiple static areas, W*H in the above condition 3 becomes (W1*H1+…+Wn*Hn), where W1…Wn represent the width of each static area, H1…Hn represent the height of each static area, and W1…Wn and H1…Hn are both integer multiples of the watermark size.

[0042] In this embodiment, the regional priority can be set in advance for each candidate static area according to the actual application requirements, and then the static area to be judged can be determined from the candidate static areas in order from high to low regional priorities, and whether the static area to be judged meets the superposition area condition can be judged in turn. Among them, the static area to be judged is at least one candidate static area. For example, the regional priority decreases in order from left to right and from top to bottom. If there is a static area to be judged that meets the superposition area condition, the first static area to be judged that meets the superposition area condition is used as the target static area.

[0043] Exemplarily, assuming that the watermark size is 8-56 pixels, the static area to be judged is a single candidate static area, and the order of area priority is: area 1> area 2> ...> area n. First, determine whether area 1 meets the superimposed area condition according to the watermark size of 56. If so, area 1 is used as the target static area. Otherwise, the font is reduced and judged again until the minimum font is 8. If area 1 does not meet the condition for all watermark sizes, continue to determine whether area 2 meets the superimposed area condition in the order of watermark sizes from 56 to 8. Repeat this process until each area is judged separately. If there is no single area that meets the superimposed area condition, any n (starting from 2 and increasing) candidate static areas are formed into an area group as the static area to be judged. According to the above single area judgment method, each area group is judged separately according to the area priority order. If there is an area group that meets the superimposed area condition, all candidate static areas in the first area group that meets the condition are used as the target static area. Through the above process, the target static area can be quickly determined, and then the target static area can be used as the superimposed watermark area. It should be noted that it is generally believed that the larger the watermark size, the better the display effect of the watermark (ie, the clearer the watermark), so when the watermark size is a range, the watermark size can be judged in order from large to small.

[0044] S130, determining a coding block division method according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division method.

[0045] The original video stream may refer to a video stream obtained by performing image acquisition by an image acquisition device. The block video stream may refer to a video stream obtained by encoding the original video stream. The position information of the superimposed watermark area may be represented by the pixel coordinates of the superimposed watermark area in the field of view image.

[0046] In this embodiment, after determining the superimposed watermark area, the coding block division method can be determined according to the position information of the superimposed watermark area. It should be noted that this embodiment does not limit the coding block division method, which can be set according to actual needs. Exemplarily, the field of view image can be divided into blocks in an equidistant manner (for example, the image frame is divided into multiple N*N blocks), or the field of view image can be divided into blocks based on the boundary of the superimposed watermark area in the field of view image.

[0047] In this embodiment, the field of view image can be divided into blocks according to the coding block division method, so as to obtain a block division result. Optionally, after determining at least one target static area from the candidate static area according to the watermark parameters to be superimposed watermark, and taking the target static area as the superimposed watermark area, it also includes: determining the coding block division method according to the boundary information of the superimposed watermark area; sending the coding block division method to the image acquisition device, so that the image acquisition device performs image encoding according to the coding block division method.

[0048] In this embodiment, optionally, the coding block division method is determined according to the position information of the superimposed watermark area, including: if the block is a Tile, the coding block division method includes a horizontal column division method and a vertical row division method; if the block is a Slice, the coding block division method includes a vertical row division method; wherein, the horizontal column division method includes: taking the right boundary of each superimposed watermark area located in the first row as the right boundary of the Tile; the vertical row division method includes: taking the lower boundary of each superimposed watermark area as the lower boundary of the Tile or Slice, and taking the upper boundary of each superimposed watermark area as the upper boundary of the Tile or Slice.

[0049] Among them, Slice is a slice and Tile is a collage. Specifically, an image defined based on the H.264 / H.265 standard can be divided into one or more slices, each of which can be decoded independently and does not depend on other slices of the same image. Slice is a horizontally continuous area in an image in the shape of a strip. In H.265, an image can be divided not only into several Slices, but also into several Tiles. The image is divided into several rectangular areas horizontally and vertically. These areas are Tiles, and Tiles can also be encoded and decoded independently. A Slice can contain multiple Tiles, and a Tile can also contain multiple Slices. Among them, H.264 / H.265 is a digital video compression format, and H.265 has a higher compression rate than H.264.

[0050] Since the position information of each superimposed watermark area has been regularized in advance, it is safe to divide the blocks according to the boundary position of the superimposed watermark area. Specifically, if the blocks are tiles, in the horizontal column division mode, the right boundary of each superimposed watermark area is used as the right boundary of the tile, the left boundary of the tile where the leftmost superimposed watermark area is located is the left boundary of the entire visual image, and the rightmost superimposed watermark area uses the right boundary of the entire visual image as the right boundary of the tile. If the area where the area is located intersects with the left boundary or right boundary of a tile in the above step, this area is no longer divided into a new tile in the horizontal direction. In the vertical row division mode, the top boundary of the visual image is used as the upper boundary of the tile or the upper boundary of the superimposed watermark area is used as the upper boundary of the tile. The lower boundary part of the superimposed watermark area or the visual image is used as the lower boundary of the tile. If the lower boundary of the tile intersects with other areas, the lower boundary of the intersection area is used as the lower boundary of the tile, and is processed in sequence.

[0051] Figure 2A A schematic diagram of Tile segmentation provided in the first embodiment of the present invention. The superimposed area represents the superimposed watermark area. Figure 2A As shown in the figure, there are 4 superimposed watermark areas in total. After block division, 9 Tile blocks can be obtained. If the block is Slice, refer to the above vertical division method of Tile and change Tile to Slice.

[0052] Figure 2B A schematic diagram of Slice partitioning provided in the first embodiment of the present invention. The superimposed area represents the superimposed watermark area. Figure 2B As shown, there are 4 superimposed watermark areas in total, and after block division, 3 Slice blocks can be obtained.

[0053] It should be noted that when the field of view image is divided into blocks by equal spacing, there may be a situation where a superimposed watermark area is located in multiple blocks at the same time, and the number of target blocks determined at this time is large. However, determining the coding block division method based on the boundary information of the superimposed watermark area can reduce the number of target blocks, thereby improving the image coding efficiency.

[0054] After determining the coding block division method, it is necessary to send the coding block division method to the image acquisition device so that the image acquisition device performs image encoding according to the coding block division method, thereby obtaining a block video stream corresponding to the original video stream. Optionally, obtaining a block video stream corresponding to the original video stream according to the coding block division method includes: determining parameter information of a target block corresponding to the superimposed watermark area according to the coding block division method; and adding edge pixel filtering parameters of the target block to the original video stream according to the parameter information of the target block.

[0055] The parameter information may be identification information (such as ID information) for uniquely characterizing the target block. The edge pixel filtering parameter may include filtering strength. Specifically, different filtering strengths affect different pixels. Considering the amount of data, only the correction values ​​of the affected pixels under the filtering strength on the chrominance component and the luminance component may be saved.

[0056] S140, obtaining a target partial image from the segmented video stream based on the position information of the superimposed watermark area, and superimposing a watermark on the target partial image.

[0057] In this embodiment, after obtaining the segmented video stream corresponding to the original video stream, the target partial image can be further obtained from the segmented video stream based on the position information of the superimposed watermark area, and the watermark is superimposed on the target partial image. Optionally, obtaining the target partial image from the segmented video stream based on the position information of the superimposed watermark area includes: obtaining the image frame to be superimposed from the segmented video stream; determining the target segmented block in the image frame to be superimposed based on the position information of the superimposed watermark area; and obtaining the target partial image according to the target segmented block.

[0058] Among them, the original video stream usually includes I frames and P frames, and some also include B frames, among which I frames are key frames, P frames are difference frames, and B frames are bidirectional difference frames. If all video frames in the original video stream are decoded and then watermark information is superimposed, it will undoubtedly consume more hardware resources and seriously reduce the performance of the server. Therefore, representative image frames in the video stream can be selected as image frames to be superimposed. Optionally, the image frame to be superimposed is an I frame, and at this time, only the I frame needs to be superimposed with watermark information. In order to ensure that the watermark information of the I frame will not be lost after encoding and decoding of the P frame and the B frame, a fixed static image area in the I frame can be selected for watermark superposition. Therefore, the watermark superposition of the entire original video stream can be achieved by superimposing the watermark only in the static image area in the I frame.

[0059] In this embodiment, optionally, obtaining a target partial image according to the target block includes: decoding the target block to obtain a target block image corresponding to the target block; obtaining a target partial image from the target block image based on the position information of the superimposed watermark area. It can be understood that since the field of view (i.e., the detection area) of the image acquisition device is fixed and the superimposed watermark area is a static area, the position of the superimposed watermark area in different image frames remains unchanged, and the position of the target block corresponding to the superimposed watermark area in different image frames remains unchanged. Therefore, after obtaining the image frame to be superimposed, the target block in the image frame to be superimposed can be determined based on the position information of the superimposed watermark area, and then the target block is decoded to obtain the target block image corresponding to the target block. It can be understood that since the target block is a partial area in the image frame, the target block image corresponding to the target block is a partial image in the image frame. After obtaining the target block image, the target partial image can be determined from the target block image according to the position information of the superimposed watermark area. Among them, the target partial image is a part of the target block image.

[0060] After obtaining the target partial image, the target partial image can be watermarked. Specifically, the pre-set watermark information can be superimposed on the target superimposed watermark area corresponding to the target partial image to achieve watermark superimposition of the target partial image. Among them, the watermark information can be represented by a watermark image with watermark information. Specifically, an RGB picture with a pure white background or a pure black background (specifically determined according to the watermark font color) can be used as a background plate for watermark production, and the required watermark data is written into the prepared RGB picture to obtain a watermark image. When the number of watermark characters is large or the watermark information has a rotation angle, it is necessary to ensure that the watermark information does not exceed the calculated superimposed watermark area range or the watermark is cropped. Exemplarily, the pixel value of the watermark image can be mixed with the corresponding pixel of the decoded target partial image through pixel-level operation, and the transparency of the mixed text can be set according to specific requirements, etc., to superimpose the watermark on the target partial image.

[0061] The technical solution of the embodiment of the present invention obtains a field of view image acquired by an image acquisition device, and determines at least one candidate static area in the field of view image; determines at least one target static area from the candidate static area according to the watermark parameters of the watermark to be superimposed, and uses the target static area as the superimposed watermark area; determines the coding block division method according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division method; obtains the target partial image from the block video stream based on the position information of the superimposed watermark area, and performs watermark superimposition on the target partial image. This technical solution can superimpose watermark information without significantly reducing the performance of the server, effectively reduces the resource consumption when superimposing watermarks, reduces the cost of watermark superimposition, and improves the efficiency of watermark superimposition. In addition, at least one candidate static area is determined from the field of view image, and at least one target static area is determined from the candidate static area as the superimposed watermark area, which improves the accuracy and flexibility of the superimposed watermark area.

[0062] In this embodiment, optionally, before the watermark is superimposed on the target partial image, the method further includes: performing data compensation processing on the target partial image according to the edge pixel filtering parameters of the target block. The accuracy of the target partial image can be improved through data compensation processing.

[0063] Embodiment 2

[0064] Figure 3 This is a flow chart of a watermark superimposition method provided in Embodiment 2 of the present invention. This embodiment is optimized based on the above embodiment.

[0065] like Figure 3 As shown, the method of this embodiment specifically includes the following steps:

[0066] S210, acquiring a field of view image acquired by an image acquisition device, and determining at least one candidate static region in the field of view image.

[0067] S220, determining at least one target static region from the candidate static regions according to the watermark parameters of the watermark to be superimposed, and using the target static region as the superimposed watermark region.

[0068] S230, determining a coding block division method according to the position information of the superimposed watermark area, and determining parameter information of a target block corresponding to the superimposed watermark area according to the coding block division method.

[0069] S240, adding edge pixel filtering parameters of the target block to the original video stream according to the parameter information of the target block.

[0070] S250, obtaining an image frame to be superimposed from the block video stream corresponding to the original video stream.

[0071] The image frame to be superimposed may be an I frame.

[0072] S260, determining a target block in the image frame to be superimposed based on the position information of the superimposed watermark area, and decoding the target block to obtain a target block image corresponding to the target block.

[0073] S270, obtaining a target partial image from the target block image based on the position information of the superimposed watermark area, and superimposing the watermark on the target partial image.

[0074] S280, encoding the target block image after the watermark is superimposed to obtain an updated target block after the watermark is superimposed.

[0075] After the target partial image is watermarked, the target sub-block image after the watermark is superimposed can be encoded to obtain the updated target sub-block after the watermark is superimposed. The encoding format can be H.264 / H.265. It can be understood that the watermark information is superimposed in the updated target sub-block.

[0076] S290, merging the updated target block with other blocks in the image frame to be superimposed to obtain a target video stream after superimposing the watermark.

[0077] In this embodiment, after obtaining the updated target block after superimposing the watermark, the updated target block is merged with other blocks in the image frame to be superimposed, so as to obtain the target video stream after superimposing the watermark. Among them, other blocks refer to all coded blocks except the target block in the image frame to be superimposed. Therefore, on the basis of not destroying the original video stream, the watermark superposition of the original video stream can be achieved only by superimposing the watermark on the target block, thereby reducing the resource consumption when superimposing the watermark, reducing the watermark superposition cost, and improving the watermark superposition efficiency.

[0078] The technical solution of the embodiment of the present invention can achieve watermark superposition on the original video stream by simply performing watermark superposition processing on the target blocks without destroying the original video stream. Therefore, without significantly reducing the server performance, the resource consumption when superimposing watermarks is further reduced, the watermark superposition cost is reduced, and the watermark superposition efficiency is improved.

[0079] Figure 4A schematic diagram of a watermark superposition method provided in Embodiment 2 of the present invention. Among them, the OSD superposition server is a watermark superposition server, and the camera is an image acquisition device. The signaling control module can confirm the optimal Slice or Tile in the video screen according to the watermark parameters to be superimposed, and then notify the camera to encode the video image according to the Slice or Tile division method calculated by the signaling control module. At the same time, the camera should save the edge pixel filtering parameters of the Tile or Silce to SEI (Supplemental Enhancement Information).

[0080] like Figure 4 As shown, the camera transmits the network video stream (i.e., the original video stream) to the OSD overlay server through the network transmission mode (the encapsulation format during transmission is generally UDP / TCP+RTP+PS+H265), and the encapsulation parsing module parses the network video stream into H.264 / H.265 data. Then, the H.264 / H.265 data is parsed according to the standard protocol by the image pre-decomposition and separation module to obtain block 1 (i.e., the target block) data, which is transmitted to the image pre-editing module, and the block 1 data is re-supplemented with the video parameter set, sequence parameter set, and image parameter set by the image pre-editing module to be packaged into standard H.265 data, which is sent to the decoding accelerator for decoding, and the edge pixel filtering parameters in the SEI are used to perform data compensation processing on the decoded data. The OSD overlay module then performs watermark overlay processing on the decoded data, and the overlayed data can be encoded into H.265 data through the encoding accelerator. The image pre-coding module obtains data from the H.264 / H.265 data as a substitute for the original data, and the image merging module merges the re-encoded data with other data to obtain the complete H.265 data with watermark superimposed. Finally, the image encapsulation module encapsulates the H.264 / H.265 data into a network video stream and transmits it to the client for decoding and display.

[0081] Figure 5 A schematic diagram of watermark superimposition signaling control provided by Embodiment 2 of the present invention. Figure 5 As shown, the OSD overlay content is first sent to the signaling control module through the client, and then the OSD overlay content is sent to the OSD overlay module through the signaling control module, and the Tile and Slice division parameters (i.e., the encoding block division method corresponding to Slice and Tile) are calculated. The encoder obtains the Tile and Slice division parameters from the signaling control module, starts encoding after setting the encoding parameters, and sends the encoded video stream to the OSD overlay module.

[0082] Embodiment 3

[0083] Figure 6This is a schematic diagram of the structure of a watermark superposition device provided in the third embodiment of the present invention. The device can execute the watermark superposition method provided in any embodiment of the present invention and has the corresponding functional modules and beneficial effects of the execution method. Figure 6 As shown, the device comprises:

[0084] The candidate static region determination module 310 is used to obtain a field of view image acquired by an image acquisition device, and determine at least one candidate static region in the field of view image;

[0085] A superimposed watermark region determining module 320, configured to determine at least one target static region from the candidate static regions according to the watermark parameters of the watermark to be superimposed, and use the target static region as the superimposed watermark region;

[0086] The coding block division mode determination module 330 is used to determine the coding block division mode according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division mode;

[0087] The watermark superimposing module 340 is used to obtain a target partial image from the block video stream based on the position information of the superimposed watermark area, and superimpose the watermark on the target partial image.

[0088] Optionally, the watermark parameters include watermark size and watermark quantity;

[0089] Accordingly, the superimposed watermark region determining module 320 is used to:

[0090] Determine the superimposed area condition according to the watermark size and the number of watermarks to be superimposed;

[0091] The static areas to be determined are determined in sequence according to the area priorities of the candidate static areas, and at least one target static area is determined according to the judgment result of the static areas to be determined and the superimposed area conditions; wherein the static areas to be determined include at least one candidate static area.

[0092] Optionally, the watermark superimposing module 340 includes:

[0093] A unit for determining an image frame to be superimposed, used for obtaining an image frame to be superimposed from a block video stream;

[0094] A target block determining unit, configured to determine a target block in the to-be-superimposed image frame based on the position information of the superimposed watermark region;

[0095] The target partial image determining unit is used to obtain the target partial image according to the target blocks.

[0096] Optionally, the target partial image determination unit is specifically configured to:

[0097] Decoding the target block to obtain a target block image corresponding to the target block;

[0098] A target partial image is obtained from the target block image based on the position information of the superimposed watermark area.

[0099] Optionally, the image frame to be superimposed is an I frame.

[0100] Optionally, the device further comprises:

[0101] An updated target block determination module is used to encode the target block image after superimposing the watermark on the target partial image, so as to obtain an updated target block after superimposing the watermark;

[0102] The target video stream determination module is used to merge the updated target block with other blocks in the image frame to be superimposed to obtain a target video stream after superimposing the watermark; wherein the other blocks refer to all coded blocks in the image frame to be superimposed except the target block.

[0103] Optionally, the coding block division mode determination module 330 is used to:

[0104] Determining parameter information of a target block corresponding to the superimposed watermark area according to the coding block division method;

[0105] The edge pixel filtering parameters of the target block are added to the original video stream according to the parameter information of the target block.

[0106] Optionally, the device further comprises:

[0107] The data compensation processing module is used to perform data compensation processing on the target partial image according to the edge pixel filtering parameters of the target block before superimposing the watermark on the target partial image.

[0108] Optionally, the coding block division mode determination module 330 is further used to:

[0109] If the block is Tile, the encoding block division method includes horizontal column division method and vertical row division method;

[0110] If the block is Slice, the encoding block division method includes a vertical division row method;

[0111] Among them, the horizontal column division method includes: taking the right edge of each superimposed watermark area located in the first row as the right edge of the Tile; the vertical row division method includes: taking the lower edge of each superimposed watermark area as the lower edge of the Tile or Slice, and taking the upper edge of each superimposed watermark area as the upper edge of the Tile or Slice.

[0112] A watermark superimposition device provided in an embodiment of the present invention can execute a watermark superimposition method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0113] Embodiment 4

[0114] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0115] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a watermark superposition method.

[0118] In some embodiments, the watermark superposition method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the watermark superposition method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the watermark superposition method in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0124] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A watermark superposition method, characterized in that: The method comprises: Acquire a field of view image acquired by an image acquisition device, and determine at least one candidate static area in the field of view image; Determine at least one target static area from the candidate static areas according to the watermark parameters of the watermark to be superimposed, and use the target static area as the superimposed watermark area; Determine a coding block division method according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division method; A target partial image is obtained from the block video stream based on the position information of the superimposed watermark area, and a watermark is superimposed on the target partial image.

2. The method according to claim 1, characterized in that in, The watermark parameters include watermark size and watermark quantity; Accordingly, determining at least one target static area from the candidate static areas according to the watermark parameters of the watermark to be superimposed includes: Determine the superimposed area condition according to the watermark size and the number of watermarks to be superimposed; The static areas to be determined are determined in sequence according to the area priorities of the candidate static areas, and at least one target static area is determined according to the judgment result of the static areas to be determined and the superimposed area conditions; wherein the static areas to be determined include at least one candidate static area.

3. The method according to claim 1, characterized in that Obtaining a target partial image from the segmented video stream based on the position information of the superimposed watermark area includes: Obtaining the image frame to be superimposed from the block video stream; Determine the target block in the image frame to be superimposed based on the position information of the superimposed watermark area; A target partial image is obtained according to the target blocks.

4. The method according to claim 3, characterized in that Obtaining a target partial image according to the target blocks includes: Decoding the target block to obtain a target block image corresponding to the target block; A target partial image is obtained from the target block image based on the position information of the superimposed watermark area.

5. The method according to claim 3, characterized in that: The image frame to be superimposed is an I frame.

6. The method according to claim 4, characterized in that After superimposing the watermark on the target partial image, the method further includes: Encoding the target block image after the watermark is superimposed to obtain an updated target block after the watermark is superimposed; The updated target block is merged with other blocks in the image frame to be superimposed to obtain a target video stream after superimposing the watermark; wherein the other blocks refer to all coded blocks in the image frame to be superimposed except the target block.

7. The method according to claim 1, characterized in that Determining the coding block division method according to the position information of the superimposed watermark area includes: If the block is Tile, the encoding block division method includes horizontal column division method and vertical row division method; If the block is Slice, the encoding block division method includes a vertical division row method; Among them, the horizontal column division method includes: taking the right edge of each superimposed watermark area located in the first row as the right edge of the Tile; the vertical row division method includes: taking the lower edge of each superimposed watermark area as the lower edge of the Tile or Slice, and taking the upper edge of each superimposed watermark area as the upper edge of the Tile or Slice.

8. A watermark superimposition device, characterized in that: The device comprises: A candidate static region determination module, used to acquire a field of view image acquired by an image acquisition device, and determine at least one candidate static region in the field of view image; A superimposed watermark region determining module, configured to determine at least one target static region from the candidate static regions according to the watermark parameters of the watermark to be superimposed, and use the target static region as the superimposed watermark region; A coding block division mode determination module, used to determine the coding block division mode according to the position information of the superimposed watermark area, so as to obtain a block video stream corresponding to the original video stream according to the coding block division mode; The watermark superimposition module is used to obtain a target partial image from the block video stream based on the position information of the superimposed watermark area, and superimpose the watermark on the target partial image.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the watermark superimposition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the watermark superimposition method according to any one of claims 1 to 7 when executed.

Images