Computer implemented method for embedding marker in image or video content and corresponding marker detection method

Abstract

To provide a computer-implemented method for embedding a marker in an image or video content.SOLUTION: A method comprises: receiving an input image or a frame for embedding; determining a binary message to be encoded within the input image or frame comprising bits sequences; detecting a region within the input image or frame, the region presenting a chosen length and height; associating each bit sequence to a corresponding encoding color determined from the color within a uniform region and an encoding rule; generating a marker color table in which each element stores an encoding color associated to a bit sequence such that the color table constitutes a color encoding of the binary message; and embedding the marker in the region. Each element of the marker color table includes an action associated with a pixel block.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to the field of computer vision. [Background technology]

[0002] There are contexts in which a given piece of content has a particular structure that is not always accessible, or in which it may not always be desirable to use available tools to access this particular structure.

[0003] This is the case, for example, with certain web applications involving screen sharing via images or streaming, which require the use of specific modules that raise issues associated with security requirements and / or user preferences that do not rely on third-party components.

[0004] Therefore, it needs to be possible to detect specific portions of image or video content without using existing methods or APIs ("Application Programming Interfaces"). An API is a type of software interface that provides services to other pieces of software. Marking is a well-known method to solve this problem. Marking may be defined as using markers that are covertly embedded in noise-resistant signals such as audio, video, or image data. This is more complex as the marking method needs to be robust against high lossy compression of the medium being marked, fast to detect, and easy to use.

[0005] Well-known trivial techniques such as encoding the least significant bits of color information per pixel cannot be used because they are lost in the compression. There are two types of approaches that allow for lossy compression: Compression-aware solutions attempt to find safe spots in well-known compression algorithms and use them to hide information in compressed media in a retrievable way. Feature-aware solutions encode messages via common error-correcting codes by modifying features of the source image.

[0006] Obviously, the former solution cannot be used for the purposes of the present invention, although there are many compression algorithms and variants available for implementation.

[0007] Sun, Shuliang, International Arab Journal of Information Technology, 15, 2018, "Image Steganography Based on Hamming Code and Edge Detection," (2018) describes a method for performing steganography by modifying detected edge pixels in an image. This requires having control of these edges in the document, which is a very strict requirement, especially when used to embed third-party content. More generally, feature-aware steganography methods always assume full control over the visual appearance of the medium receiving the hidden information, which is also a very strict requirement.

[0008] In view of the above, there are currently no existing satisfactory solutions that provide compression-resistant steganography techniques that offer tractable computation-time detection methods. Summary of the Invention

[0009] The present invention aims to improve this situation. Towards this goal, the applicant proposes a computer-implemented method for embedding markers in image or video content, comprising the following operations: a) receiving an input image or frame for embedding; - b) determining a binary message encoded within said input image or frame that includes two or more bit sequences having the same number of bits, said binary message including at least a header portion that includes at least two consecutive bit sequences that are not identical; - c) detecting a region in said input image or frame such that the color within said region is uniform and said region exhibits a selected length and height; - d) associating each possible bit sequence with a corresponding coded color determined from the colors within the uniform region and the coding rule, such that each of said colors is different from the others; - e) generating a marker color table, each element of which stores an encoding color associated with a bit sequence of the binary message, such that the color table constitutes a color encoding of the binary message; and -f) embedding the marker into the region by directionally adding pixel blocks containing at least a selected number of pixels in the direction of addition, wherein the pixels in a given pixel block are each colored with the encoding color of a respective element of a marker color table, and each element of the marker color table is associated with at least one pixel block.

[0010] This method is advantageous because it allows embedding markers that can be detected with detection methods that have linear complexity and are resistant to lossy compression.

[0011] In various embodiments, the method may exhibit one or more of the following features: - operation e) using a horizontal addition direction to create a pixel block including four pixels in the addition direction and having a square shape; - operation e) further comprising repeating each pixel block of the header portion a selected number of times before adding the next pixel; the header portion includes bit sequences arranged such that each bit sequence is different from its immediately adjacent bit sequences, and each combination of two adjacent bit sequences is unique in the header portion; The binary message includes a header portion and a payload portion including at least two bit sequences, and operation b) includes applying an error correction code to the payload portion to obtain a robust payload portion, and interlacing the robust payload portion.

[0012] The invention further relates to a computer-implemented method for detecting markers embedded in image or video content according to the method of any one of the preceding claims, comprising the following operations: a) receiving an input image or frame for detection; - b) analyzing the input image or frame by pixel groups aligned along an image analysis direction, the pixels of each pixel group being separated in the image in the image analysis direction by a number of pixels equal to the size of a pixel block, the pixel groups having a size of the number of bit sequences in the header; - c) for each group of pixels, finding the color of each pixel and matching that color with one of the coded colors of the bit sequence; -d) comparing the resulting set of bit sequences with the set of bit sequences of the header part taking into account their positions, and upon detecting a selected number of matching sets of bit sequences, storing the position of the first pixel in the corresponding group of pixels as a potential marker start position.

[0013] In various embodiments, the method may exhibit one or more of the following features: - operations b), c), and d) are performed sequentially by analyzing a number of pixel groups equal to the number of pixels selected in the additional direction in parallel, -The image analysis direction is horizontal, the binary message encoded in the embedded marker further comprises a payload portion, the method further comprising an operation e) of analyzing the image according to the image analysis direction to search for all pixels corresponding to pixel blocks of the payload portion, detecting the color of each pixel and matching said color with one of the coded colors of the bit sequence; If operation e) fails to match the pixel to the coded color, operations b) to d) are repeated. the payload portion has been coded according to the coding method described above, the method further comprising an operation f) of decoding the bit sequence accordingly, and if operation f) is successful, searching an area around the start of a potential marker in an additional direction with a square window having a size equal to the selected number of pixels, the window being centred on the start of the potential marker and identifying the pixel that implicitly defines the image area that best corresponds in colour to the originally embedded marker; and If operation f) fails to decrypt the payload portion, operations b) to d) are resumed.

[0014] The invention also relates to a computer program comprising instructions for carrying out the method according to the invention, a data storage medium having recorded thereon such a computer program, and a computer system comprising a processor coupled to a memory having recorded thereon such a computer program. [Brief explanation of the drawings]

[0015] Other features and advantages of the present invention will become readily apparent from the following description of the drawings, which show exemplary embodiments of the invention.

[0016] [Figure 1] 1 shows a general schematic diagram of a system according to the present invention; [Figure 2] FIG. 2 illustrates an exemplary embodiment of a marker embedding function performed by the embedder of FIG. 1. [Figure 3] FIG. 3 shows a schematic illustration of a marker generated using the function of FIG. 2. [Figure 4] 2A and 2B illustrate exemplary embodiments of marker detection functions performed by the detector of FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0017] The drawings and the following description constitute a large proportion of positive and clearly defined features, and as a result, they are not only useful in understanding the invention, but can also be used to contribute to its definition if the need arises.

[0018] 1 illustrates a general schematic diagram of a system according to the present invention. The system 2 includes a memory 4, an embedder 6, and a detector 8.

[0019] The memory 4 stores all data used by the system 2, whether long-term or temporary. It receives all inputs and outputs. The main types of data stored in the memory 4 are images or video / streaming frames 10, embedding and encoding parameters 12, binary messages 14, and marked images 16.

[0020] As will readily emerge from the embodiments described herein, the system may be any and all types of computers, such as personal computers, laptops, tablets, mobile phones, etc. Images processed according to the present invention may be displayed by a display connected to the system 2.

[0021] In what follows, the expression "image" will be used to refer to any or a selection of media that is the subject of System 2. More precisely, System 2 deals with images, but these images may come from files that are themselves images, in part or in whole, but they may also be frames of a video stream, whether recorded or live, or subparts of another file, such as DOM elements within a frame.

[0022] In the examples described herein, memory 4 may be implemented in any suitable manner, i.e., a hard disk drive, a solid state drive, flash memory, memory embedded in the processor, remote storage accessible in the cloud, or any other suitable means.

[0023] Some of the data may be stored in a memory separate from memory 4, which may itself consist of a separate unit. Some of the data may be dropped if desired, and there is no requirement that all the data be stored in one memory.

[0024] In the examples described herein, the embedder 6 and detector 8 are computer programs running on one or more processors, including any known means for performing automated computations, such as a CPU, a GPU, a CPU and / or GPU grid, a remote computation grid, a specially configured FPGA, a specially configured ASIC, a specially configured chip such as a SOC or NOC, an AI specially designed chip, etc.

[0025] The embedder 6 is arranged to receive an input image 10, embedding and encoding parameters 12, and a binary message to be encoded and embedded 14. The embedder 6 processes this data and returns a marked image 16 which can be stored in memory 4 or transmitted to another party.

[0026] Conversely, the detector 8 receives the putative marked image 16, as well as the embedding and encoding parameters 12. The detector 8 processes this data and returns a binary message 14 or another type of relevant information.

[0027] FIG. 2 illustrates an exemplary embodiment of the functions performed by embedder 6.

[0028] The functionality begins with a variable input operation 200 where the inputs of the function of Figure 2 are received. This may be performed by a function Inp() that is provided with the input image Img, the marker generation parameters Par, and the message BinMsg as arguments to be used as global variables.

[0029] This can be done by a human machine interface (HMI). Any type of HMI can be used as long as the user provides an interface through specifying a file containing these global variables or otherwise specifying entries in memory 4. For example, this can be done by pointing to a specified area on the videoconferencing application's display, with or without a mouse or keyboard. This can also be done automatically.

[0030] Operation 200 is followed by a determine message components operation 210 in which the header and payload of the marker to be embedded in the image are determined. This may be done by a function Prepare() that receives as arguments the message BinMsg and the parameter Par and returns a pair of vectors Header[] and Payload[].

[0031] In the example described here, the vector Header[] contains the binary header of the marker to be embedded and is always the same. It may be included in the parameter Par, which is hard-coded in the function Prepare() or determined otherwise. Alternatively, the vector Header[] may be constructed according to the parameter Par or extracted from the message BinMsg.

[0032] The vector Payload[] is the payload of the message BinMsg and contains information intended to be transmitted in the marker. In one embodiment, the payload may contain information about the DOM object intended to be shared. In another embodiment, the payload may contain an identifier for the image being marked. In yet another embodiment, the payload may be used for tracking purposes. In some embodiments, the vector Payload[] may be omitted.

[0033] The Header[]Payload[] vector Payload[] has a size that is fixed or implicitly fixed according to the parameter Par. If the message BinMsg has a size that is smaller than the vector Payload[], the vector Payload[] may be zero-padded. In another alternative embodiment, the function Prepare() also performs the operations of operation 250, described further below, and the vector Payload[] contains an encoded version of the payload.

[0034] Operation 210 is followed by a marker start position determination operation 220. This may be performed by a function Locate() that receives an image Img and a parameter Par as arguments and may return a marker start point StartPoint and a base color RefColor. In alternative embodiments, operation 220 may be performed before operation 210, or they may be performed in parallel.

[0035] The function Locate() in operation 220 is used to detect in the image Img, an area where a marker can be stored. As will appear further below, the present invention makes it possible to store markers that can be inconspicuous in a manner related to steganography. The size of the marker itself is indicated by the parameter Par and takes the general form of a line of pixels with varying colors that encode information where the original image had a uniform or very homogeneous color. In the case of an inconspicuous marker, the color variation is chosen so that it is not easily distinguished by a viewer of the content. Therefore, to embed such a marker, it is necessary to find an area of ​​the image Img that is sufficiently homogeneous or of uniform color to encompass the size of the marker.

[0036] The result of the function Locate() of operation 220 contains the coordinates of the leftmost point of a first region in the image Img that has a uniform color RefColor and is large enough to contain the marker generated by the embedder 6. The size of the marker is known in advance, as it is derived from the parameter Par.

[0037] In the example shown in FIG. 3, the markers are made up of pixel blocks 30, each made up of 4x4 pixels and each having a square shape. This allows for resilience to any type of compression, countering the effects of discarding high-frequency information. Indeed, if a compression algorithm modifies the edge pixels of a pixel block, the central region will be relatively unaffected. As a result, the square shapes may be considered to define sacrificial pixels (the outer edges of the pixel block), thereby ensuring the algorithm's agnosticism and ability to recover from compression artifacts. More generally, pixel blocks 30 can be rectangular or square in shape, as long as both dimensions are three or longer.

[0038] In the header portion 32, the pixel block is repeated twice as block 34, which makes it possible to detect the header portion of the marker even if the mouse pointer is over it, as in the case of external visual artifacts. The message portion 36 is made up of one pixel block having the same size as the pixel block in the header portion 32.

[0039] The logic behind this difference is that it is important to be able to detect the header portion of the marker, which is intended to be shorter so as not to carry any meaningful information. For this reason, repeating pixel blocks is a good way to provide resilience in contexts where error coding is not available due to the lack of a priori knowledge of the presence and exact location of the header. Conversely, the payload portion of the marker is typically intended to be longer than the header portion and may be subject to error coding, as will appear below, which is more optimal than duplicating pixel blocks. In some embodiments, the header may be made arbitrarily long and the payload may be chosen to be arbitrarily short.

[0040] In various embodiments, the size of the pixel blocks and their shapes may vary and may be different from one another, and the pixel blocks of a message portion may be repeated two or more times.

[0041] Operation 220 is followed by a color encoding determination operation 230 in which a function ColorMix() receives the base color RefColor determined in operation 220 as well as the parameter Par, to determine the set of colors that will be returned in the table EncodingColorBitSequences[] to be used to color the pixel block of the marker. The table EncodingColorBitSequences[] may be two-dimensional, storing both bit sequences and corresponding encoding colors, or may be one-dimensional, with the bit sequence being implicit.

[0042] As mentioned before, the parameter Par contains the size of the bit sequence, which indicates how many different colors are needed to encode different bit sequences. For example, for a bit sequence size of 2, the possible bit sequences are 00, 01, 10, and 11, meaning that four colors are needed to encode different bit sequences based on the reference color RefColor. For a bit sequence size of 3, eight colors are needed. In general, 2^n colors will be needed to encode a bit sequence of size n.

[0043] The applicant has found that using a bit sequence size equal to 2 bits is advantageous in order to reduce the number of colors required to encode the bit sequence (a larger bit sequence size means that a pixel block encodes more bits, but also means that it is more difficult to keep the markers discreet, as there are more different colors).

[0044] There are many methods that can be used to associate a set of colors with a bit sequence. One of these methods is to take the RGB (red, green, blue) values ​​of the reference color RefColor, which is how colors are calculated in most common computer applications, particularly in web applications, and modify the R and / or B channels. One reason for not modifying the G channel is that the human eye is particularly sensitive to green, and not modifying the G channel allows for a less noticeable marker, for example, for steganography. Alternatively, all channels may be modified. For example, a fixed value may be added or subtracted, encoding +3 or −3, to either the R or B channel, or both. To promote unobtrusiveness, a tradeoff is made to ensure a minimum absolute fixed value while ensuring good robustness when compounding. A value of 3 has shown good results in that regard. Applicant has discovered that values ​​up to 6 can be maintained with good unobtrusive results compatible with steganography applications.

[0045] Applicant has also found it advantageous to add a value if the reference color channel value is less than 128, and subtract a value otherwise. Indeed, this allows any reference color RefCol value to be used, even if a fixed value is not used. In an alternative embodiment, operation 220 may be modified to reject starting points that are incompatible with the fixed value, i.e., if they are too close to the RGB extreme limits. For example, by using a fixed value of +3 or −3, operation 220 may reject starting points that are less than 3 or greater than 252, with the color channels R, G, or B being modified. Of course, values ​​other than 3 may be used. Furthermore, two different boundary values ​​may be used.

[0046] This means that for a reference color with RGB values ​​(130,64,127), the encoding is - The bit sequence 00 is (130,64,127) -Bit sequence 01 is (130,64,130) - The bit sequence 10 is (127,64,127) - The bit sequence 11 is (127,64,130).

[0047] Alternatively, the value may be different from 3, may be different for addition and subtraction, may be only addition, or only subtraction, etc.

[0048] Applicant has also discovered that it is advantageous to assign colors to bit sequences in a manner that minimizes the Hamming distance between colors that are likely to be confused with each other. Indeed, the Hamming distance is the default distance when comparing strings (especially strings of bits), which is consistent with the use of Hamming correction, described below, whose error correction capabilities rely on the Hamming distance of encoded and decoded messages.

[0049] For clarity, recall that the Hamming distance between two strings of equal length is the number of positions at which corresponding symbols differ: that is, it measures the minimum number of substitutions required to change one string into another, or the minimum number of errors that can be made to convert one string into another.

[0050] This is done by optimizing the sum of the differences between the Manhattan RGB distances between the colors and the Hamming distances between the corresponding bit sequences, which is equivalent to minimizing the following expression:

[0051]

number

[0052] where A RGB and B RGB specifies a color derived from the reference color RefCol value, bits and B bitsis the color A in the resulting table EncodingColorBitSequences[]. RGB and B RGB Specifies a bit sequence that can be associated with

[0053] For clarity, it is recalled that the Manhattan RGB distance between two colors is the sum of the distances of the respective RGB channels of these colors, i.e., the two colors C1 and C2 |R(C1)-R(C2)| + |G(C1)-G(C2)| + |B(C1)-B(C2)|.

[0054] This allocation can be calculated by the function ColorMix(), or may be pre-calculated in a look-up table for performance purposes.

[0055] Once the header, payload, and encoding color have been determined, a marker may be generated. This is done by three operations: 240, 250, and 260. Operations 240 and 250, along with operation 260, may be performed sequentially or in parallel.

[0056] The marker header generation operation 240 executes a function MarkerHeader() that receives a vector Header[], a table EncodingColorBitSequences[], and a parameter Par and returns a marker header table MarkerHeader[] that contains the colors for the pixel block encoding the marker header.

[0057] As mentioned above, the parameter Par includes the size and shape of the pixel block, as well as redundant parameters for the marker header pixel block. The marker header table MrkHeader[] may implicitly specify pixels starting from the leftmost pixel and proceeding progressively to the right for pixels in a given row, and progressively lower for pixels in the row below.

[0058] The payload encoding operation 250 executes a function Encode() that receives as arguments a payload vector Payload[] and a parameter Par, and returns a vector ErrorCodedPayload[] of bit sequences corresponding to the encoded payload. The purpose of this encoding is primarily to make the payload more robust to very lossy encoding. To do so, Applicant has found it advantageous to take the vector Payload[] and encode it with a Hamming error code, here Hamming(8,4). For example, this means that the vector Payload[] for a payload of 1100 0101 would be encoded as 01111000 01001011.

[0059] The advantage of using the Hamming (8;4) error code is that it provides an additional parity bit that allows detection of one additional error to the optimal Hamming (7;4) error code, and its output length is a multiple of the input length, which eliminates the need for padding. Error codes other than the Hamming error code may be used.

[0060] Encode() then further interlaces the result to spread possible error bursts across different Hamming chunks. Error codes other than Hamming error codes may be used, and interlacing may be optional.

[0061] This means that if the Hamming encoded payload is chopped into blocks of size 8 (the size of a Hamming block) and those blocks are intertwined like two combs (the first bit of the first block is followed by the first bit of the second block, etc., up to the first bit of the last block, then the second bit of the first block is followed by the second bit of the second block, etc.), then 01111000 01001011 becomes 00111010 11000101.

[0062] Finally, the function Encode() stores the result as a bit sequence in the vector ErrorCodedPayload[]. Incidentally, if the size of the bit sequence is equal to 2, the bit sequence can be simultaneously interlaced, which allows the function Encode() to execute faster. As mentioned above, operation 250 may be executed simultaneously with operation 210.

[0063] The marker error coded payload generation operation 260 executes a function MarkerPayload[] that receives as arguments a vector ErrorCodedPayload[], a table EncodingColorBitSequences[] and a parameter Par, and returns a marker payload table MarkerPayload[] that contains the colors for the pixel block that encodes the marker payload.

[0064] The function MarkerPayload[] is very similar to the function MarkerHeader(), except that, as mentioned above, the rules for generating pixel blocks are likely to be different for the payload.

[0065] Finally, once the marker header and marker payload have been generated, the embedder 6 completes its function in an image marking operation 270, which executes a function Embed() that receives the marker header table MarkerHeader[], the marker payload table MarkerPayload[], the starting point StartPoint, and the image Img as arguments, and returns a marker-embedded image EmbeddedImg. The function Embed() simply starts at the starting point StartPoint in the image Img, and replaces the pixels following the starting point StartPoint with pixels colored according to the marker header table MarkerHeader[].

[0066] The result is an image with unobtrusive markers, which, due to the choice of encoding colors, are highly resistant to lossy compression. In the web context described above, this allows for specifying objects or sub-objects without relying on any external libraries or add-on JavaScript, which poses another security risk. As noted above, in some embodiments, the markers may be reduced to just a header, especially if the markers can be placed in areas of interest.

[0067] FIG. 4 illustrates an exemplary embodiment of the functions performed by detector 8 .

[0068] As will be shown below, detector 8 is able to determine whether the input image contains markers introduced by embedder 6. In the above web context, the image sharing device would use embedder 6 upstream in the stream, and the receiving device would use detector downstream in the stream.

[0069] As will appear below, the detector 8 may return the payload message or may return other information related to the use of the embedded image.

[0070] The function in Figure 4 contains several loops that are meant to execute as follows: The image is advantageously analyzed horizontally by processing a number of pixels equal to the size of a pixel block side. Any given loop is therefore only able to detect at most one pixel with information per pixel block and line, making it possible to convert color to information pixel by pixel with great redundancy in order to fully benefit from the robustness of the pixel block design. - The color transitions are compared with the bit sequence transitions of the header in order to try and detect the header as quickly as possible. As soon as a selected number of correctly placed bit sequences are identified, the image analysis is stopped and the payload is tried and decoded. In the absence of information to infer the correct content of the payload, the color decoding of the payload is performed very strictly, and any error in the payload pixel block color detection causes the decoding attempt to fail, facilitating decoding failure due to incorrect message decoding; When the right edge of the image is encountered or a known size of the header is parsed, a new group of pixels is generated until it is considered that the header has been found or parsing all pixels of the image has failed.

[0071] The function starts with an input operation 400 in which the function Inp() receives as arguments the input image Img (the image EmbImg resulting from the embedder 6 in the example shown in FIG. 1 is used) and the parameters Par.

[0072] This can be done by a human machine interface (HMI). Any type of HMI can be used as long as the user provides an interface through specifying a file containing these global variables or otherwise specifying entries in memory 4. For example, this can be done by pointing to a specified area on the videoconferencing application's display, with or without a mouse or keyboard. This can also be done automatically.

[0073] The input operation 400 returns the image analysis starting point specified by the coordinates (i0,j0) of the image. Usually, i0 and j0 will be equal to 0 (meaning starting from the top left corner of the image). However, the parameter Par may indicate differently.

[0074] Operation 400 is followed by an index initialization operation 402 where local variables i, j, and x are initialized. x will identify the different pixel groups that the function is trying to identify using the marker header, while the indices i and j will be used to look up pixel colors in the image.

[0075] After operation 402, the pixel group color search loop begins with a reset pixel group analysis position operation 404 where a local variable y starts at 0. The variable y will indicate the position of the pixel being analyzed within the respective pixel group.

[0076] Consequently, after operation 404, four operations 406, 408, 410, 412 are executed sequentially or in parallel, in which the four groups of pixels of increasing x index are filled in their positions y by a function Col() that receives the input image Img, the increment value of index i, and the index j as arguments and returns the color in the table of pixel groups GP[].

[0077] Because the input Img may have undergone a high degree of lossy compression before being received by the detector 8, there is a high probability that the color of the pixel will not match the color in the table EncodingColorBitSequences[]. As a result, the function Col() attempts to determine which of the colors in the table EncodingColorBitSequences[] is closest to the color of the given pixel.

[0078] This distance may be based on the Manhattan RGB distance within a tolerance threshold. While using a tolerance value of 0 would obviously result in false negatives, applicants have found that it is desirable for the tolerance threshold to be no greater than the minimum color distance between the closest elements of the table EncodingColorBitSequences[]; a tolerance threshold that is too large can cause completely unrelated colors to be interpreted as corresponding to colors in EncodingColorBitSequences[], leading to false positive marker header detections. Applicants have found a reasonable empirical value to be half the minimum color distance between the closest elements of the table EncodingColorBitSequences[], e.g., [1 / 2 × d(RGB(±0,±0,±0),RGB(±0,±0,±3)] = 2. If the function Col() is not able to associate a color from the table EncodingColorBitSequences[] with the analyzed pixel, the table element GP[x,y] is either left invalid or filled with a value indicating the failed correspondence.

[0079] Looking at operations 406-412, it appears that this loop fills table GP[] as a horizontal sliding window of image Img. Consequently, operation 412 is followed by operation 414, in which index i is incremented by 4 to analyze the next four horizontal pixels of image Img. Note that if the header pixel block is repeated during the filling process, index i may be increased by 8 or another multiple of 4. This process and the horizontality of the markers makes it possible to take advantage of a speculation cache in the CPU, which, based on a given pixel of a given line, fills the cache with the remainder of that image line (as the image is stored in memory pixel by pixel, from left to right, then top to bottom). The analysis can be done differently, at the cost of reduced efficiency.

[0080] Operation 414 is followed by header detection operation 416, which determines whether one of the pixel groups contains a set of colors that corresponds to a marker header. To do so, the function HdDtct() is executed in operation 416 to compare the color transitions exhibited in the pixel group GP[x] with the color transitions predicted from the marker header. If a selected number of exactly corresponding transitions, e.g., two out of three, are found in those predicted spots in the header, the marker header is considered to have been potentially identified in the image Img.

[0081] In this case, operation 416 is followed by a payload detection operation 418 in which the function PLDtct() is executed. This function horizontally analyzes all pixels from the marker header presented below to the marker's payload length. Each pixel is processed by the function Color() to determine whether it is associated with a color in the table EncodingColorBitSequences[ ], which then attempts to perform decoding of the resulting payload. This is done by first deinterlacing the bit sequence corresponding to the detected color and then Hamming decoding the resulting message (or decoding a function of the error code used by the embedder 6). Of course, if interlacing and / or error coding were not used, deinterlacing and decoding are not performed. In the absence of information to infer the correct content of the payload, the color decoding of the payload in the function PLDtct() is performed very strictly. Any error in the payload pixel block color detection will cause the function PLDtct() to return a negative result, facilitating a decoding failure due to the decoding of an incorrect message.

[0082] If the decoding is successful, the marker header is considered to have been successfully identified. Therefore, operation 418 is followed by an optional operation 419 in which a function Ref() receives the table GP[] and the image Img as variables and returns the leftmost position of the determined marker header.

[0083] The function Ref() is responsible for searching the leftmost pixel of the pixel group that resulted in the successful detection of the marker header to see if there is a better starting analysis point. To this end, the function Ref() restores all pixels based on the starting point of a window with the marker header as its upper left corner, the leftmost pixel of the pixel group that resulted in the successful detection, and sides the size of the pixel block side. Then, for each resulting candidate marker header, the function Ref() calculates the sum of the Manhattan RGB distances between the candidate marker header's pixels and the marker header's color intensity as embedded. Advantageously, these Manhattan RGB distances are constrained by the maximum Manhattan RGB distance between all encoded colors to account for cases where the marker header is at least partially obscured (e.g., by the mouse cursor, which would generate an abnormally high Manhattan RGB value). The calculation by the function Ref() can be best seen as an optimization of the following equation:

[0084]

number

[0085] where:

[0086]

number

[0087] is the value to be optimized, and M W and M H where ∇ represents the width and height of the marker reconstructed from the decoded payload using the exact speck colors, ∇ is the search area, and maxdist is the Manhattan RGB distance between the two furthest colors in the table EncodingColorBitSequences[].

[0088] The function Ref() then returns the best candidate, i.e., the starting point with the smallest sum of Manhattan RGB distances. The detection then terminates with the return of the starting point and / or the decoded payload. Functions other than Manhattan RGB distance may be used, but that constitute the best combination with the use of Hamming error coding according to the applicant's research.

[0089] If either function HdDtct() or PLDtct() returns a negative value, it means that the next pixel group must be analyzed and tested. First, operation 420 executes function GP(), which tests whether the current loop has filled the pixel group up to the length of the marker header. If not, the index y is incremented in operation 422, and the loop resumes using operations 406-412.

[0090] If the pixel group has been filled, then these candidates are not valid and operation 424 executes function I() to determine if the right edge of the image img has been reached. If not, a new group is filled in operation 426 by incrementing index x by 4 and operation 404 resets index y to 0, restarting the loop with the new pixel group.

[0091] If the right edge of image Img has been reached, index j is incremented in operation 428 to analyze the next line of image Img, and function J() is executed in operation 430 to determine if the bottom right corner of image Img has been reached. If so, no marker header has been found, and the function ends in operation 432. If not, index i is reset to 0 in operation 434, index x is incremented by 4 in operation 426, and a new group is filled by resetting index y to 0 in operation 404, restarting the loop with the new group of pixels.

[0092] This results in filling the buffer horizontally, which is believed to be particularly advantageous, as horizontal filling allows taking advantage of the processor's speculative cache.

[0093] The use case of the above invention is related to sharing one's screen or window. Traditionally, web browsers have not provided the possibility to detect a DOM (Document Object Model) element within a frame of a getDisplayMedia stream in order to take a screenshot of this element without using html2canvas (which is the method used to create screenshots using JavaScript). For better understanding, getDisplayMedia() is a method of the MediaDevices interface that prompts the user for permission to capture the contents of the display or a part of it (such as a window) as a MediaStream; the resulting stream can be recorded using the MediaStream Recording API or transmitted as part of a WebRTC session. The present invention allows for easy sharing of a part of a screen or window without using these traditional means.

[0094] The above are intended only as examples of the use of the present invention and should not be used to limit its scope, as there are many other settings in which it can be used.

Claims

1. 1. A computer-implemented method for embedding markers into image or video content, comprising: a) receiving an input image or frame for embedding; b) determining a binary message encoded within said input image or frame comprising two or more bit sequences having the same number of bits, said binary message comprising at least a header portion comprising at least two consecutive bit sequences that are not identical; c) detecting a region in the input image or frame such that the color within the region is uniform and the region exhibits a selected length and height; d) associating each possible bit sequence with a corresponding coded color determined from the colors in the uniform region and a coding rule, such that each of the colors is different from the others; e) generating a marker color table, each element of which stores an encoding color associated with a bit sequence of the binary message, such that the color table constitutes a color encoding of the binary message; f) embedding the marker into the region by directionally adding pixel blocks containing at least a selected number of pixels in the direction of addition, wherein pixels in a given pixel block are each colored with the encoding color of an element of the marker color table, each element of the marker color table being associated with at least one pixel block; method.

2. Operation e) uses an additional horizontal direction to create a pixel block including four pixels in the additional direction and having a square shape. The computer-implemented method of claim 1 .

3. and operation e) further comprises repeating each pixel block of the header portion a selected number of times before adding the next pixel block.

3. The computer-implemented method of claim 1 or 2.

4. the header portion includes bit sequences arranged such that each bit sequence is different from its immediately adjacent bit sequences, and each combination of two adjacent bit sequences is unique in the header portion; 3. The computer-implemented method of claim 1 or 2.

5. the binary message includes a header portion and a payload portion including at least two bit sequences, and operation b) includes applying an error correction code to the payload portion to obtain a robust payload portion, and interlacing the robust payload portion; 3. The computer-implemented method of claim 1 or 2.

6. 1. A computer-implemented method for embedding markers into image or video content, comprising: a) receiving an input image or frame for detection; b) analyzing the input image or frame by pixel groups aligned along an image analysis direction, the pixels of each pixel group being separated in the image in the image analysis direction by a number of pixels equal to the size of a pixel block, the pixel groups having a size of the number of bit sequences in the header; c) for each group of pixels, finding the color of each pixel and matching said color to one of the coded colors of the bit sequence; d) comparing the resulting set of bit sequences with the set of bit sequences of the header portion taking into account their positions, and upon detecting a selected set of matching bit sequences, storing the position of a first pixel in the corresponding pixel group as a potential marker start position; method.

7. and operations b), c), and d) are performed sequentially by analyzing a number of pixel groups equal to the number of pixels selected in the additional direction in parallel. The computer-implemented method of claim 6.

8. The image analysis direction is horizontal.

8. The computer-implemented method of claim 6 or 7.

9. the binary message encoded in the embedded marker further comprises a payload portion, and the method further comprises an operation e) of analyzing the image according to the image analysis direction to search for all pixels corresponding to pixel blocks of the payload portion, detecting a color of each pixel, and matching said color with one of the coded colors of the bit sequence; The computer-implemented method of claim 6.

10. If operation e) fails to match the pixel with the encoded color, then resume operation d) from operation b); 10. The computer-implemented method of claim 9.

11. the payload portion is coded according to claim 5, further comprising an operation f) of decoding the bit sequence accordingly, and if operation f) is successful, searching an area around the start of a potential marker in an additional direction with a square window having a size equal to the selected number of pixels, the window being centred on the start of the potential marker and identifying the pixels that implicitly define an image area that best corresponds in colour to the originally embedded marker.

11. A computer-implemented method according to claim 9 or 10.

12. If operation f) fails to decrypt the payload portion, operations b) to d) are resumed. The computer-implemented method of claim 11.

13. 10. A method for implementing the method of claim 1, comprising: Computer program.

14. having recorded thereon the computer program of claim 13, Data storage medium.

15. 14. A computer system comprising a processor coupled to a memory, the memory having recorded thereon the computer program of claim 13. Computer system.

Images