Image processing method and device

Abstract

The invention provides an image processing method and device, and relates to the technical field of image processing, and the method can encode at least one unit pattern of a first part in a graphic code with at least three different pattern parameters to obtain at least three frames of images, so that when the graphic code is decoded based on the at least three frames of images, the image processing efficiency is improved. The graphic code can be obtained by decoding correctly without being influenced by refresh fringes.

Description

Image processing method and device Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image processing method and apparatus. Background Art

[0002] Graphic codes (such as bar codes, QR codes, etc.) can be used to carry information, so that electronic devices can decode the graphic codes to obtain the information.

[0003] In related technologies, graphic codes can be encoded and hidden within images or videos for alternating display at the transmitter. However, due to factors such as frame-by-frame image refresh and the rolling shutter exposure of the sensor capturing the graphic codes, the target image captured by the receiver, which is alternating with the image displayed by the transmitter, may contain a fusion area between the previous and next frames (also known as a refresh stripe). If the area of ​​the captured target image displaying this refresh stripe falls within the display area where the graphic code is located, the portion of the graphic code located in that display area cannot be decoded, resulting in the graphic code being unable to be correctly decoded.

[0004] Summary of the Invention

[0005] To address the aforementioned technical issues, the present application provides an image processing method and apparatus. In this method, at least one unit pattern in the first portion of a graphical code can be encoded using at least three different pattern parameters to generate at least three frames of images. Thus, when decoding the graphical code based on the at least three frames of images, the graphical code can be accurately decoded without being affected by refresh streaks.

[0006] In one possible implementation, the present application provides an image processing method. The method includes: obtaining a first graphic code, the first graphic code including a first portion for representing target information; encoding the first graphic code to generate at least three frames of images of the first graphic code, the at least three frames of images including a first image, a second image, and a third image; wherein at least one first unit pattern in the first portion of the first image is encoded with a first image parameter, the at least one first unit pattern in the first portion of the second image is encoded with a second image parameter, and the at least one first unit pattern in the first portion of the third image is encoded with a third image parameter; and alternately displaying the at least three frames of images to transmit the target information.

[0007] The first graphic code is a machine-readable code, and the pattern in the first graphic code can be any pattern, such as small square grids, dots, etc.

[0008] In addition, the same or similar patterns in the first graphic code may be distributed in different positions in the graphic code, and the pattern in the first graphic code where the same or similar patterns exist may be referred to as a unit pattern.

[0009] For example, when the first graphic code is a two-dimensional code, the unit pattern may be a small black grid or a small white grid.

[0010] When the first graphic code is a barcode, the unit pattern may be a black stripe pattern or a white stripe pattern.

[0011] In addition, the distribution of the unit patterns in the first graphic code may be continuous or discrete, which is not limited here.

[0012] The pattern can be a continuous block pattern or a discrete dot pattern.

[0013] Furthermore, the first graphical code may include a first portion, and optionally, may include a second portion.

[0014] The first part may be composed of a pattern representing bit 1 in the first graphical code, and the second part may be composed of a pattern representing bit 0 in the first graphical code. For example, if the first graphical code is a black and white two-dimensional code, the unit pattern in the black and white two-dimensional code may include black small grids and white small grids, wherein the black small grids are the unit pattern representing bit 1, and the white small grids are the unit pattern representing bit 0.

[0015] Alternatively, the first portion may be a pattern representing bit 0 in the first graphic code, and the second portion may be a pattern representing bit 1 in the first graphic code.

[0016] Furthermore, the first graphical code may include the first portion but not the second portion. For example, if the first graphical code is a circular code, the dots displayed in the circular code represent bit 1, while other positions where dots are not displayed represent bit 0, such that the second portion representing bit 0 may not be displayed in the circular code.

[0017] The first graphic code includes the first part, so that the target information represented by the first graphic code can be represented by the bit 1 or bit 0 represented by the unit pattern of each pixel position in the first part. The target information can be any information carried in the first graphic code, such as a link, a password, an account number, etc., which can be configured according to needs and is not limited here.

[0018] Taking this method applied to the sending end, the first graphic code is a black and white two-dimensional code as an example. For example, if the first part is all black small squares, then the unit pattern included in the first part is black small squares. Then the black small squares located at the same pixel position (for example, the first row and first column) in the black and white two-dimensional code can be encoded with at least three image parameters (for example, encoded as three colors represented by a, b, and c respectively); although the black small squares located at another pixel position (for example, the first row and second column) in the black and white two-dimensional code are also encoded with at least three image parameters, for example, encoded as 3 colors or 3 brightnesses, the encoding result of the black small squares at the other pixel position in the black and white two-dimensional code (for example, encoded as three colors represented by b, c, and a respectively, or encoded as three colors represented by a', b', and c' respectively) can be different from or the same as the encoding result of the black small squares at the above-mentioned other positions (for example, the above-mentioned first row and first column), and there is no limitation here.

[0019] In this way, the sending end encodes each black square in the black and white QR code with at least three image parameters, and the three image parameters can be brightness or color parameters, so that in the at least three frames of images obtained by encoding the black and white QR code, the black squares located at the same pixel position have different brightness (for example, at least 3 brightnesses are encoded respectively), or different colors (for example, at least 3 colors are encoded respectively), or different colors and brightness.

[0020] The transmitting end can alternately display the at least three frames of images obtained by encoding, thereby achieving the transmission effect of the target information carried by the black and white two-dimensional code.

[0021] The alternating display means that the at least three frames of images can be displayed alternately, but there is no restriction on their display order.

[0022] In this way, by alternately displaying the at least three frames of images, the small black grids at the same pixel position in the black and white QR code are alternately displayed with three brightness levels (or three colors) during the alternating display of the at least three frames of images, thereby presenting the effect of light and dark flashing and / or color changes of each small black grid in the black and white QR code, so that the human eye cannot recognize the black and white QR code before encoding from the at least three frames of images displayed alternately.

[0023] This application does not limit the encoding method of the second part of the black and white QR code (for example, the small white grid representing bit 0).

[0024] In some embodiments, the sending end may not encode the second part of the black and white QR code. For example, the original white color (RGB value is 255) is always maintained in the above-mentioned at least three frames of images. However, due to the brightness changes of the black small grids, the brightness of the white small grids can be confused with the brightness of the black small grids after post-encoding, so that the white small grids cannot be recognized by the naked eye in at least three frames of images, thereby achieving the purpose of hiding the black and white QR code.

[0025] Alternatively, in some embodiments, the second part of the black and white QR code can also be encoded, but there is no restriction on how many colors or brightnesses the small white grids at the same pixel position in the second part can be encoded into. For example, they can be the same brightness or color, or they can be encoded into two brightnesses or colors, or three or more brightnesses or colors. There is no restriction here.

[0026] In some embodiments, after the white small squares in the second part of the black and white QR code are encoded, in the above-mentioned at least three frames of images, the color or brightness change of the white small squares in the time sequence is different from the color or brightness change of the black small squares in the same time sequence, so that when the receiving end uses the at least three frames of images for decoding, the difference result of the black small squares in the same time sequence is different from the difference result of the white small squares, so as to distinguish the black small squares from the white small squares.

[0027] For example, the black squares in the black and white QR code are encoded as three brightness levels a, b, and c in the sequentially displayed images 1, 2, and 3, respectively, and the white squares can be sequentially encoded as three brightness levels a, c, and a in the above three frames of images, so that the absolute value of the brightness difference between the white squares in any two frames of images 1 to 3 in time sequence is not exactly the same as the absolute value of the brightness difference between the black squares.

[0028] In related art, the areas representing bit 1 or bit 0 in a graphic code are encoded in two colors to alternately display the two encoded frames. However, due to factors such as frame-by-frame image refresh and the rolling shutter exposure used by the sensor capturing the graphic code, when the receiving end captures the two frames displayed alternately by the transmitting end, the captured image will contain refresh stripes between the current and previous frames (i.e., the fusion area of ​​the two frames). The transmitting end alternates between the two frames, so that the position, brightness, and color of the refresh stripes between the two frames captured by the receiving end are very similar. When the receiving end performs a difference calculation between the two captured frames, the refresh stripes will contain pixels with a difference result of 0, resulting in these pixels being completely black. The receiving end cannot quantize the difference result and cannot binarize the area with the refresh stripes to 1. If the refresh stripe area is the display area of ​​the encoded graphic code, the portion of the graphic code in the area containing the refresh stripe cannot be decoded, resulting in the problem of incorrect decoding of the graphic code.

[0029] However, in an embodiment of the present application, the transmitting end alternately displays the same position (first unit pattern) in the first graphic code and encodes the color and / or brightness separately. For example, the pixel at the same position is encoded into three colors (represented by G, B, and W, respectively). The fusion area in any frame of the image captured by the receiving end is the fusion area of ​​the current frame and the previous frame. However, because the transmitting end encodes the same position of the graphic code into at least the above three colors (and / or three brightnesses), the brightness (or color) of the fusion area (e.g., the refresh stripe) corresponding to the pixel at the same position in any two frames of the image captured by the receiving end (even two consecutive frames) is different. Therefore, when the receiving end performs a difference between the two frames of the image, the difference result of the refresh stripe will not include a position of 0, so that the difference result of the refresh stripe will not contain a completely black area. Therefore, even if the refresh stripe includes the encoded pattern of the first graphic code, the fusion area where the refresh stripe is located can be quantized normally, thereby being binarized to 1, so that the first graphic code can be correctly decoded using the at least three frames of the image displayed alternately.

[0030] In a possible implementation, the at least three frames of images are alternately displayed at a frequency that cannot be perceived by human eyes, for example, the frequency may be 60 FPS.

[0031] In a possible implementation, the at least three frames of images are alternately displayed at a frequency at which the at least three frames of images can be perceived by a sensor associated with the receiving end.

[0032] In a possible implementation, the first image parameter, the second image parameter, and the third image parameter are different in a color space.

[0033] For example, the same unit pattern in the first graphic code is encoded as three colors in a color space, which can be RGB or YUV, without limitation. In this way, the same pixel position in the encoded graphic code can be displayed alternately in at least three colors when at least three frames of images are alternately displayed.

[0034] In some embodiments, the at least three colors may include two colors that are opposite to each other in the color space, such as red and green. When displayed alternately at a higher frequency, the naked eye sees yellow, and the other encoded color may also be yellow or other colors. In this way, the human eye does not perceive the color change of the unit pattern in the first area of ​​the graphic code in the at least three frames of images displayed alternately, thereby achieving the purpose of hiding the graphic code.

[0035] In the embodiment of the present application, pixels at the same position in the first graphic code can be encoded into at least three colors in the color domain to alternately display at least three frames of encoded images. This not only avoids the influence of refresh stripes, but also facilitates the correct decoding of the first graphic code by the receiving end based on the at least three frames of images. In addition, the human eye cannot perceive the color changes of the unit patterns in the graphic code, thereby achieving the purpose of hiding the graphic code.

[0036] In a possible implementation manner, the first image parameter, the second image parameter, and the third image parameter have different brightness.

[0037] For example, the same first unit pattern in the first graphic code is encoded with at least three brightness levels. Thus, the same pixel position in the encoded graphic code can be displayed alternately with at least three brightness levels during the alternating display of at least three frames of images, thereby achieving a visual effect of a flickering unit pattern and facilitating concealment of the graphic code.

[0038] In the embodiment of the present application, pixels at the same position in the first graphic code can be encoded into at least three brightness levels in the brightness domain to alternately display at least three frames of the encoded image. This not only avoids the influence of refresh stripes, but also facilitates the correct decoding of the first graphic code by the receiving end based on the at least three frames of the image. Moreover, each first unit pattern is displayed in a flashing manner, making the first graphic code imperceptible to the human eye, thereby achieving the purpose of concealing the graphic code.

[0039] In a possible implementation manner, the first graphic code further includes a second part; and the second parts of the at least three frames of images are encoded using the same image parameters.

[0040] For an introduction to the second part, refer to the above description. In this embodiment, the second part is illustrated as a pattern representing bit 0. The unit patterns representing bit 0 in the first graphic code are all encoded with the same color, or the same brightness, or the same color and brightness, so that in at least three frames of encoded images, the pattern representing bit 0 does not change in color and / or brightness. When the receiving end performs the corresponding difference processing, the difference result of the area representing bit 0 can be 0 (i.e., a completely black area), while the difference result of the first area representing bit 1 does not contain 0. This allows the receiving end to distinguish between the first and second parts, thereby improving decoding efficiency. Furthermore, the sending end does not need to perform complex encoding on the second part, thereby improving encoding efficiency.

[0041] In a possible embodiment, the first graphic code also includes a second part; at least one second unit pattern in the second part of the first image is encoded with a fourth image parameter, at least one second unit pattern in the second part of the second image is encoded with a fifth image parameter, and at least one second unit pattern in the second part of the third image is encoded with a sixth image parameter.

[0042] Continuing with the black and white QR code as an example, the second unit pattern in the second part can be a small white grid representing bit 0. In this embodiment, the small white grid located at the same pixel position in the black and white QR code can also be encoded in three colors (and / or brightness), so that the at least three frames of encoded images displayed alternately can not only make the black grid have three colors (and / or brightness) changes, but also make the white grid in the black and white QR code have three colors (and / or brightness) changes.

[0043] The encoding strategy for the second unit pattern in this embodiment is similar to the encoding strategy for the first unit pattern in the above embodiment, and will not be further described in detail. For example, the three colors or brightnesses encoded by the white small grids at different pixel positions in a black and white QR code can be the same or different, and this is not limited here.

[0044] In addition, in order to enable the receiving end to decode the first graphic code using the above-mentioned at least three frames of images, the difference result of the first part between two frames of images in the same time sequence in the at least three frames of images encoded by the sending end is different from the difference result of the second part.

[0045] For example, at least three frames of images are image 1, image 2, and image 3. In the settings of the above-mentioned first image parameter to the sixth image parameter, it is necessary to ensure that the absolute value of the difference result of the black area (all black small squares) between at least one group of images (two frames of images among the above-mentioned three frames of images) is different from the absolute value of the difference result of the white area (all white small squares), so that the receiving end can use the different difference results of the two areas to distinguish between the white area and the black area.

[0046] In a possible implementation, the fourth image parameter, the fifth image parameter, and the sixth image parameter are different in a color space.

[0047] For example, the same second unit pattern in the first graphical code is encoded with at least three brightness levels. Thus, the same pixel position in the encoded graphical code can be displayed alternately with at least three brightness levels during the alternating display of at least three frames of images, thereby creating a visual effect of a flickering second unit pattern and facilitating concealment of the graphical code.

[0048] In the embodiment of the present application, pixels at the same position in the first graphic code can be encoded into at least three brightness levels in the brightness domain to alternately display at least three frames of the encoded image. This is not only unaffected by refresh stripes, but also facilitates the correct decoding of the first graphic code by the receiving end based on the at least three frames of image. Moreover, each second unit pattern is displayed in a color-changing manner, making the first graphic code imperceptible to the human eye, thereby achieving the purpose of concealing the graphic code.

[0049] Furthermore, by performing three separate encodings on both the first and second unit patterns, the transmitter can increase the difficulty for the human eye to perceive the hidden first graphical code. Furthermore, this also provides greater flexibility and possibilities for the receiver's decoding strategy when decoding the first graphical code, enriching the decoding methods. For example, when performing a difference calculation between two captured image frames, the receiver is not limited to decoding only when the difference result in black areas does not contain zero (i.e., a completely black area). The receiver can also perform the difference calculation so that the difference result in white areas does not contain zero (i.e., a completely black area) for decoding.

[0050] In a possible implementation, the fourth image parameter, the fifth image parameter, and the sixth image parameter have different brightness.

[0051] For example, the same second unit pattern in the first graphical code is encoded with at least three brightness levels. Thus, the same pixel position in the encoded graphical code can be displayed alternately with at least three brightness levels during the alternating display of at least three frames of images, thereby creating a visual effect of a flickering second unit pattern and facilitating concealment of the graphical code.

[0052] In the embodiment of the present application, pixels at the same position in the first graphic code can be encoded into at least three brightness levels in the brightness domain to alternately display at least three frames of the encoded image. This not only avoids the influence of refresh stripes, but also facilitates the correct decoding of the first graphic code by the receiving end based on the at least three frames of the image. Moreover, each second unit pattern is displayed in a flashing manner, so that the human eye cannot perceive the first graphic code it carries, thereby achieving the purpose of hiding the graphic code.

[0053] Furthermore, by performing three separate encodings on both the first and second unit patterns, the transmitter can increase the difficulty for the human eye to perceive the hidden first graphical code. Furthermore, this also provides greater flexibility and possibilities for the receiver's decoding strategy when decoding the first graphical code, enriching the decoding methods. For example, when performing a difference calculation between two captured image frames, the receiver is not limited to decoding only when the difference result in black areas does not contain zero (i.e., a completely black area). The receiver can also perform the difference calculation so that the difference result in white areas does not contain zero (i.e., a completely black area) for decoding.

[0054] In a possible implementation, the alternately displaying the at least three frames of images to transmit the target information includes: alternately displaying the at least three frames of images on a background image, wherein a pattern in the background image is related to a pattern in the first graphic code.

[0055] Among them, the sending end can display the at least three frames of images in a layer-overlay manner as the upper layer of each frame of the background image, so that the at least three frames of images are fused and displayed alternately in the background image without changing the frame rate of the background image.

[0056] In this embodiment, by displaying at least three encoded frames of images on a background image, and the pattern in the background image is the same as or similar to the pattern in the first graphic code (for example, the first unit pattern and the second unit pattern), the graphic code is less likely to be perceived by the human eye, thereby improving the hiding effect of the graphic code.

[0057] In a possible implementation, the background image is a dynamic image.

[0058] The background image may be a dynamic image, such as a video.

[0059] In this way, when the background image changes dynamically and at least three frames of images displayed after encoding the graphic code also change dynamically, it is more difficult for the human eye to perceive the graphic code, thereby improving the hiding effect of the graphic code.

[0060] In a possible implementation manner, the size of the pattern in the first graphic code is smaller than or equal to the size of the related pattern in the background pattern.

[0061] Among them, the size of the pattern in the first graphic code (for example, at least one of the first unit pattern and the second unit pattern) is smaller than or equal to the size of the related pattern in the background pattern, so that the human eye is less likely to perceive the graphic code, thereby improving the hiding effect of the graphic code.

[0062] In a possible implementation, an average value of the first image parameter, the second image parameter, and the third image parameter is related to an image parameter of the background image.

[0063] The color average (or brightness average) of the three colors or three brightnesses encoded for the same unit pattern can be related (identical or similar) to the color (or brightness) of the relevant pattern in the background image. In this way, the at least three frames of encoded images with color (or brightness) variations can not only be fused on the background image, but also have a high degree of color (or brightness) fusion, making the hidden graphic code difficult to see with the naked eye, thereby preventing the hidden QR code from being exposed.

[0064] The above implementation method regarding the average value of the size and image parameters can also be applied to the three image parameters of the second unit pattern, with similar effects, and will not be repeated here.

[0065] In a possible implementation, encoding the first graphic code to generate at least three frames of images of the first graphic code includes: encoding the first graphic code based on a fourth image to generate at least three frames of images of the first graphic code.

[0066] When encoding the first graphic code into the at least three frames of images, the first graphic code can be encoded not only directly but also based on a fourth image. For example, the color or brightness of the pattern in the fourth image can be used to determine the color or brightness of the corresponding unit pattern in the at least three frames of images. The fourth image can be the background image mentioned in the above embodiment or another image, without limitation.

[0067] In addition, in some embodiments, when the at least three frames of images are alternately displayed, the fourth image or the background image may be displayed in place of the at least three frames of images, or may be displayed superimposed on the fourth image or the background image.

[0068] In this way, the at least three frames of images obtained by encoding are more complex, and it is more difficult for the human eye to directly recognize the original first graphic code.

[0069] In a possible implementation, the alternately displaying the at least three frames of images to transmit the target information includes: alternately displaying the first image, the second image, and the third image, and looping the alternate display process n times, where n is a positive integer.

[0070] The alternating display of the first, second, and third images is not limited to the order of the first, second, and third images; any order of the three images may be used. Furthermore, during each cycle of the alternating display of the three images, the order of display of the three images between different alternating display processes is not limited to being the same or different.

[0071] For example, the first image, the second image, and the third image are represented as image M1, image M2, and image M3, respectively.

[0072] For example, when displayed in a loop, the following sequences may be displayed: image M1, image M2, image M3, image M2, image M1, image M3, image M1, image M2, image M3, image M3, image M2, image M1, etc.

[0073] In this way, by alternately displaying at least three frames of images and cyclically displaying the alternate display process, it is more difficult for the human eye to directly recognize the original first graphic code.

[0074] In a possible implementation, during the n alternating display processes of the at least three frames of images, there are at least two alternating display processes in which the display order of the images is different.

[0075] For example, in a cyclic display, the first alternating display process sequentially displays: image M1, image M2, image M3; the second alternating display process of the cycle sequentially displays: image M2, image M1, image M3. The display order of the at least three frames of images is different between the two alternating display processes.

[0076] In this way, by alternately displaying at least three frames of images and cyclically displaying the alternate display process, it is more difficult for the human eye to directly recognize the original first graphic code.

[0077] In one possible embodiment, the at least three frames of images also include at least one fifth image, wherein the first part of the fifth image is encoded with the first image parameters; and the alternating display of the at least three frames of images to transmit the target information includes: alternating display of the first image, the second image, the third image, and the at least one fifth image, and looping the alternating display process n times.

[0078] The fifth image may be any type of pattern. Optionally, the pattern in the fifth image is related to (identical to or similar to) the pattern in the first graphic code.

[0079] For example, the fifth image is also a graphic code, and the first part of the fifth image (eg, the part representing bit 1) is encoded using the first image parameter.

[0080] When at least three frames of images are displayed alternately, the at least three frames of images further include at least one fifth image, and the fifth images of different frames may be the same image or different images.

[0081] In this way, when the first image, the second image, and the third image are alternately displayed, at least one frame of the fifth image that encodes the first part in a constant sequence can be inserted at the same position during each alternating display process to increase the encoding complexity of the first graphic code and the difficulty for the human eye to recognize the first graphic code from the alternately displayed images.

[0082] In addition, the at least three frames of images may also include a sixth image. Similar to the fifth image, the second part of the sixth image is encoded with the fourth image parameter (or the fifth image parameter or the sixth image parameter).

[0083] In a possible embodiment, the image display order corresponding to the at least one alternating display process of the at least three frames of images is the first image, the second image, and the third image, and the second image parameter is related to the average of the first image parameter and the third image parameter.

[0084] In which, in the process of alternately displaying three frames of images, when they are displayed in the order of the first image, the second image, and the third image, the above-mentioned second image parameters corresponding to the second image displayed in the middle can be the average of the first image parameters and the third image parameters corresponding to the first image and the third image respectively (or close to the average). In this way, when the three frames of images are alternately displayed, the above-mentioned at least one first unit pattern in the first area can be uniformly changed in color or brightness.

[0085] Of course, during the n-times alternating display process of at least three frames of images, the display order of the first image, the second image, and the third image corresponding to other alternating display processes may change.

[0086] Similarly, in some embodiments, the fifth image parameter may also be related to the average of the fourth image parameter and the sixth image parameter.

[0087] In one possible embodiment, the present application provides an image processing method. The method includes: shooting at least three frames of images alternately displayed on a screen of a transmitting end to obtain at least two frames of images; wherein the at least three frames of images are encoding results of a first graphic code; wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first portion of the first image is encoded with a first image parameter, the at least one first unit pattern in the first portion of the second image is encoded with a second image parameter, and the at least one first unit pattern in the first portion of the third image is encoded with a third image parameter; for at least one group of two frames of images in the at least two frames of images, calculating the difference in image parameters; and obtaining the first graphic code based on the difference to obtain target information represented by the first graphic code.

[0088] Taking the method applied to the receiving end as an example, the receiving end can shoot at least three frames of images alternately displayed after the first graphic code is encoded by the sending end. It can shoot two frames of images, or three frames of images, or more than three frames of images.

[0089] Among them, at least one first unit pattern of the first part of the at least three frames of images alternately displayed by the sending end is encoded with different image parameters. The specific encoding process can refer to the various embodiments of the above-mentioned image processing method applied to the sending end, and will not be repeated here.

[0090] Due to the effects of frame-by-frame image refresh and the use of rolling shutter exposure by the sensor that captures the graphic code, when the receiving end captures the two frames of images alternately displayed by the transmitting end, there is a high probability that the captured image will contain refresh stripes between the current frame and the previous frame (i.e., the fusion area of ​​the two frames). However, because the corresponding pixel position in the at least three frames of images encoded by the transmitting end can be encoded differently in the color domain and / or brightness domain, taking brightness as an example, the first unit pattern displayed alternately at three brightness levels in the at least three frames of images will not have exactly the same brightness in the fusion area in the at least two frames of images captured by the receiving end. Therefore, the receiving end can perform a difference on at least one group of images (wherein a group of images includes two frames) from the at least two frames of images captured. The difference result indicates that there must be no completely black area with a value of 0 at the position corresponding to the refresh stripes. Therefore, the receiving end can decode the first graphic code based on this difference, thereby obtaining the target information expressed by the first graphic code.

[0091] In a possible implementation, each of the at least two frames of images includes a fusion area, wherein the fusion area is a fusion image of two adjacently displayed frames of images among the at least three frames of images displayed on the screen.

[0092] Each of the at least two frames of images captured by the receiving end may include a fusion area (eg, refresh stripes) between the current frame and the previous frame displayed by the sending end.

[0093] In one possible embodiment, the at least two frames of images include a fourth image and a fifth image captured sequentially; and calculating the difference in image parameters of at least one group of two frames of images among the at least two frames of images includes calculating the difference in image parameters of the fourth image and the fifth image.

[0094] The receiving end can capture at least three frames of images alternately displayed by the transmitting end, and sequentially capture a fourth image, a fifth image, and so on. The receiving end can perform a difference calculation on two consecutively captured images (here, the fourth and fifth images). Because the transmitting end encodes at least one first unit pattern in the first area with three pattern parameters to obtain at least three displayed images, the difference result here does not contain any pixels with a difference of 0 in the refresh stripe (blending area). The receiving end can use this set of difference values ​​to directly decode and obtain the first graphic code.

[0095] In a possible implementation manner, the number of the at least two frames of images is 3 or more.

[0096] In this embodiment, the receiving end may continuously shoot three or more frames of images when shooting at least three frames of images alternately displayed by the sending end.

[0097] In one possible embodiment, calculating the difference in image parameters for at least one group of two frames of the at least two frames of images includes: calculating the difference in image parameters for at least two groups of images among the at least three frames of images captured, respectively, to obtain at least two groups of differences, wherein each group of the at least two groups of images includes two frames of images; and obtaining the first graphic code based on the differences to obtain the target information represented by the first graphic code includes: fusing the at least two groups of differences to obtain the first graphic code.

[0098] Considering that using only two captured image frames for subtraction may not fully decode the first graphical code based on this single subtraction result, and may only decode a portion of the first graphical code. Therefore, in this embodiment, the receiving end can select at least two sets of images from three or more captured image frames and perform subtraction on each. The receiving end can then fuse the at least two sets of subtraction results to obtain the first graphical code, thereby improving decoding accuracy.

[0099] For example, the first and third frames of the shot are selected to calculate the difference, and the first and second frames of the shot are selected to calculate the difference, thereby obtaining two sets of difference results.

[0100] In one possible implementation, the fusing of the at least two groups of differences to obtain the first graphic code includes: binarizing the at least two groups of differences to obtain at least two groups of binarization results; and performing an AND operation on the at least two groups of binarization results to obtain the first graphic code.

[0101] In this embodiment, when fusing at least two groups of difference results, each group of difference results can be first binarized to obtain a binary image (with 1 or 0 representing each pixel point), and then a logical AND operation is performed on the at least two groups of binarized results to obtain a first graphic code.

[0102] In a possible implementation, fusing the at least two sets of difference values ​​to obtain the first graphic code includes: adding the at least two sets of difference values ​​to obtain an addition result; and binarizing the addition result to obtain the first graphic code.

[0103] In this embodiment, when fusing at least two groups of difference results, each group of difference results may be first added pixel by pixel, and then the added result may be binarized to obtain the first graphic code.

[0104] In one possible embodiment, the present application provides an image processing device. The image processing device includes: an acquisition module for capturing at least three frames of images alternately displayed on a screen of a transmitting end to obtain at least two frames of images; wherein the at least three frames of images are encoding results of a first graphic code; wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, the at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and the at least one first unit pattern in the first part of the third image is encoded with a third image parameter; an encoding module for calculating the difference in image parameters for at least one group of two frames of the at least two frames of images; and a display module for acquiring the first graphic code based on the difference to obtain target information represented by the first graphic code.

[0105] In a possible implementation, the first image parameter, the second image parameter, and the third image parameter are different in a color space.

[0106] In a possible implementation manner, the first image parameter, the second image parameter, and the third image parameter have different brightness.

[0107] In a possible implementation manner, the first graphic code further includes a second part; and the second parts of the at least three frames of images are encoded using the same image parameters.

[0108] In a possible embodiment, the first graphic code also includes a second part; at least one second unit pattern in the second part of the first image is encoded with a fourth image parameter, at least one second unit pattern in the second part of the second image is encoded with a fifth image parameter, and at least one second unit pattern in the second part of the third image is encoded with a sixth image parameter.

[0109] In a possible implementation, the fourth image parameter, the fifth image parameter, and the sixth image parameter are different in a color space.

[0110] In a possible implementation, the fourth image parameter, the fifth image parameter, and the sixth image parameter have different brightness.

[0111] In a possible implementation, the display module is specifically configured to alternately display the at least three frames of images on a background image, and a pattern in the background image is related to a pattern in the first graphic code.

[0112] In a possible implementation, the background image is a dynamic image.

[0113] In a possible implementation manner, the size of the pattern in the first graphic code is smaller than or equal to the size of the related pattern in the background pattern.

[0114] In a possible implementation manner, an average value of the first image parameter, the second image parameter, and the third image parameter is related to an image parameter of the background image.

[0115] In a possible implementation, the encoding module is specifically configured to encode the first graphic code based on the fourth image to generate at least three frames of images of the first graphic code.

[0116] In a possible implementation, the display module is specifically configured to alternately display the first image, the second image, and the third image, and to loop the alternate display process n times, where n is a positive integer.

[0117] In a possible implementation, during the n alternating display processes of the at least three frames of images, there are at least two alternating display processes in which the display order of the images is different.

[0118] In one possible embodiment, the at least three frames of images also include at least one fifth image, wherein the first part of the fifth image is encoded with the first image parameters; the display module is specifically used to alternately display the first image, the second image, the third image, and the at least one fifth image, and to loop the alternating display process n times.

[0119] In a possible embodiment, the image display order corresponding to the at least one alternating display process of the at least three frames of images is the first image, the second image, and the third image, and the second image parameter is related to the average of the first image parameter and the third image parameter.

[0120] The effects of the image processing devices of the above embodiments are similar to the effects of the image processing methods of the above embodiments, and are not described in detail here.

[0121] In one possible embodiment, the present application provides an image processing device. The image processing device includes: a shooting module, configured to shoot at least three frames of images alternately displayed on a screen of a transmitting end to obtain at least two frames of images; wherein the at least three frames of images are encoding results of a first graphic code; wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, the at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and the at least one first unit pattern in the first part of the third image is encoded with a third image parameter; a calculation module, configured to calculate the difference in image parameters for at least one group of two frames of the at least two frames of images; and an acquisition module, configured to acquire the first graphic code based on the difference to obtain target information represented by the first graphic code.

[0122] In a possible implementation, each of the at least two frames of images includes a fusion area, wherein the fusion area is a fusion image of two adjacently displayed frames of images among the at least three frames of images displayed on the screen.

[0123] In a possible implementation, the at least two frames of images include a fourth image and a fifth image captured sequentially; and the calculation module is specifically configured to calculate a difference in image parameters between the fourth image and the fifth image.

[0124] In a possible implementation manner, the number of the at least two frames of images is 3 or more.

[0125] In one possible embodiment, the calculation module is specifically used to calculate the difference in image parameters for at least two groups of images among the at least three frames of images captured, thereby obtaining at least two groups of difference values, wherein each group of the at least two groups of images includes two frames of images; and the acquisition module is specifically used to fuse the at least two groups of difference values ​​to obtain the first graphic code.

[0126] In a possible implementation, the acquisition module is specifically configured to: perform binarization processing on the at least two groups of differences to obtain at least two groups of binarization results; and perform an AND operation on the at least two groups of binarization results to obtain the first graphic code.

[0127] In a possible implementation, the acquisition module is specifically configured to: add the at least two groups of difference values ​​to obtain an addition result; and perform binarization processing on the addition result to obtain the first graphic code.

[0128] The effects of the image processing devices of the above embodiments are similar to the effects of the image processing methods of the above embodiments, and are not described in detail here.

[0129] In one possible implementation, the present application provides an image processing device. The image processing device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from a memory and send the signals to the processors, the signals including computer instructions stored in the memory; when the processors execute the computer instructions, the processors can implement the image processing method described in any of the above implementations.

[0130] The effects of the image processing device of this embodiment are similar to the effects of the image processing methods of the above embodiments, and will not be described in detail here.

[0131] In one possible implementation, the present application provides a computer-readable storage medium storing a computer program, which, when executed on a computer or processor, causes the computer or processor to execute the image processing method in any one of the aforementioned implementations.

[0132] The effects of the computer-readable storage medium of this embodiment are similar to the effects of the image processing methods of the above embodiments, and are not described in detail here.

[0133] In one possible implementation, the present application provides a computer program product, which includes a software program, and when the software program is executed by a computer or a processor, the image processing method in any one of the above implementations is executed.

[0134] The effects of the computer program product of this embodiment are similar to the effects of the image processing methods of the above embodiments, and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0135] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0136] FIG1 is a schematic diagram of an exemplary QR code;

[0137] FIG2 is a schematic diagram showing an exemplary image refresh display;

[0138] FIG3 is a schematic diagram illustrating an exemplary process of image capture;

[0139] FIG4a is a schematic diagram illustrating an image processing process in the prior art;

[0140] FIG4 b is a schematic diagram illustrating an image processing process of the present application;

[0141] FIG5a is a schematic diagram of an image decoded in the prior art;

[0142] FIG5 b is a schematic diagram showing an exemplary image decoded by the present application;

[0143] FIG6a is a schematic diagram illustrating an exemplary processing process of a sending end;

[0144] FIG6 b is a schematic diagram illustrating an exemplary processing process of the sending end;

[0145] FIG7a is a schematic diagram showing a plurality of graphic codes;

[0146] FIG7 b is a schematic diagram of an exemplary ring code;

[0147] FIG8 is a schematic diagram exemplarily illustrating an image decoded by a receiving end;

[0148] FIG9 is a schematic structural diagram of a device provided in an embodiment of the present application;

[0149] FIG10 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0150] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0151] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0152] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0153] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0154] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0155] A QR code, also known as a two-dimensional barcode, uses black and white geometric patterns distributed in a fixed pattern across a plane (two dimensions) to record data symbols. The code cleverly utilizes the concept of "0" and "1" bit streams, which form the foundation of computer logic, using geometric shapes corresponding to binary numbers to represent textual numerical information. It is automatically recognized by image input devices or photoelectric scanning equipment for automatic information processing. It shares some common characteristics with barcode technology: each code system has its own specific character set; each character occupies a certain width; and has certain verification functions. It also features automatic recognition of information in different rows and the ability to handle image rotation changes.

[0156] In order for the computer to recognize the QR code, it is necessary to use the computer's internal logic: use the numbers "0" and "1" as codes, and use several geometric shapes corresponding to binary to represent textual numerical information. The white block in the QR code represents binary "0", and the black block represents binary "1". The camera can interpret the information contained in the QR code by recognizing the color and arrangement.

[0157] As shown in Figure 1, a standard QR code can be identified based on six components:

[0158] Quiet zone 101: This is the blank border around the QR code. Without this border, the QR code reader would be unable to determine what is and is not included in the QR code due to interference from external factors.

[0159] Position pattern 102: A QR code typically contains three black squares in the lower left, upper left, and upper right corners. These squares tell the QR code reader that it is looking at a QR code and the location of the QR code's outer border.

[0160] Alignment pattern 103: For example, another smaller square located somewhere near the lower right corner of the QR code, used to ensure that the QR code can still be read when tilted or at an angle.

[0161] Timing pattern 104: It is an L-shaped line between the three squares of the finding pattern 102. The timing pattern 104 helps the reader identify the individual squares in the entire QR code, while making it possible to read a damaged QR code.

[0162] Version information 105: For example, a small information area near the image-finding pattern 102 in the upper right corner of the QR code. It identifies the version of the QR code being read.

[0163] Data cell 106: The actual information conveyed by the remaining small squares of the QR code, that is, the URL, phone number or message contained therein.

[0164] A QR code is a graphical code. In this application, a graphical code can be used to carry target information (such as a link, phone number, message, account password, etc., without limitation). The graphical code can also include a barcode, a pattern code including a custom pattern, etc., without limitation. Regardless of the type of graphical code, similar to a QR code, the graphical code can include a graphic corresponding to "0" or "1" to indicate the target information carried by the graphical code. The electronic device decodes the graphic representing "0" or "1" in the graphical code to obtain the target information.

[0165] At present, display devices such as electronic billboards, computers and TV screens composed of light-emitting diodes (LEDs) or liquid crystal displays (LCDs) can be seen everywhere. These display devices can display information, such as videos or images. In some scenarios, the existing technology can embed some target information (such as links, account passwords, etc., without limitation) in the form of graphic codes (such as QR codes, bar codes, pattern codes including custom patterns, etc.) in the above-mentioned video or image displayed by the display device based on the information push method of visible light communication (VLC), and make it impossible for the human eye to perceive the embedded graphic code through the video or image displayed by the display device. The electronic device can take a picture of the display content of the display device and decode the captured image to obtain the graphic code embedded in the video or image, thereby obtaining the target information.

[0166] In some application scenarios, the display device can display a video of an actor's outfit. The electronic device captures the outfit image displayed on the display device and decodes the captured outfit image to obtain target information such as product introductions and purchase links about the clothes embedded in the outfit video.

[0167] In some application scenarios, electronic devices can be interconnected to synchronize data, such as a new phone connecting to an old one to clone the old phone's data, or a phone connecting to a smartwatch. When these electronic devices are interconnected, the screen of one device can carry a graphic code within an image or video. The color, brightness, and layout of this graphic code are related to the image or video. This means that although the screen displays both the image or video and the graphic code, the graphic code cannot be discerned from the image or video by the naked eye, effectively hiding the graphic code.

[0168] For example, in a scenario where a new phone and an old phone are interconnected to clone the data of the old phone to the new phone, the above image is an image composed of black and white grids similar to the QR code graphic, and a QR code is also displayed on the image in a layered manner. The QR code carries the target information of the Wi-Fi service set identifier (SSID) and password for establishing a connection. Then the old phone displays the image carrying the QR code on the screen, and the new phone scans the image to decode the QR code carried in the image, and processes the QR code to obtain the Wi-Fi SSID and password, thereby realizing the interconnection between the old phone and the new phone and performing data cloning.

[0169] For another example, in the scenario where a mobile phone and a smart watch are interconnected, the screen of the smart watch can display a particle motion effect, which also hides a graphic code composed of particles that are invisible to the human eye. The graphic code carries target information for interconnection. The mobile phone shoots the particle motion effect displayed on the smart watch and decodes the captured picture frames (frame sequence) to obtain the target information expressed by the hidden graphic code, so as to use the target information to realize the interconnection between the mobile phone and the smart watch.

[0170] In the related art, when an electronic device as a transmitter embeds target information into an original image or video in the form of a graphic code, taking the graphic code as a QR code as an example, and other graphic codes in the same way, the electronic device can encode the black area in the QR code (the area where all black grids are located) into a color A1 (for example, red), and the white area in the QR code (the area where all white grids are located) into a color B1 (for example, green), to obtain a frame of encoded image C1; encode the black area in the QR code (the area where all black grids are located) into another color A2 (for example, green); and encode the white area in the QR code (the area where all white grids are located) into another color B2 (for example, red), to obtain a frame of encoded image C2. Among them, the color A1 is different from the color A2, the color B1 is different from the color B2, and whether the color A1 and the color B1 are the same is not limited, that is, the related art can encode the white area in the QR code into two colors and the black area into two colors. In this way, an encoded graphic code can be obtained, which is output on the display of an electronic device as two alternating frames (respectively, image C1 and image C2) displayed at a fast rate, such as 60 frames per second (e.g., a playback frequency of 60 Hertz (Hz)). Specifically, when red and green are rapidly refreshed and displayed, they appear yellow to the human eye. Thus, both the black and white areas of the graphic code appear to the human eye as flickering yellow. The graphic code can be used as a background image or part of a background image in the original image or video output by the electronic device, allowing the graphic code to be hidden in the original image or video displayed on the display screen. In this way, this related technology can utilize the fact that the human eye's visual temporal memory is insensitive to color to achieve hidden encoding of QR codes.

[0171] Therefore, the related art adopts a method of encoding the graphic code into two frames of images with different colors (such as the above-mentioned image C1 and image C2) to realize the encoding of the graphic code.

[0172] However, when another electronic device serving as the receiving end captures the two frames of images alternately displayed on the display screen of the electronic device serving as the transmitting end, it will capture a fusion of the two frames of images (also known as the refresh stripes of the two frames of images). If there are refresh stripes in the image captured by the receiving end, and the refresh stripes contain a graphic code hidden in the area where the graphic code is located in the captured image, the receiving end will be unable to decode the complete graphic code, thereby resulting in the problem of being unable to correctly decode the target information.

[0173] The following Figures 2, 3, 4a and 5a describe in detail the above technical problems existing in the methods in the related art.

[0174] FIG2 is a schematic diagram showing the principle of refreshing and displaying an image on the screen of the transmitter. As shown in FIG2 , the screens of current electronic devices (such as the transmitter) all use a line-by-line refresh method to refresh and display image frames. FIG2 takes the refreshing and displaying of black and white images as an example. FIG2 (1) shows the i-th frame (for example, i=1, not limited) image displayed on the screen of the transmitter, which is a black image. Then, as shown in FIG2 (2), the image displayed on the screen in the frame refresh state, the transmitter can refresh the image displayed on the screen line by line. During the frame refresh process, the i+1-th frame image, that is, the next frame image (for example, a white image), can replace the previous frame image (here, a black image) line by line, so that the display range of the next frame image becomes larger and larger until it fills the entire screen, so as to refresh and display the i+1-th frame image (here, a white image) shown in FIG2 (3).

[0175] In the example of Figure 2, the screen of the transmitter displays a black image in the i-th frame and a white image in the i+1-th frame. When refreshing, the white image can update the black image row by row from top to bottom in the direction of the arrow shown in Figure 2 (2) until the entire screen is a white image. This is done by analogy to achieve the alternate display of black and white images by the transmitter. Therefore, the existing screen refresh principle will cause the previous frame and the next frame to coexist in the image displayed by the transmitter. Similarly, the images C1 and C2 that are alternately displayed in the related art will also coexist on the screen of the transmitter.

[0176] FIG3 is a schematic diagram showing an exemplary embodiment of the principle of capturing an image at a receiving end.

[0177] Referring to Figure 3, current electronic devices commonly use CMOS sensors, which utilize a rolling shutter exposure method to capture images. As shown in Figure 3, using an image sensor array with five rows and five columns as an example, the sensor can be exposed sequentially, column by column, from right to left, as indicated by the arrows. For example, while the first column is being exposed, the other columns are not exposed. After the first column is exposed, the second column begins to be exposed, and so on, until all five columns (here) are exposed, thereby capturing an image.

[0178] Then, during the exposure process of the camera sensor of the electronic device, if the subject moves, the movement will be recorded during the rolling exposure process.

[0179] For example, in the above-mentioned related art solution, when the transmitter plays the alternating images C1 and C2 on the screen at a frequency of 60 Hz, the mobile phone as the receiver shoots the alternating images on the screen of the transmitter at 60 FPS (Frames Per Second). Then, the screen of the transmitter is refreshed line by line and the camera sensor of the receiver is exposed line by line (or column by column), which may cause the fusion of images C1 and C2 in the video frame shot by the receiver. The color in the fusion area of ​​the two frames of the image is the color obtained by averaging the colors of the current frame and the previous frame in proportion (for example, the percentage of fusion is related to the distance, which is not limited here).

[0180] Please refer to Figures 4a and 5a. For example, the transmitter carries an encoded QR code in the original video. The original video is a dynamic video with constantly changing black and white grids. The QR code is encoded into the aforementioned images C1 and C2 using the scheme of the related art. For example, the black areas in the QR code are encoded as black, and the white areas in the QR code are encoded as white to obtain image C1, and the black areas in the QR code are encoded as white, and the white areas in the QR code are encoded as black to obtain image C2. In this way, when images C1 and C2 are synthesized and displayed in the original video, the human eye cannot distinguish the QR code from the display screen of the transmitter, presenting a dynamic video in which the colors of the black and white grids constantly change between black and white, similar to the effect of a dynamic grid grayscale change.

[0181] FIG5a(1) shows an example of a frame of a grid video with an encoded QR code displayed on a display screen of a transmitting end. i The receiving end can capture three frames of the grid video displayed on the display screen of the sending end, which are image R i , image R i+1 , image R i+2 , and decode the three frames of images to obtain the QR code.

[0182] In FIG4a , a single small grid at the same position in the grid video is taken as an example to illustrate the decoding process at the receiving end. The same applies to other small grids, which will not be described here.

[0183] For example, the content displayed at the q1 position shown in FIG5a(1) is a small grid in the encoded QR code. The small grid at the q1 position can be continuously refreshed and displayed between black and white according to the encoding scheme of the relevant technology.

[0184] As shown in FIG4a(1), it shows the images F displayed on the screen in sequence at the sending end. i 、Image F i+1 、Image F i+2 The three frames of images of the small grid at the position q1 shown in Figure 5a(1) are image f i 、Image f i+1 、Image f i+2 .

[0185] 4a, B represents black (RGB value is 0), G represents gray (RGB value is 127), and W represents white (RGB value is 255).

[0186] Among them, the image f i The color of B is black, and the image f i+1 The color is white, and the image f i+2 The color of is black, and so on, the image f i+3 The color is white, which will not be described here.

[0187] Combining the screen refresh principle of the transmitter and the image capture principle of the receiver as described in FIG2 and FIG3 , FIG4a(2) shows three frames of images (image R i , image R i+1 , image R i+2 ), the three frames of images of the small grid at position q1 shown in Figure 5a(1) are image r i 、Image r i+1 、Image r i+2 .

[0188] As shown in Figure 4a(2), the image r captured by the receiving end i There is also the current frame, here is the image f i (black area represented by B), and the previous frame, here is image f i-1 (white area represented by W), and there is also an image f between the black area and the white area i and image f i-1The fusion area is the gray area represented by G.

[0189] As shown in Figure 4a(2), the image r captured by the receiving end i There is also image f i+1 (white area represented by W) and image f i (black area represented by B), and there is an image f between the black area and the white area i and image f i+1 The fusion area is the gray area represented by G.

[0190] Similarly, as shown in Figure 4a(2), the image r captured by the receiving end i+1 There is also image f i+2 (black area represented by B) and image f i+1 (white area represented by W), and there is also an image f between the black area and the white area i+1 and image f i+2 The fusion area is the gray area represented by G.

[0191] Figure 4a(2) shows three frames of images obtained by the receiver from photographing the display screen of the transmitter at the position q1. In each frame of the image, there is a fusion area between the current frame and the previous frame displayed by the transmitter. The fusion area is between the black area and the white area and extends from the lower left corner to the upper right corner.

[0192] Next, as shown in FIG4a(3), in order to decode the small grid at the position q1 shown in FIG5a(1) to obtain the image of the QR code at the position q1, the receiving end can perform difference processing on the three frames of images at the position q1 shown in FIG4a(2) above. Specifically, the receiving end can perform difference processing on the image r shown in FIG4a(2). i and image r i+1 Perform the difference and then calculate the absolute value to obtain the image shown in Figure 4a(3) |r i -r i+1 |; and the receiving end can process the image r as shown in FIG4a(2) i and image r i+2 Perform the difference and then calculate the absolute value to obtain the image shown in Figure 4a(3) |r i -r i+2 |.

[0193] As shown in Figure 4a(3), the image |r i -r i+2 | is a black area represented by B (RGB value is 0), then the receiving end can i -r i+2|Can be binarized to 0.

[0194] As shown in Figure 4a(3), the image |r i -r i+1 |Although there are white areas (RGB value is 255), the image |r i -r i+1 If there is a black area represented by B (RGB value is 0) in the diagonal area of ​​|, the receiving end cannot convert the image corresponding to a single small grid |r i -r i+1 | quantization, resulting in the image |r i -r i+1 | It also cannot be binarized to 1. Thus, the encoding method of the graphic code using the related art will cause the decoding end to be unable to binarize some small grids corresponding to the QR code to 1, but to binarize them to 0, thus displaying them as white small grids in the QR code.

[0195] Returning to FIG5a, the process shown in FIG4a is the decoding process for the small grid at position q1 shown in FIG5a(1). If the small grid at position q1 is a black grid (the binarization result is 1) showing the hidden QR code, then because there is a black area in the result after differential decoding of the small grid shown in FIG4a(3), it can only be binarized to 0, resulting in the small grid at position q1 being unable to be decoded.

[0196] The receiving end can follow the principle of the process of FIG4a to display three frames of images in sequence on the display screen of the sending end: image F i 、Image F i+1 、Image F i+2 Shoot to get three frames of image R i , image R i+1 , image R i+2 As shown in Figure 5a(2), the receiving end can process the image R i , image R i+1 Perform the difference and then calculate the absolute value to get the image |R i -R i+1 |binarized image, where image|R i -R i+1 The image of the small grid at position q1 in | is the image shown in Figure 4a(3) |r i -r i+1 |binarized image; Similarly, as shown in Figure 5a(3), the receiving end can i , image R i+2 Perform the difference and then calculate the absolute value to get the image |R i -R i+2 |binarized image, where image|R i -Ri+2 The image of the small grid at position q1 in | is the image shown in Figure 4a(3) |r i -r i+2 |The binarized image.

[0197] From the decoding results of the receiving end shown in Figures 5a(2) and 5a(3), it can be seen that the image shown in Figure 5a(3) does not recover the QR code, and the image shown in Figure 5a(2) only decodes a partial QR code, and the lower right corner cannot be decoded.

[0198] Therefore, when employing the related art method of encoding a graphic code at the transmitter end and embedding the encoded graphic code within the original video or image, the graphic code is encoded as two colors that are alternately displayed within the original video or image. When the receiver captures the video or image displayed by the transmitter end, the captured target image may contain refresh stripes of the alternating two-color graphic code. If the area of ​​the captured target image displaying these refresh stripes falls within the display area where the graphic code is located, the portion of the graphic code within that display area cannot be decoded, resulting in the problem of incorrect decoding of the graphic code.

[0199] In response to the problem in the related art that when refresh stripes are present in an image captured by a receiving end, and the area where the refresh stripes are located is the display location of a hidden graphic code, the graphic code cannot be decoded. The present application provides an encoding method and a decoding method, as well as a transmitting end and a receiving end. The transmitting end can use the encoding method to encode the graphic code using three or more colors (or three or more brightness levels) to encode a multi-frame image (three or more frames). At least three of the multi-frame images have different colors and / or at least three of the multi-frame images have different brightness. For example, two of the multi-frame images have been encoded with brightness and color so that their brightness and color are different. The transmitting end can alternately display the multi-frame images. Optionally, the multi-frame images can serve as background elements or part of background elements in the original image or video, or can be superimposed on the upper layer of the original image or video, or a frame in the original video can be replaced with the multi-frame images, or the multi-frame images can be inserted before or after a frame in the original video, without limitation.

[0200] Optionally, the transmitting end may alternately display the multiple frames of images at a frequency at which the at least three frames of images cannot be perceived by human eyes.

[0201] Optionally, the multiple frames of images are alternately displayed at a frequency at which the at least three frames of images can be perceived by a sensor associated with the receiving end.

[0202] The graphic code may be a machine-readable code, a one-dimensional or two-dimensional information symbol, an Aztec code, a data matrix code, a QR code, a barcode, or the like.

[0203] Specifically, there may be a first area represented by bit "1" (also expressed as the first part) in the graphic code, and a second area represented by bit "0" (also expressed as the second part). When encoding, the sender may encode at least one of the first area and the second area using three or more colors (and / or brightness) to encode multiple frames of images (three or more frames of images).

[0204] For example, if the graphic code is a QR code, then the QR code contains black small grids represented by bit "1" (all black small grids in the QR code are referred to as black areas below), and white small grids represented by bit "0" (all white small grids in the QR code are referred to as white areas below). Then the transmitter can encode the black areas in the QR code into three brightness levels, while the white areas remain unchanged (or are encoded into one other brightness value), to encode three frames of images. Alternatively, the transmitter can encode the white areas in the QR code into three brightness levels, while the black areas remain unchanged (or are encoded into one other brightness value), to encode three frames of images. Alternatively, the sending end can encode the QR code to obtain three frames of images, wherein the black area of ​​the QR code is encoded into three brightness levels, for example, the brightness levels of the black areas corresponding to the three frames of images are a, b, and c, respectively, and the white area of ​​the QR code is also encoded into three brightness levels, for example, the brightness levels of the white areas corresponding to the three frames of images cannot be a, b, and c, that is, they cannot be exactly the same as a, b, and c, but can be a, c, b, or a, a, a, or c, a, b, etc., to ensure that when the two frames of images are differentiated, the difference results for the black area and the white area are different, so as to encode and obtain three frames of images, wherein the changes in the two brightness levels of the black area cannot be exactly the same as the changes in the two brightness levels of the white area, so as to avoid the receiving end being unable to distinguish between the black area and the white area based on the difference when taking the difference between the two frames of images.

[0205] Then, after the receiving end captures the display image of the transmitting end to obtain at least three frames of images (with different colors and / or brightness), since the graphic code is encoded as at least three frames of images, and at least one target area in the black area (an example of the first area represented by bit 1) and the white area (an example of the first area represented by bit 0) in the three frames of images has different brightness or color, then in the at least three frames of images captured by the receiving end, the pixel values ​​corresponding to the same pixel position (corresponding to the target area, such as the black area) are different (reflected in different colors and / or brightness). Then, after the receiving end performs a difference on the two frames of images captured, the RGB value of the target area after the difference is not 0, and it will not be the black area shown in Figure 4a (3). Even if the difference result of the area (such as the white area) other than the target area in the image after the difference (also expressed as the difference value) is 0, the difference result of the target area (such as the black area) is greater than 0. In this way, the white area and the black area in the image after the difference can be distinguished, and the receiving end can also binarize the image after the difference.

[0206] In the related art, a transmitting end alternately displays two frames of images. When a receiving end photographs the two frames of images alternately displayed by the transmitting end, the two photographed frames are two consecutive frames of images. The position, brightness and color of the refresh stripes in the two consecutive frames of images are basically unchanged, or very close. Then, after the receiving end takes the difference between the two consecutive frames of images, the position of the refresh stripes may exist in the black area shown in Figure 4a (3).

[0207] However, in the technical solution of the present application, the transmitting end alternately displays at least three frames of images. Therefore, when the receiving end captures the at least three frames of images alternately displayed by the transmitting end, at least three frames of images can be captured, rather than two consecutive frames of images. In this way, between the at least three frames of images captured by the receiving end (for example, between the first and third frames, or between the second and fourth frames, or between the second and fifth frames, etc., which are non-consecutive frames), one of the brightness and color of the refresh stripes can change. In this way, when the receiving end performs a difference (for example, a difference in the brightness domain or the color domain) on at least one group of two frames of the at least three frames of images captured, the difference result will not be 0, and there will be no black area in the area where the refresh stripes are located in the image after the difference. This allows quantization of the area where the refresh stripes are located, so that even if refresh stripes are present in the captured image, it does not affect the correct decoding of the graphic code by the receiving end.

[0208] In this way, the transmitting end loops and sequentially displays at least three frames of images to represent bit 1 (or bit 0) in the graphic code, thereby achieving hidden display of the graphic code without being perceived by the human eye. The at least three frames of images can be patterns of three colors or three brightness levels. The pattern can be a continuous block pattern (such as a QR code) or a discrete dot pattern (such as a ring code composed of discrete dots). This application does not impose any restrictions on the pattern in the graphic code.

[0209] If a refresh stripe region exists in the image captured by the receiving end, even if the refresh stripe is located at the display position of the graphic code, the position corresponding to the refresh stripe region in the difference result of the captured image captured by the receiving end will not be zero, so that the refresh stripe region will not be differencing into a black area, which can facilitate the receiving end to correctly decode the graphic code and obtain the target information carried in the graphic code. In this way, the present application can eliminate the influence of the refresh stripe (referring to the fusion area of ​​two frames of image) on the differential decoding of the hidden graphic code, and even if the graphic code is covered by the refresh stripe, it can still be correctly decoded.

[0210] The application scenario of this application can be a scanning scenario in which a light-emitting array screen displays a graphic code that is not perceptible to the human eye. For example, the human eye can see a dynamic image on the screen, but cannot tell that it is a graphic code. In other words, the encoded multi-frame images displayed by this application are not perceptible to the human eye when they are displayed alternately. However, when the encoded multi-frame images are displayed alternately, they are visible or perceptible to a sensor associated with a receiving end that is receiving information or is to be paired with a transmitting end.

[0211] Taking the graphic code as a QR code as an example, the application scenario of the present application can be that the sending end alternately displays at least three frames of images generated by encoding the graphic code for pairing the sending end and the receiving end, and the receiving end scans or takes a photo of the display content of the sending end's display screen to decode the graphic code, thereby extracting the target information in the graphic code.

[0212] The transmitting end can be any electronic device including a display screen, and the receiving end can be any electronic device including a camera. The types of electronic devices may include, but are not limited to, mobile phones, tablet computers, personal computers, laptops, wearable devices (smart watches, smart bracelets, smart glasses, etc.), in-vehicle devices (such as in-vehicle displays), and smart TVs.

[0213] For example, the application scenarios of the present application may specifically include: a mobile phone scanning the screen of another mobile phone, a mobile phone scanning the screen of a tablet computer, a mobile phone scanning the screen of a personal computer (PC), a mobile phone scanning the screen of a smart watch, a tablet computer scanning the screen of a mobile phone, a tablet computer scanning the screen of another tablet computer, a tablet computer scanning the screen of a PC, a tablet computer scanning the screen of a smart watch, etc.

[0214] The following describes the processing process of the sending end and the processing process of the receiving end of this application in conjunction with specific embodiments.

[0215] The following uses Examples 1 to 4 as examples to introduce the processing procedures of the sending end of this application.

[0216] Example 1

[0217] In this example 1, the sending end may encode the QR code into three frames of images and load the three frames of images into the original video to hide the QR code in the original video for display.

[0218] The original video can be any video; in addition, the original video can also be replaced by a static or dynamic image, which is not limited here.

[0219] In Example 1, the graphic code to be encoded is a QR code. The same method applies to other graphic codes (e.g., barcodes, circular codes, etc.), and is not described in detail here. The pattern inside the barcode is a vertical bar shape, and the pattern inside the circular code is a dot shape. The dots inside the circular code can be the same or different in size, and there is no limitation here.

[0220] In some embodiments, the pattern in the original video may be the same as the pattern in the graphic code, so that the graphic code can be hidden in the original video and is not easily visible to human eyes.

[0221] For example, in Example 1, the pattern in the original video is a grid pattern, which is the same as the pattern structure of the small grids in the QR code.

[0222] The resolution, number of grids, and position of each frame image in the original video in this Example 1 are the same as the resolution, number of coding blocks (also grid patterns), and position of the QR code to be encoded.

[0223] The original video in Example 1 is also expressed as a random checkerboard slow-motion video.

[0224] In Example 1, the QR code to be encoded is a QR code with a resolution of 21x21, each unit pattern is a small square (also expressed as a small grid), and the original video is a video with a resolution of 21x21, and its unit pattern is also a small grid of equal size (colors include black and white). The original video consists of multiple frames of images of the small grid undergoing dynamic changes.

[0225] The original video in this example 1 is a random chessboard slow-motion video, whose unit pattern and resolution are the same as the QR code to be encoded. Then the sender can use the original video to encode the QR code to obtain at least one frame of image (as at least one of the three frames of image).

[0226] In other embodiments, the unit pattern in the original video may not be a small grid, but a star pattern, a circular pattern, a random point, or a single point, etc., which is not limited here.

[0227] FIG6 a is a schematic diagram showing the processing process of the sending end in Example 1, and FIG7 a is a diagram showing images related to Example 1.

[0228] Referring to FIG. 6a , the process may include the following steps:

[0229] S101, the sending end generates a first black and white QR code QR0 based on target information.

[0230] Among them, the first two-dimensional code QR0 is also expressed as a black and white two-dimensional code QR0.

[0231] The target information is information that needs to be carried in the QR code, such as a link, account number, password, etc., which is not limited here.

[0232] For example, the sender may perform a QR code encoding on the text "Hello World" to obtain the black and white QR code shown in FIG7a(1), namely, the first QR code QR0.

[0233] Optionally, in step S102, the sending end generates a second two-dimensional code QR0 in reverse color based on the first two-dimensional code QR1. r

[0234] Among them, the sending end can generate a reverse color QR code QR based on the black and white QR code QR0 r (Also expressed as the second QR code QR r ), where the RGB value of the black grid in the black and white QR code is [0, 0, 0], and the RGB value of the white grid is [255, 255, 255].

[0235] Then the sender can modify the RGB values ​​(expressed as [R, G, B]) of the black area (all black grids) in the black and white QR code QR0 to [255, 255, 255], and modify the RGB values ​​of the white area (all white grids) in the black and white QR code QR0 to [0, 0, 0], thereby changing the black grids in the black and white QR code QR0 to white grids, and changing the white grids to black grids, so as to obtain the inverted QR code QR0 as shown in Figure 7a (2). r In this way, when encoding the first QR code, the encoding strategy originally used for the black area can be applied to the inverted QR code. r (or image QR r ) to achieve the interchange of encoding strategies for white areas and black areas.

[0236] Among them, the black small squares in the black and white QR code represent bit 1, and the white small squares represent bit 0. The white small squares in the reverse color QR code represent bit 1, and the black small squares represent bit 0.

[0237] In some embodiments, when the sender processes the first QR code to obtain the second QR code, it is not limited to inversion processing, but can also change the brightness (or color) of the black small grids and white small grids in the first QR code, for example, adjusting the RGB value of the black small grid in the first QR code to [20,20,20], and adjusting the RGB value of the white small grid in the first QR code to [200,200,200] to obtain the second QR code.

[0238] S201, the sending end extracts a frame of image from the original video, for example, extracts the i-th frame of image, with O i express.

[0239] FIG7a(3) shows the image O extracted from the original video (here is a random chessboard slow motion video). i .

[0240] The present application does not limit the execution order between S201 and S101, and the two can be executed in parallel or serially.

[0241] After S201 and S102 , the sending end may execute S103 .

[0242] In this embodiment, the brightness of the black and white two-dimensional code can be encoded to obtain three frames of images with different brightness, namely, image N i , image R i 、Image O' i The encoding process may include S103 and S202.

[0243] S103, the sending end sends the second QR code QR r With the i-th frame image O i Fusion generates two frames of images, namely image N i and image R i .

[0244] After S201, optionally, in S202, the sending end may perform a i Perform gain adjustment to obtain image O' i .

[0245] Referring to Table 1, the transmitter can use Formula 1 to calculate the image QR r The white part in the image is encoded, and the QR r The encoding result (RGB value) of the white part in is a shown in Table 1, so as to obtain the image N shown in Figure 7a (4)i ;

[0246] And use formula 2 to calculate image QR r The white part in is encoded, and the encoding is b shown in Table 1 to obtain the image R shown in Figure 7a (4) i ;

[0247] And the image O is obtained by formula 3 i The white part in the image is encoded, and the encoding is obtained as shown in Table 1, and the image O' shown in Figure 7a (4) is obtained. i .

[0248] Among them, the sending end obtains the image N after encoding i , image R i And the image O' i When the encoding method of the black part in the corresponding image is not limited, for example, the black part can be encoded in the above three frames of images (image N i , image R i And the image O' i ) are uniformly encoded as the same brightness d, or encoded as image O i Corresponding image QR r The pixel values ​​of the black part in the image are not limited here. Table 1 shows a possible image N i , image R i And the image O' i .

[0249] Table 1

[0250] a=O i +g1*QR r , formula 1;

[0251] b=O i -g2*QR r , formula 2;

[0252] c=g3*O i , formula 3;

[0253] Among them, g1 and g2 in formula 1 and formula 2 are for image QR r The gain value when the RGB value of the white area (each white pixel) in the image is adjusted, and g3 in formula 3 is the gain value of the image O i The gain value when adjusting the gain is the RGB value of the white area in the image.

[0254] Among them, through formula 1, the image QR r When adjusting the gain of the R, G, and B channels, the same gain value (i.e. g1) is used to adjust the image QRr The brightness (or grayscale) of the image QR can also be calculated by formula 2. r The same gain (here is g2) is adjusted for the R, G, and B channels to change the image QR r The brightness of the image O can also be calculated by formula 3. i The same gain (here is g3) is adjusted for the R, G, and B channels to change the image O i brightness.

[0255] In some embodiments, when the sending end encodes the black and white QR code QR0 to obtain three frames of images, it can also encode the image QR r The three RGB channels are adjusted with different gains, thereby obtaining three frames of images of different colors of the black part and / or white part of the black and white QR code QR0, image N i , image R i 、Image O' i .

[0256] Among them, g1, g2, and g3 are preset gain values, which can make the fused image N i , image R i 、Image O' i There are obvious differences in the differential results of any two frames of images, so that the receiving end can decode the above black and white two-dimensional code QR0 based on the differential results.

[0257] Among them, image N i , image R i 、Image O' i These are three frames of images with different brightness (or grayscale) obtained by encoding the black and white two-dimensional code QR0.

[0258] This embodiment encodes the black and white two-dimensional code QR0 to obtain three frames of images: Image N i , image R i 、Image O' i , where the three frames of images correspond to the inverted QR code QR r The white area of ​​the image is encoded into three brightness levels through gain adjustment, such as a, b, and c in Table 1 and Formulas 1 to 3; optionally, the corresponding inverted QR code in the three frames of image is r The encoded brightness of the black area is d.

[0259] For example:

[0260] Example 1: For example, image O i The grayscale value (i.e. brightness value) is 127, g1=g2=0.5, g3=1.

[0261] Among them, image Oi Since it is a black and white image, the grayscale values ​​of the three channels R, G, and B are the same, which is the mean of the values ​​of the three channels R, G, and B. i The gray value of image O i The mean of the RGB values ​​of all pixels.

[0262] And the image QR r The RGB value of the white area is [255,255,255], and the RGB value of the black area is [0,0,0]. Therefore, the grayscale value of each white pixel is 255, and the grayscale value of each black pixel is 0.

[0263] Then according to formula 1, we can calculate a=127+0.5*255=255;

[0264] Then according to formula 2, we can calculate b = 127 - 0.5 * 255 = 0;

[0265] Then according to formula 3, we can calculate c = 1 * 127 = 127;

[0266] Then the sender can use the black area in the black and white QR code QR0 (corresponding to the inverted QR code QR r The three frames of images obtained by encoding (white area in the image): Image N i , image R i 、Image O' i Among them, image N i Corresponding to the reverse color QR code r The grayscale value of the pixel in the white area is a=255, for example, the RGB value is [255,255,255], and the image R i Corresponding to the reverse color QR code r The grayscale value of the pixel in the white area is b=0, for example, the RGB value is [0,0,0], image O' i Corresponding to the reverse color QR code r The grayscale value of the pixel in the white area is c=127, for example, the RGB value is [127,127,127].

[0267] The above image N i , image R i 、Image O' i In the middle, it corresponds to the inverted QR code r The grayscale value of the pixel point in the black area is d, for example, d=127, for example, the RGB value is [127,127,127].

[0268] Similarly, Example 2: For example, image O iThe grayscale value (i.e. brightness value) is 127, g1=g2=0.5, g3=2, for example, d=127, then a=255, b=0, c=255. Then the image N i , image R i 、Image O' i Corresponding to the reverse color QR code r The RGB values ​​of the pixels in the white area of ​​image N are [255, 255, 255], [0, 0, 0], [255, 255, 255]. i , image R i 、Image O' i Corresponding to the reverse color QR code r The RGB values ​​of the pixels in the black area are all [127,127,127].

[0269] In some embodiments, g1 and g2 may also be different.

[0270] In this example 1, since the resolution, unit pattern, and number of patterns of the original video are the same as the corresponding parameters of the black and white QR code QR0, the sending end can send a frame of image O in the original video. i Perform gain adjustment to change the image O i The brightness of the image O' is obtained i , as a black and white QR code after adjusting the brightness, image O' i It can also be used as a frame image after adjusting the brightness of the black and white QR code QR0.

[0271] In other embodiments, the sending end may also directly send the image O i As a frame image after encoding the black and white two-dimensional code QR0, the three frames of images obtained can be: Image N i , image R i 、ImageO i .

[0272] After S202 and S103 , the sending end may execute S104 .

[0273] S104, the sending end can send the i-th frame image O in the original video i Replace with image N i , image R i 、Image O' i , and in the original video, the three frames of images are played sequentially and looped n times.

[0274] Wherein, n is a positive integer.

[0275] For example, if the frame rate of the original video is 60 FPS and n=1, then one frame of the original video (frame i) is replaced by three frames of images (frame Ni , image R i 、Image O' i ), so that the visual frame rate of the processed video is 20FPS, which is similar to a frame image (i-th frame) of the original video being played three times, similar to the effect of slow motion.

[0276] When n is greater than 1, for example, n=2, the sending end can send image N i , image R i 、Image O' i After playing the sequence once, follow the image N i , image R i 、Image O' i The three frames of images are played once in a loop in the order shown in FIG.

[0277] In some embodiments, the sending end plays the image N in a loop. i , image R i 、Image O' i The order in which they are played can also be different each time through the loop.

[0278] S105, the sending end loops through S103, S202 and S104 based on the next frame image in the original video to update the three frames of image encoded with the black and white QR code QR0 in combination with the next frame image, and repeats this loop until the last frame image of the original video.

[0279] Table 1 above gives a possible three-frame image (image N i , image R i And the image O' i ) corresponds to the image QR r The encoding results of the white and black parts in . Among them, corresponding to the image QR r The white part in the image can be played in three brightness levels: a, b, and c in chronological order, corresponding to the image QR r The black part in the video can be played synchronously in chronological order with a brightness of d, d, and d.

[0280] In some embodiments, the average of the maximum and minimum values ​​of a, b, and c in Table 1 and Formulas 1 to 3 is equal to the middle value of a, b, and c (i.e., a value that is neither the maximum nor the minimum). In this way, the image N played by the transmitter can be i , image R i 、Image O' i Between, about image QR r The brightness difference of the white part in the image is small, making the sequentially played image N i , image R i 、Image O' iThe brightness change of the above-mentioned white part is relatively smooth and not very obvious, making it difficult for users to identify the hidden QR code from the played video.

[0281] In some embodiments, when a, b, and c in Table 1 above each represent the same brightness but different colors, then similarly, one color among a, b, and c is the average of the other two colors, so that the image N played by the transmitter is i , image R i 、Image O' i The color changes of the white part are relatively close, for example, from red to orange, and then from orange to yellow, and there will be no sudden color change, for example, from red to black.

[0282] In some embodiments, when a, b, and c in Table 1 above each represent the same brightness but different colors, the average value of the colors of a, b, and c may be close to the color of the corresponding i-th frame in the original video (for example, the difference in RGB values ​​is less than a threshold value). In this way, the encoded QR code image with color changes (i.e., image N i , image R i 、Image O' i ) is not easily visible to the naked eye when displayed in the original video, thus avoiding exposing the hidden QR code. For example, if the color of the image in frame i is yellow, a can represent red, b can represent green, and c can represent yellow. In this way, the red and green frames alternately flash to yellow, and the three frames alternately display yellow, which is the same color as the superimposed or fused image in frame i, making the QR code image difficult for users to detect.

[0283] In some embodiments, when three frames of images are generated after encoding a black and white QR code, it is not limited to the method of fusing the black and white QR code with the i-th frame image in the original video. It is also possible to only adjust the gain of the black part and / or the white part of the black and white QR code to obtain three frames of images, wherein the three images have three brightness levels with respect to the above-mentioned black part and / or three brightness levels with respect to the white part.

[0284] In some embodiments, the transmitting end is not limited to fusing the black and white QR code with the i-th frame image in the original video to obtain the above-mentioned three encoded frames. The generated three encoded frames can also be used as the upper layer and the corresponding three frames in the original video as the lower layer to synthesize the three encoded frames (image N) in a layer-by-layer manner. i , image R i 、Image O' i ) is displayed above the original video layer, so that the frame rate of the original video does not change.

[0285] By using the scheme of Example 1 at the sending end, the brightness (or color) of the black grid in the black and white QR code is encoded into three frames of images (image N) with different brightness (or color) by reasonably setting the gain value and combining the video frames of the original video. i , image R i 、Image O' i ), it is possible to add a frame of image on the basis of the double-frame differential video, so that the color or brightness within the grid of the three-frame image is controlled by the video frame of the original video and the black and white QR code and gain parameters; when the encoded three-frame image is displayed in the original video, the processed video still looks like the original video to the naked eye (for example, the processed random checkerboard slow-motion video still looks like a random checkerboard slow-motion video), for example, the video effect is a black and white grid similar to a chessboard that changes in brightness or color to present a flickering effect. However, after the processed video displayed by the sending end is photographed by the receiving end, since the above three frames of images have different colors or brightness with respect to the black area in the black and white QR code, there is no completely identical pattern (same brightness and color) in the same position between the multiple frames of images photographed by the receiving end. Then, even if the QR code is covered by the refresh stripes of any two of the three frames of images (referring to the fusion area of ​​the two frames of images), the receiving end can also distinguish the black area and the white area in the QR code by differential, so that the QR code can be correctly decoded and the target information in the black and white QR code can be extracted.

[0286] In some embodiments, the black and white two-dimensional code in Example 1 may also be a circular code.

[0287] Figure 7b illustrates an exemplary ring code of the present application. The two dashed circles in the ring code shown in Figure 7b do not exist in the ring code itself. The arrangement of the black dots is shown here simply to illustrate that the two circles shown in dashed lines constitute a ring code. As shown in Figure 7b, the black circles can be positioning patterns (with the same function as the positioning pattern of a QR code). The squares between the black circles represent bit 1s, carrying the target information. The remaining area within the ring formed by the two dashed circles, without showing any squares, represents bit 0s.

[0288] When the transmitting end encodes the circular code shown in FIG7b, the gain can be adjusted to encode the square area representing bit 1 in the circular code (all the squares shown in FIG7b) into three colors respectively, so as to obtain three frames of images after encoding the circular code. The colors of the squares representing bit 1 are different among the three frames of images (for example, image 1, image 2, and image 3).

[0289] Regarding the pattern representing bit 0 in the circular code shown in FIG7b (ie, other unshown squares located between the circular positioning patterns), the transmitting end does not process it, so that the pattern representing bit 0 remains unchanged in the above three frames of images.

[0290] Optionally, the average color of the three frames of images obtained by encoding the circular code shown in Figure 7b at the sending end can be close to the color of the pattern (such as dots or squares) of the video frames in the original video, so that it is difficult for the naked eye to detect the circular code from the video carrying the circular code.

[0291] The sending end can superimpose the three frames of images on the upper layer of the three frames of the original video in the order of image 1, image 2, and image 3 in a layered manner without changing the frame rate of the original video. The processed video can superimpose the above three frames of images on the basis of the original video and play the three frames of images in the above order and in a loop.

[0292] In some embodiments, the original video may be a dynamically changing video similar to a ring, and the original video may include dots or squares that change color. The sizes of the dots and squares in the ring code may be related to the sizes of the dots or squares in the original video (e.g., the same as or smaller than the sizes of the corresponding patterns in the original video), so that the ring code cannot be recognized by the naked eye in the played video.

[0293] In some embodiments, when the three frames of images are superimposed on the original video, the patterns in the three frames of images (such as the black dots and squares shown in Figure 7b) can maintain their positions unchanged, or can move clockwise or counterclockwise along the direction of the dotted circle. However, during the movement, the relative positions of the various patterns in the circular code remain unchanged.

[0294] Example 2

[0295] The implementation process of Example 2 is mostly the same as that of Example 1. The difference is that in Example 2, when encoding the black and white QR code, not only the area representing bit 1 (corresponding to the black area in the black and white QR code and the white area in the inverted QR code) can be encoded with different brightness, but also the area representing bit 0 (corresponding to the white area in the black and white QR code and the black area in the inverted QR code) can be encoded with different brightness; In addition, the encoded image can be more than three frames. In this Example 2, 6 frames of images can be encoded, respectively with A i 、B i 、C i 、D i 、E i , G i express.

[0296] FIG6 b is a schematic diagram showing the processing procedure of the sending end in Example 2.

[0297] Referring to FIG. 6b , the process may include the following steps:

[0298] S301: The sending end generates a first black and white QR code QR0 based on target information.

[0299] The implementation principle of S301 is the same as the implementation principle of S101 shown in FIG6 b in Example 1, and will not be repeated here.

[0300] S201, the sending end extracts a frame of image from the original video, for example, extracts the i-th frame of image, with O i express.

[0301] The implementation principle of S201 is the same as the implementation principle of S201 shown in FIG6 b in Example 1, and will not be repeated here.

[0302] After S201 and S301, in S303, the sending end combines the first QR code QR0 with the i-th frame image O i Fusion generates six frames of images, namely image A i Image B i 、Image C i , Image D i 、Image E i , image G i .

[0303] Table 2 is an exemplary diagram of a possible encoding result of the six frames of images.

[0304] Table 2

[0305] The "white portion" in Table 2 represents all white cells in the black-and-white QR code QR0 to be encoded, that is, all cells represented by bit 0; the "black portion" in Table 2 represents all black cells in the black-and-white QR code QR0 to be encoded, that is, all cells represented by bit 1. In Table 2, the values ​​of the white portion in the six frames (e.g., a, b, c) represent the brightness values ​​of the white portion in the corresponding six frames (or RGB values, where the values ​​of the three RGB channels are the same, which is the brightness value); similarly, the values ​​of the white portion in the six frames (e.g., d, e, f) represent the brightness values ​​of the black portion in the corresponding six frames (or RGB values, where the values ​​of the three RGB channels are the same, which is the brightness value).

[0306] Among them, in the first two-dimensional code QR0, the RGB value of the black area is 0, which is not convenient for gain processing. Therefore, when the sending end executes S303 (the same applies to S103 in the above example 1), the RGB value of the black area in the first two-dimensional code QR0 can be adjusted to a non-zero value QR b , where 1≤QR b≤255. In Example 2 to Example 4 below, the sender can adjust the RGB value of the black area in the first QR code QR0 to a custom value QR b , here it is 255, so as to encode the black area.

[0307] The following describes the image A in Table 2 in combination with methods 1.1) to 1.6). i Image B i 、Image C i , Image D i 、Image E i , image G i How to generate:

[0308] Method 1.1), for image A i The generation method is: the output signal of the white part is a, and the output signal of the black part is d;

[0309] Where a=O i +ga*QR0,d=O i +gd*QR b , ga, gd are generated images A i The corresponding gain of the white part and the black part. Its operation principle is similar to that of formula 1 and formula 2 in the above example 1.

[0310] Among them, a is the position of the white pixel in the black and white QR code QR0. i d is the position of the black pixel in the black and white QR code QR0. i The pixel values ​​in .

[0311] Among them, image O i It can be an image other than a completely black image or a completely white image, that is, image O i The RGB values ​​of all pixels cannot be all [0,0,0], nor all [255,255,255].

[0312] Similarly, in method 1.2), for image B i The generation method is: the output signal of the white part is b, the output signal of the black part is d, where b=0 i +gb*QR0、d=O i +gd*QR b , gb, gd are generated images B i The gain corresponding to the white part and the black part.

[0313] Method 1.3), for image C i The generation method is: the output signal of the white part is c, the output signal of the black part is e, where c=0i +gc*QR0,e=O i +ge*QR b , gc and ge are the gains corresponding to the white and black parts when generating the Ci frame.

[0314] Method 1.4), for image D i The generation method is: the output signal of the white part is a, the output signal of the black part is e, where a=0 i +ga*QR0,e=O i +ge*QR b , ga, ge are generated images D i The gain corresponding to the white and black parts of the frame.

[0315] Method 1.5), for image E i The generation method is: the output signal of the white part is b, the output signal of the black part is f, where b=0 i +gb*QR0, f=O i +gf*QR b , gb, gf are generated images E i The gain corresponding to the white part and the black part.

[0316] Method 1.6), for image G i The generation method is: the output signal of the white part is c, the output signal of the black part is f, where c=0 i +gc*QR0, f=O i +gf*QR b , gc, gf are generated images G i The gain corresponding to the white part and the black part.

[0317] This embodiment takes brightness domain encoding of an image as an example, so that the brightness of the white area and the black area of ​​the six frames of images generated above vary. Other embodiments may also perform color encoding.

[0318] In combination with the above-mentioned methods 1.1) to 1.6), an example of encoding in the luminance domain is given using the method in Table 2.

[0319] Among them, the grayscale value (i.e., brightness value) of the white area (also expressed as the white part) in the black and white QR code QR0 to be encoded is the average of the RGB values ​​of all pixels, which is 255 here; similarly, the grayscale value of the black area (also expressed as the black part) in the black and white QR code QR0 after adjustment is the above QR b , here is 255; for example, image O i The grayscale value is 127.

[0320] Here is an encoding example:

[0321] Assuming ga = 0.5, gb = 0, gc = -0.5, gd = 0.5, ge = 0, gf = -0.5, the transmitter can calculate the following according to the formulas in the above methods 1.1) to 1.6): a = 0 i +ga*QR0=255,b=0 i +gb*QR0=127,c=0 i +gc*QR0=0,d=O i +gd*QR b =255, e=0 i +ge*QR b =127, f=0 i +gf*QR b = 0. Combined with Table 2, Table 3 shows the specific grayscale values ​​corresponding to the white part and the black part in the 6 frames of images after encoding in this application.

[0322] Table 3

[0323] It should be understood that the gain values ​​of ga, gb, gc, gd, ge, and gf are independent of each other.

[0324] In some embodiments, when the gain of the white area in the black and white two-dimensional code is adjusted, the average value of the three gains ga, gb, and gc is 0. Referring to Table 3, this can make the image A i Image B i 、Image C i The brightness change of the white area is uniform, and the brightness change is 127, and the image D i 、Image E i , image G i The brightness changes in the white area are uniform, all 127.

[0325] In addition, when the transmitter adjusts the gain of the black area in the black and white QR code, the average of the three gains gd, ge, and gf can be 0. Referring to Table 3, in this way, image A i Image B i 、Image C i , Image D i 、Image E i , image G i The brightness change of the black area between each group of two frames is uniform. For example, a group of images is image A. i Image B i , the other set of images is image C i , Image D iThe brightness change between these two sets of images is 128; one set of images is image C i , Image D i , the other set of images is image E i , image G i , the brightness change between the two sets of images is 127; the brightness value 128 and the brightness value 127 are close to achieve uniform brightness change in the black area.

[0326] In addition, in the six frames of images, the brightness changes of the black area and the white area are also the same, all close to 127.

[0327] Of course, referring to Table 3, the brightness change of the white part between two adjacent frames of images is not necessarily the same 127, and there can be differences. It only needs that the brightness difference of the white part between two adjacent frames of images in the six frames is not too large. For example, in the above Table 3, image B i The brightness value of the corresponding white part can also be 125. The encoding rules for the corresponding black areas in the above six frames of images are similar, and are not limited to the brightness difference being 127.

[0328] In some embodiments, similar to Table 2 and related descriptions, the six frames of images generated by the transmitting end may also be the encoding results shown in the following Tables 4 to 9a.

[0329] Table 4 is a possible encoding result of the six frames of images obtained by encoding in FIG. 6 b .

[0330] Table 4

[0331] Table 5 is a possible encoding result of the six frames of images obtained by encoding in FIG. 6 b .

[0332] Table 5

[0333] Table 6 is a possible encoding result of the six frames of images obtained by encoding in FIG. 6b .

[0334] Table 6

[0335] Table 7 is a possible encoding result of the six frames of images obtained by encoding in FIG. 6 b .

[0336] Table 7

[0337] Table 8 is a possible encoding result of the six frames of images obtained by encoding in FIG. 6b .

[0338] Table 8

[0339] Table 9a is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0340] Table 9a

[0341] Regarding the calculation method of a to e in the above Tables 4 to 9a, please refer to the introduction of the corresponding formulas in the above Methods 1.1) to 1.6), which will not be repeated here.

[0342] It should be understood that the above Tables 2 to 9a only exemplify some of the encoding results in this Example 2. By adopting the encoding scheme of Figure 6b in this Example 2, more encoding results can be generated.

[0343] In this Example 2, although 6 frames of images are obtained through encoding, in other embodiments, any number of frames greater than 3 frames of images may be obtained.

[0344] After S202 and S103 , the sending end may execute S104 .

[0345] S304, the sending end can send the i-th frame image O in the original video i Replace with image A i Image B i 、Image C i , Image D i 、Image E i , image G i , and in the original video, the six frames of images are played sequentially and cyclically n times.

[0346] Wherein, n is a positive integer.

[0347] The implementation principle of S304 is the same as the implementation principle of S104 shown in FIG6 b in Example 1, and will not be repeated here.

[0348] S305, the sending end loops through S303 and S304 based on the next frame image in the original video to update the six frames of image encoded with the black and white QR code QR0 in combination with the next frame image, and repeats this loop until the last frame image of the original video.

[0349] Different from Example 1 above, in this Example 2, the black part representing bit 1 in the black and white QR code can be encoded using at least two brightnesses, and the white part representing bit 0 in the black and white QR code can be encoded using at least three brightnesses to obtain 6 frames of encoded images. In these six frames of images, there are three brightnesses (for example, a, b, and c) for the brightness values ​​corresponding to the encoded white parts, and at least two brightnesses (for example, at least two of d, e, and f) for the brightness values ​​corresponding to the encoded black parts. In this way, between the six frames of images obtained by encoding the black and white QR code at the transmitting end, there can be not only brightness changes (at least three brightnesses) for the white areas, but also brightness changes for the black areas. This can also solve the technical problems existing in the prior art.

[0350] In addition, although the results of encoding the white area of ​​the black and white QR code in Tables 3 to 9a above are cycled in the order of a, b, and c in the six frames, in other embodiments, the encoding results of the white area of ​​two adjacent frames of the output six frames may be the same, for example, in image A i Image B i 、Image C i , Image D i 、Image E i , image G i The encoding results for the white area are a, a, b, b, c, c, or any permutation and combination of a, b, c, c, a, b.

[0351] In other words, the present application does not impose any restrictions on the order in which the various brightness levels after encoding the white area or black area of ​​the black and white two-dimensional code are arranged in the six-frame image.

[0352] In some embodiments, the encoding method of the white area or the black area in the black and white two-dimensional code at the transmitting end can also be expanded.

[0353] For example, in Example 2, the encoding method of the white area in the black and white two-dimensional code at the transmitting end is a cycle of abcabc as shown in Tables 3 to 9a. In other embodiments, when the image generated by the encoding is more than 6 frames, a string of constant sequences can be interspersed in the cycle process of the above encoding results. For example, the encoding method for the white area (such as abcabc) can be expanded to any one of aaabcaaabc, abbbbcabbbbc, abccccabcccc, acbcacbc, etc. The expansion principle of the encoding method for the black area is similar. Among them, the underlined ones in the encoding method of the above example are the newly added constant sequences. At the receiving end, a decoding strategy corresponding to the encoding method is adopted to perform differential processing on the captured video frames with corresponding frame numbers to decode and obtain the black and white two-dimensional code. That is, after the encoding method is adjusted, the decoding method also needs to be adjusted.

[0354] For example, Table 9b shows an example of encoding the white area in the black and white two-dimensional code according to the encoding method acbcacbc to obtain 8 frames of images (image A i Image B i 、Image C i , Image D i 、Image E i , image G i 、Image H i , image K i ), for example, encoding the black area of ​​a black-and-white QR code as d. The calculations for a, b, c, and d can be found in Methods 1.1) through 1.6) above and will not be repeated here.

[0355] Table 9b

[0356] Example 3

[0357] The implementation process of Example 3 is mostly the same as that of Example 2. Example 3 also uses the process shown in Figure 6b to encode the black and white QR code to generate 6 frames of images, respectively. i 、B i 、C i 、D i 、E i , G i express.

[0358] A possible method for generating six frames of images generated in S303 shown in FIG6 b is described below.

[0359] In a possible implementation, Table 10 shows the six frames of images (Image A i Image B i 、Image Ci , Image D i 、Image E i , image G i ) is a possible encoding result.

[0360] Table 10

[0361] The definitions and explanations of “white part” and “black part” in Table 10 are the same as those in Table 2 above and will not be repeated here.

[0362] The following describes the image A in Table 10 in combination with methods 2.1) to 2.6). i Image B i 、Image C i , Image D i 、Image E i , image G i How to generate:

[0363] Method 2.1), for image A i The generation method is: the output signal of the white part is a, and the output signal of the black part is d;

[0364] Where a=O i +ga*QR0,d=O i +gd*QR b , ga, gd are generated images A i The corresponding gain of the white part and the black part. Its operation principle is similar to that of formula 1 and formula 2 in the above example 1.

[0365] Method 2.2), for image B i The generation method is: the output signal of the white part is b, the output signal of the black part is e, where b=0 i +gb*QR0、e=O i +ge*QR b , gb, ge are generated images B i The gain corresponding to the white part and the black part.

[0366] Method 2.3), for image C i The generation method is: the output signal of the white part is c, the output signal of the black part is d, where c=0 i +gc*QR0、d=O i +gd*QR b , gc and gd are the gains corresponding to the white and black parts when generating the Ci frame.

[0367] Method 2.4), for image D iThe generation method is: the output signal of the white part is a, the output signal of the black part is e, where a=0 i +ga*QR0,e=O i +ge*QR b , ga, ge are generated images D i The gain corresponding to the white and black parts of the frame.

[0368] Method 2.5), for image E i The generation method is: the output signal of the white part is b, the output signal of the black part is d, where b=0 i +gb*QR0、d=O i +gd*QRB, gb, gd are the generated image E i The gain corresponding to the white part and the black part.

[0369] Method 2.6), for image G i The generation method is: the output signal of the white part is c, the output signal of the black part is e, where c=0 i +gc*QR0,e=O i +ge*QR b , gc, ge are generated images G i The gain corresponding to the white part and the black part.

[0370] In combination with the above-mentioned methods 2.1) to 2.6), Table 10 is used to provide an example of encoding in the luminance domain.

[0371] The grayscale value (i.e., brightness value) of the white area (also referred to as the white part) in the black and white two-dimensional code QR0 to be encoded is the average of the RGB values ​​of all pixels, which is 255 here. Similarly, the grayscale value of the black area (also referred to as the black part) in the black and white two-dimensional code QR0 is 0. For example, in image O i The grayscale value is 127.

[0372] Here is an encoding example:

[0373] Assuming ga = 0.5, gb = 0, gc = -0.5, gd = 0.5, ge = -0.5, the transmitter can calculate according to the formulas in the above methods 2.1) to 2.6) to obtain: a = 0 i +ga*QR0=255,b=0 i +gb*QR0=127,c=0 i +gc*QR0=0,d=O i +gd*QR b =255, e=0 i +ge*QR b= 0. Combined with Table 10, Table 11 shows the specific grayscale values ​​corresponding to the white part and the black part in the 6 frames of images after encoding in this application.

[0374] Table 11

[0375] In some embodiments, similar to Table 10 and related descriptions, the six frames of images generated by the transmitting end may also be the encoding results shown in the following Table 12.

[0376] Table 12 is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0377] Table 12

[0378] In Example 2 above, the brightness of the black area of ​​the black-and-white QR code changed every two frames in the six encoded images. In Example 3, the brightness of the black area changed once in each frame, and only two brightness levels, d and e, were encoded for the black area of ​​the black-and-white QR code. With this solution, the video still looks like the original (for example, a processed random checkerboard slow-motion video still looks like a random checkerboard slow-motion video), but the pattern captured by the receiving camera can be correctly decoded.

[0379] Example 4

[0380] The implementation process of Example 4 is mostly the same as that of Example 2. Example 4 also uses the process shown in Figure 6b to encode the black and white QR code to generate 6 frames of images, respectively. i 、B i 、C i 、D i 、E i , G i express.

[0381] A possible method for generating six frames of images generated in S303 shown in FIG6 b is described below.

[0382] In a possible implementation, Table 13 shows the six frames of images (Image A i Image B i 、Image C i , Image D i 、Image E i , image G i ) is a possible encoding result.

[0383] Table 13

[0384] The definitions and explanations of “white part” and “black part” in Table 13 are the same as those in Table 2 above and will not be repeated here.

[0385] The following describes the image A in Table 13 in combination with methods 3.1) to 3.6). i Image B i 、Image C i , Image D i 、Image E i , image G i How to generate:

[0386] Method 3.1), for image A i The generation method is: the output signal of the white part is a, and the output signal of the black part is d;

[0387] Where a=O i +ga*QR0,d=O i +gdQR b *, ga, gd are generated images A i The corresponding gain of the white part and the black part. Its operation principle is similar to that of formula 1 and formula 2 in the above example 1.

[0388] Method 3.2), for image B i The generation method is: the output signal of the white part is b, the output signal of the black part is d, where b=0 i +gb*QR0, gb, gd are generated images B i The gain corresponding to the white part and the black part.

[0389] Method 3.3), for image C i The generation method is: the output signal of the white part is c, the output signal of the black part is d, where c=0 i +gc*QR0、d=O i +gd*QR b , gc and gd are the gains corresponding to the white and black parts when generating the Ci frame.

[0390] Method 3.4), for image D i The generation method is: the output signal of the white part is a, the output signal of the black part is e, where a=0 i +ga*QR0,e=O i +ge*QR b , ga, ge are generated images D i The gain corresponding to the white and black parts of the frame.

[0391] Method 3.5), for image Ei The generation method is: the output signal of the white part is b, the output signal of the black part is e, where b=0 i +gb*QR0、e=O i +ge*QR b , gb, ge are generated images E i The gain corresponding to the white part and the black part.

[0392] Method 3.6), for image G i The generation method is: the output signal of the white part is c, the output signal of the black part is e, where c=0 i +gc*QR0,e=O i +ge*QR b , gc, ge are generated images G i The gain corresponding to the white part and the black part.

[0393] In combination with the above-mentioned methods 3.1) to 3.6), Table 13 is used to provide an example of encoding in the luminance domain.

[0394] The grayscale value (i.e., brightness value) of the white area (also referred to as the white part) in the black and white QR code QR0 to be encoded is the average of the RGB values ​​of all pixels, which is 255 here. Similarly, the grayscale value of the black area (also referred to as the black part) in the black and white QR code QR0 after adjustment is QR b , here is 255; for example, image O i The grayscale value is 127.

[0395] Here is an encoding example:

[0396] Assuming ga = 0.5, gb = 0, gc = -0.5, gd = 0.5, ge = -0.5, the transmitter can calculate the following according to the formulas in the above methods 3.1) to 3.6): a = 0 i +ga*QR0=255,b=0 i +gb*QR0=127,c=0 i +gc*QR0=0,d=O i +gd*QR b =255, e=0 i +ge*QR b = 0. Combined with Table 13, Table 14 shows the specific grayscale values ​​corresponding to the white part and the black part in the 6 frames of images after encoding in this application.

[0397] Table 14

[0398] It should be understood that the gain values ​​of ga, gb, gc, gd, and ge are independent of each other.

[0399] In some embodiments, when the gain of the white area in the black and white two-dimensional code is adjusted, the average value of the three gains ga, gb, and gc is 0, so that the image A i Image B i 、Image C i The brightness change of the white area is uniform, and the image D i 、Image E i 、 Image G i The brightness change of the white area is uniform; in addition, when the transmitter adjusts the gain of the black area in the black and white QR code, the average value of the three gains gd and ge can be 0, so that image A i Image B i 、Image C i , Image D i 、Image E i , image G i The brightness change of the black area between every three frames is uniform.

[0400] In some embodiments, similar to Table 13 and related descriptions, the six frames of images generated by the transmitting end can also be any encoding result shown in the following Tables 15 to 19.

[0401] Table 15 is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0402] Table 15

[0403] Table 16 is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0404] Table 16

[0405] Table 17 is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0406] Table 17

[0407] Table 18 is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0408] Table 18

[0409] Table 19 is a possible encoding result of the six frames of images obtained by encoding in FIG6b.

[0410] Table 19

[0411] Regarding the calculation methods of a to e in the above Tables 13 to 19, please refer to the introduction of the corresponding formulas in the above Methods 1.1) to 1.6), which will not be repeated here.

[0412] It should be understood that the above Tables 13 to 19 only exemplify some of the encoding results in this Example 4. By adopting the encoding scheme of Figure 6b in this Example 4, more encoding results can be generated.

[0413] In this example 4, although 6 frames of images are obtained by encoding, in other embodiments, any number of frames greater than 3 frames of images may be obtained.

[0414] In Example 4 above, the black areas of the black-and-white QR code in the six encoded frames can be encoded as two brightness levels, d and e. With this solution, the video still looks like the original (for example, a processed random checkerboard slow-motion video still looks like a random checkerboard slow-motion video), but the pattern captured by the receiving end's camera can be correctly decoded.

[0415] In addition, although the results of encoding the black area of ​​the black and white QR code in Tables 13 to 19 above can be cycled in any arrangement and combination of d and e in the six-frame image, this application does not impose any restrictions; similarly, when the sending end encodes the black area in the black and white QR code as 3 or more brightness levels, in the generated six-frame image (not limited to six frames, as long as there are three or more frames), the 3 or more brightness levels can also be freely combined and arranged in any way in the six-frame image, so that the six-frame image can have 3 or more brightness changes in the black area of ​​the black and white QR code.

[0416] It should be understood that the encoding scheme of the transmitting end of the present application is not limited to Examples 1 to 4. The processing process of the transmitting end of the present application can also be applied to the processing of other graphic codes, and the encoding method can also encode at least one type of area in the color domain representing bit 1 and another type of area representing bit 0 in the graphic code into 3 or more colors. The method is the same and will not be repeated here.

[0417] The following describes the processing process of the sending end and the processing process of the receiving end of this application in conjunction with specific embodiments.

[0418] The following uses Examples 5 to 8 as examples to introduce the processing process of the receiving end of this application.

[0419] Example 5

[0420] Since the encoding scheme of the sending end and the decoding scheme of the receiving end match each other, the sending end uses the method of Example 1 for encoding, and the receiving end can use the method of Example 5 for decoding to obtain the black and white two-dimensional code QR0.

[0421] The sending end encodes the black and white QR code QR0 through the above example 1 to generate image N i , image R i And the image O' i And merge the three frames of image into the original video for loop playback.

[0422] Please refer to FIG. 4 b and FIG. 5 b , and compare FIG. 4 b and FIG. 5 b of the present application with FIG. 4 a and FIG. 5 a in the prior art.

[0423] FIG5b(1) shows an example of a frame image F of the encoded random checkerboard slow motion video displayed on the display screen of the transmitting end. i The receiving end can capture at least two frames of random checkerboard slow motion video displayed on the display screen of the sending end, here are four frames of images, namely, image R i-1 , image R i , image R i+1 , image R i+2 , and decode three consecutive frames of images to obtain the black and white two-dimensional code QR0.

[0424] In other embodiments, the receiving end may also decode at least two frames of the three consecutive frames of images to obtain the black and white two-dimensional code.

[0425] In FIG4 b , a single small grid at the same position in the random checkerboard slow-motion video is taken as an example to illustrate the decoding process at the receiving end. The same applies to other small grids, which will not be described here.

[0426] For example, the content displayed at position q1 shown in FIG5b(1) is a coded black and white QR code (e.g., image N i ), the small grid at the q1 position (a small black grid in the black and white QR code, represented by bit 1, corresponding to a small white grid in the inverted QR code) can be continuously refreshed and displayed between three brightness levels (a, b, and c shown in Table 1) according to the encoding scheme described in Example 1. The three brightness levels are respectively i The brightness of the small grid at position q1, image R i The brightness of the small grid at position q1 and the image O' i The brightness of the small grid at position q1 in the image.

[0427] As shown in FIG4b(1), it shows the image F in the encoded random checkerboard slow motion video displayed on the screen in sequence by the transmitter. i-1 、Image F i 、Image F i+1 、Image F i+2 In the figure, the four frames of images of the small grid at the position q1 shown in Figure 5b(1) are respectively image f i-1 、Image f i 、Image f i+1 、Image f i+2 .

[0428] 4b, B represents black (RGB value 0), G represents gray (RGB value 127), W represents white (RGB value 255), BG represents RGB value 64, and GW represents RGB value 191. This is for example only and is not intended to limit the present application.

[0429] FIG4b(1) shows a small grid at position q1 shown in FIG5b(1), and four frames of images are sequentially displayed in the video displayed at the transmitting end, where image f i-1 、Image f i 、Image f i+1 The brightness of the image is different, and their brightness values ​​(RGB values) can be a, b, c as shown in Table 1 of Example 1, and the image f i+2 The brightness value of the image f i-1 The brightness value is the same, and so on, the image f i+3 The brightness value of B is black (RGB value is 0), and the image f i+4 The brightness value of W represents white (RGB value is 255), which will not be described here.

[0430] Combining the screen refresh principle of the transmitter and the image capture principle of the receiver, as described in Figures 2 and 3, the rolling shutter and progressive scan update mechanisms produce diagonal fusion areas. In the image captured by the receiver, the upper left corner of each small grid represents the current frame, the lower right corner represents the previous frame, and the center area represents the gradual fusion of the two frames. It should be noted that the striped pattern of the fusion area in the image captured by the receiver will change (direction, width, etc.) depending on the camera sensor and screen operating mechanism of the receiver, and this is not restricted here.

[0431] FIG4b(2) shows four frames of images (images Ri and Ri) that can be obtained by sequentially capturing the display content of the display screen of the transmitting end. -1 , image R i , image R i+1 , image R i+2 ), the four frames of images of the small grid at position q1 shown in Figure 5b(1) are image ri-1 、Image r i 、Image r i+1 、Image r i+2 .

[0432] It should be understood that when the receiving end captures the display screen content of the sending end's display screen for decoding to obtain a black and white QR code, it is not limited to capturing four frames of images, but only needs to capture at least two frames of images. In addition, the at least two frames of images captured can be two consecutive frames of images (for example, image R i , image R i+1 ), or two discontinuous frames of images (e.g., image R i and image R i+2 ) etc., no restrictions here.

[0433] As shown in Figure 4b(2), the image r captured by the receiving end i-1 There is also the current frame, here is the image f i-1 (gray area represented by G), and the previous frame, here is image f i-2 (white area represented by W), and there is also an image f between the gray area and the white area i-1 and image f i-2 The fusion area, here is the area represented by GW.

[0434] As shown in Figure 4b(2), the image r captured by the receiving end i There is also image f i (black area represented by B) and image f i-1 (Gray area represented by G), and there is an image f between the gray area and the black area i and image f i-1 The fusion area, here is the area represented by BG.

[0435] Similarly, as shown in Figure 4b(2), the image r captured by the receiving end i+1 There is also image f i+1 (white area represented by W) and image f i (black area represented by B), and there is an image f between the black area and the white area i and image f i+1 The fusion area is the gray area represented by G.

[0436] Similarly, as shown in Figure 4b(2), the image r captured by the receiving end i+2 There is also image f i+1 (white area represented by W) and image f i+2(Gray area represented by G), and there is an image f between the gray area and the white area i+1 and image f i+2 The fusion area, here is the area represented by GW.

[0437] Compared with the three frames of images taken by the receiving end shown in FIG4a(2) in the related art, since the sending end only uses two colors to encode the black area (or white area) in the QR code, the sending end only displays the black image shown in FIG4a(1) alternately (for example, the image f shown in FIG4a(2) i ) and the white image shown in FIG4a(1) (eg image f i+1 ). The fusion area in any frame of the image shown in Figure 4a(2) is the fusion area of ​​the current frame and the previous frame, that is, the fusion area of ​​the black image and the white image. As a result, the fusion areas of the two consecutive frames of the image shown in Figure 4a(2) taken by the receiving end may have the same color, such as the gray represented by G in Figure 4a(2). This results in the difference result of the two consecutive frames of the image taken by the receiving end (as shown in Figure 4a(3) |r i -r i+1 |), there may be a black area (represented by B) in the fusion area, and the fusion area is in the diagonal direction of the small grid image, but the fusion area has a certain width, which makes it easy for more pixel points (for example, 10 to 25, etc., not limited) to have a difference value of black, resulting in the problem of being unable to decode.

[0438] However, in the embodiment of the present application, as shown in FIG4b(1), the transmitting end alternately displays images encoded in three colors (represented by G, B, and W, respectively). Similarly, the fusion area in any frame of image taken by the receiving end is the fusion area of ​​the current frame and the previous frame. However, because the transmitting end encodes the same area of ​​the QR code in the above three colors, as shown in FIG4b(2), the brightness (or color) of the fusion area between the pixels corresponding to the same position (here, the above-mentioned q1 position) of any two frames of image taken by the receiving end (even if they are two consecutive frames) are different.

[0439] Please refer to the four frames of images at the position q1 obtained by the receiving end shooting the display screen of the sending end as shown in Figure 4b(2). In each of the four frames of images, there is a fusion area between the current frame displayed by the sending end and the previous frame. The fusion area extends from the lower left corner to the upper right corner and has a certain width. However, the brightness of the fusion area in the three consecutive frames of images shot by the receiving end as shown in Figure 4b(2) is different.

[0440] Next, as shown in FIG4b(3), in order to decode the small grid at the position q1 shown in FIG5b(1) to obtain an image of the QR code at the position q1, the receiving end may perform difference processing on at least one group of images (one group of images is two frames) among the three consecutive frames of images at the position q1 shown in FIG4b(2) above.

[0441] In this embodiment, the receiving end may perform difference processing on the three consecutive frames of images at the position q1 shown in FIG4b(2). Specifically, the receiving end may perform difference processing on the image r shown in FIG4b(2). i and image r i+1 Perform the difference and then calculate the absolute value to obtain the image shown in Figure 4b(3) |r i -r i+1 |; and the receiving end can process the image r as shown in FIG4b(2) i and image r i+2 Perform the difference and then calculate the absolute value to obtain the image shown in Figure 4b(3) |r i -r i+2 |; and the receiving end can process the image r as shown in FIG4b(2) i+1 and image r i+2 Perform the difference and then calculate the absolute value to obtain the image shown in Figure 4b(3) |r i+1 -r i+2 |.

[0442] In other embodiments, the receiving end may also perform difference processing on only two frames of captured images to obtain any one of the three images shown in FIG4b(3). Even if the receiving end performs difference processing on two consecutive frames of captured images, since the brightness of the respective fusion areas of any two frames of captured images is different (because the transmitting end encodes the black area or white area into three brightness levels), after the receiving end performs difference processing on the two consecutive frames of captured images, the difference result in the fusion area is not zero, and thus the black area shown in FIG4a(3) will not be obtained by difference processing.

[0443] Alternatively, in other embodiments, the receiving end may also perform a difference operation on any two frames of the three captured images, for example, to obtain the image shown in FIG4b(3): |r i -r i+1 |、Image|r i -r i+2 |. Optionally, the receiving end can receive the obtained image|r i -r i+1 |、Image|r i -r i+2 |Do fusion and binarization, or, from image|r i -r i+1|、Image|r i -r i+2 |Select a frame of image for binarization.

[0444] In this embodiment, as shown in FIG4b(3), three difference results: image|r i -r i+1 |、Image|r i -r i+2 |、Image|r i+1 -r i+2 |There is no black area (represented by B) with an RGB value of 0 as shown in FIG. 4a (3) in the prior art.

[0445] In this embodiment, the transmitting end uses three frames (eg, image N) when displaying the encoded black and white two-dimensional code image. i , image R i And the image O' i ) loop display mode, and in the three-frame loop display scheme, because the third frame is introduced, after the receiving end differs the captured image, there will be no completely black area (RGB value is 0) in the absolute value of the difference.

[0446] As shown in Figure 4b(3), the three types of differential small grids (small squares) only have darkened areas in the diagonal direction, but do not include black areas (RGB value is 0, represented by B). If there is a black area in a small grid, then the small grid cannot be quantized to bit 1 through a reasonable threshold and can only be bit 0. Therefore, the receiver can set a reasonable threshold for each small square shown in Figure 4b(3) to quantize the small grid at position q1 shown in Figure 5a(1) to bit 1, thereby achieving decoding of the small grid at position q1.

[0447] Then the receiving end can follow the principle of the process in FIG4b to display the images (here are four frames of images) on the display screen of the sending end in sequence: Image F i-1 、Image F i 、Image F i+1 、Image F i+2 Shoot to get four frames of images: Image R i-1 , image R i , image R i+1 , image R i+2 As shown in Figure 5b(2), the receiving end can process the image R i , image R i+1 Perform the difference and then calculate the absolute value to get the image |R i -R i+1 | is a binary image, where the image |R i -R i+1The image of the small grid at position q1 in | is the image shown in Figure 4b(3) |r i -r i+1 |binarized image; Similarly, as shown in Figure 5b(3), the receiving end can i , image R i+2 Perform the difference and then calculate the absolute value to get the image |R i -R i+2 | binary image, where the image |R i -R i+2 The image of the small grid at position q1 in | is the image shown in Figure 4b(3) |r i -r i+2 |The binarized image.

[0448] From the decoding results of the receiving end shown in Figures 5b(2) and 5b(3), we can see that the image shown in Figure 5b(2) does not recover the QR code, while the image shown in Figure 5b(3) is the correctly decoded black and white QR code QR0. In this way, even under the influence of the refresh stripes, the receiving end can also correctly extract the black and white QR code.

[0449] In the above example 1, the sending end encodes the black area in the black and white QR code (ie, the white area in the inverted QR code) into three brightness levels to obtain the image N i , image R i 、Image O' i After the receiving end performs differential processing according to the above process, the black small grid in the black and white QR code at position q1 shown in Figure 5b is used as an example to illustrate. According to the process of Figure 4b, the black small grid can be binarized to 1, and the white area in the black and white QR code (corresponding to the black area in the inverted QR code, refer to Table 1) is encoded as the same brightness (for example, d). Therefore, according to the principle of the process of Figure 4b, after the receiving end performs differential processing on the captured image, the white area in the black and white QR code can be differentially converted to 0 (that is, the differential result is a completely black area, and the RGB value represented by B is 0); then the receiving end can distinguish the differential results of the white area (differential result is 0) and the black area (differential result is not 0, as shown in Figure 4b (3), there is no completely black area represented by B in any small grid of the differential result) in the black and white QR code, thereby achieving correct decoding of the black and white QR code hidden in the video.

[0450] FIG5b(3) above shows a method of performing a difference between two captured images, binarizing the difference result, and optionally performing a color inversion process on the binarization result to restore a black and white QR code.

[0451] When the receiving end obtains the original black and white QR code QR0 based on the difference result, whether the binarized difference result (also expressed as a difference image) is inverted, that is, the black grids in the difference image are adjusted to white grids, and the white grids in the difference image are adjusted to black grids, is related to whether the sending end has inverted the black and white QR code QR0, and is also related to which two frames of images are captured and selected by the receiving end for difference.

[0452] In this example 5, when the sender performs encoding through example 1, the black and white QR code QR0 is inverted, and as shown in Table 1, the inverted QR code QR r The encoding result of the black area in is d, and its differential value is zero, then the black area is binarized to 0 (represented by a small white grid in the differential image). Therefore, in this example 5, the receiving end can invert the image after the differential result is binarized (that is, the above-mentioned differential image) to obtain the black and white QR code QR0.

[0453] In some embodiments, the receiving end may determine whether to perform color inversion processing on the difference image based on the following strategy:

[0454] For example, if the color of the pattern carrying the information in the binarized differential image is the same as the background color (the color of the border of the QR code), such as white, the receiving end can invert the color of the binarized differential image to obtain a black and white QR code QR0; on the contrary, if the color of the pattern carrying the information in the binarized differential image is different from the background color (the color of the border of the QR code is white), it means that the color of the pattern carrying the information in the binarized differential image is black (black represents bit 1 in the QR code), and the receiving end does not need to perform inversion processing.

[0455] In some embodiments, among the at least three frames of images captured by the receiving end, a single differential result of two frames of images may not be able to directly restore the black and white QR code. For example, the binarized image of the differential result shown in Figure 5b (2) only restores part of the QR code.

[0456] Then in some embodiments, combined with the encoding scheme of Example 1 above, the receiving end of the present application can also fuse multiple differential results to obtain a complete black and white QR code.

[0457] Specifically, please refer to Figure 8:

[0458] FIG8(1) is a frame of image in a video displayed at the transmitting end, for example, image Fi.

[0459] The image M shown in FIG8 (2) is the image R captured by the receiving end according to the principle of the process in FIG4b. i and image R i+1 Perform the difference and then calculate the absolute value to get the image |R i-R i+1 |The binarized image.

[0460] The image N shown in FIG8 (3) is the image R captured by the receiving end according to the principle of the process in FIG4b. i and image R i+2 Perform the difference and then calculate the absolute value to get the image |R i -R i+2 |The binarized image.

[0461] As can be seen from Figure 8(2) and Figure 8(3), the two-dimensional codes displayed by the image M and image N restored by the receiving end are both incomplete. However, the incomplete parts of the same black-and-white two-dimensional code are complementary to each other. Therefore, the receiving end can perform a logical AND operation on the image M and image N according to the pixels to obtain the complete black-and-white two-dimensional code shown in Figure 8(4).

[0462] Table 20

[0463] Specifically, the receiving end can assign the black pixel points (RGB value is 0) in image M and image N to 0, and assign the white pixel points (RGB value is 255) to 1, and then perform the logical AND operation on the pixels of image M and image N according to the logical AND operation process in Table 20, thereby obtaining a 21*21 0, 1 matrix; finally, the 0, 1 matrix is ​​restored to the black and white QR code shown in Figure 8 (3) in the manner of 0 being a black pixel point and 1 being a white pixel point.

[0464] In this embodiment, the receiving end may first binarize each differential result to obtain a binary image, and then fuse each binary image (for example, a logical AND operation, not specifically limited) to obtain a complete QR code.

[0465] In some embodiments, when the receiving end fuses two differential results, it may also first add the differential results, and then binarize the added differential results to obtain a complete QR code.

[0466] In this way, the receiving end can obtain a complete QR code by fusing multiple differential results.

[0467] Example 6

[0468] Since the encoding scheme of the sending end and the decoding scheme of the receiving end match each other, the sending end uses the method of Example 2 for encoding, and the receiving end can use the method of Example 6 for decoding to obtain the black and white two-dimensional code QR0.

[0469] The sender encodes the black and white QR code QR0 through the above example 2 to generate six frames of images: Image A i Image B i 、Image C i , Image D i 、Image E i , image G i And merge the six frames of images into the original video for loop playback.

[0470] Most of the procedures in Example 6 are the same as those in Example 5. The following mainly introduces the differences between the two:

[0471] The receiving end can capture the image R of the video displayed by the sending end i and image R i+3 Perform the difference and calculate the absolute value after the difference, then perform the binarization process to obtain the inverted QR code r ;

[0472] For example, the receiving end can perform the absolute difference between the first and fourth frames of the image (first perform the difference, then calculate the absolute value of the difference result), or perform the absolute difference between the second and fifth frames of the image to obtain the inverted QR code. r .

[0473] Optionally, the receiving end can obtain the inverted QR code r Perform inversion processing to obtain a black and white QR code QR0. Regarding whether to perform inversion processing, please refer to the relevant introduction of Example 5, which will not be repeated here.

[0474] Example 7

[0475] Since the encoding scheme of the sending end and the decoding scheme of the receiving end match each other, the sending end uses the method of Example 3 for encoding, and the receiving end can use the method of Example 7 for decoding to obtain the black and white two-dimensional code QR0.

[0476] The sender encodes the black and white QR code QR0 through the above example 3 to generate six frames of images: Image A i Image B i 、Image C i , Image D i 、Image E i , image G i And merge the six frames of images into the original video for loop playback.

[0477] Most of the procedures in Example 7 are the same as those in Example 6. The following mainly introduces the differences between the two:

[0478] The receiving end can capture the image R of the video displayed by the sending end iand image R i+2 Perform the difference and calculate the absolute value after the difference to obtain the difference result (the principle is similar to the three difference results shown in Figure 4b (3));

[0479] For example, the receiving end can perform absolute differential on the second and fourth frames of the image taken (meaning first perform differential and then calculate the absolute value of the differential result), or perform absolute differential on the third and fifth frames of the image taken) to obtain multiple differential results.

[0480] In some embodiments, a single differential result may not be able to completely restore the QR code after binarization, so the receiving end can fuse multiple differential results to obtain the inverted QR code. r For the specific fusion solution, please refer to the introduction of the relevant fusion solution in Example 5 above, which will not be repeated here.

[0481] Optionally, the receiving end can obtain the inverted QR code r Perform inversion processing to obtain a black and white two-dimensional code QR0.

[0482] Example 8

[0483] Since the encoding scheme of the sending end and the decoding scheme of the receiving end match each other, the sending end uses the method of Example 4 for encoding, and the receiving end can use the method of Example 8 for decoding to obtain the black and white two-dimensional code QR0.

[0484] The sender encodes the black and white QR code QR0 through the above example 4 to generate six frames of images: Image A i Image B i 、Image C i , Image D i 、Image E i , image G i And merge the six frames of images into the original video for loop playback.

[0485] Most of the procedures in Example 8 are the same as those in Example 6. The following mainly introduces the differences between the two:

[0486] The receiving end can capture the image R of the video displayed by the sending end i , respectively with the image R i+1 , image R i+2 , image R i+3 Perform the difference and calculate the absolute value of each difference to obtain the difference result (the principle is similar to the difference result shown in Figure 4b (3)). The difference result here includes: Image | R i -R i+1 |、Image|R i -R i+2 |、Image|Ri -R i+3 |;

[0487] In some embodiments, a single differential result may not be able to completely restore the QR code after binarization, so the receiving end can fuse multiple differential results to obtain the inverted QR code. r For the specific fusion solution, please refer to the introduction of the relevant fusion solution in Example 5 above, which will not be repeated here.

[0488] Finally, the receiving end can get the inverted QR code r Perform inversion processing to obtain a black and white two-dimensional code QR0.

[0489] Since the screen is refreshed line by line (60Hz) and the camera is also exposed line by line (for example, 60fps), the receiving end can capture the slowly moving inclined refresh stripes during the photo shooting process (including the i-th frame and the i-1-th frame and the average of the two frames). However, the existing differential encoding scheme will be affected by the refresh stripes and thus cannot correctly differentially decode the graphic code. The sending end of the present application adds at least one new frame on the basis of the double-frame differential video. In the above examples 5 to 8 of the present application, there is no exactly the same pattern corresponding to the same position in the different video frames of the video shot by the receiving end, and then a complete QR code can be obtained by differential to solve the influence of refresh stripes. After adopting the technical solution of the present application, the graphic code hidden in the video is not easy to be found by the human eye, but the receiving end can correctly decode the graphic code by shooting the image, thereby decoding the target information carried by it in the graphic code.

[0490] In one possible embodiment, the present application provides an image processing device. The image processing device includes: an acquisition module for capturing at least three frames of images alternately displayed on a screen of a transmitting end to obtain at least two frames of images; wherein the at least three frames of images are encoding results of a first graphic code; wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, the at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and the at least one first unit pattern in the first part of the third image is encoded with a third image parameter; an encoding module for calculating the difference in image parameters for at least one group of two frames of the at least two frames of images; and a display module for acquiring the first graphic code based on the difference to obtain target information represented by the first graphic code.

[0491] In a possible implementation, the first image parameter, the second image parameter, and the third image parameter are different in a color space.

[0492] In a possible implementation manner, the first image parameter, the second image parameter, and the third image parameter have different brightness.

[0493] In a possible implementation manner, the first graphic code further includes a second part; and the second parts of the at least three frames of images are encoded using the same image parameters.

[0494] In a possible embodiment, the first graphic code also includes a second part; at least one second unit pattern in the second part of the first image is encoded with a fourth image parameter, at least one second unit pattern in the second part of the second image is encoded with a fifth image parameter, and at least one second unit pattern in the second part of the third image is encoded with a sixth image parameter.

[0495] In a possible implementation, the fourth image parameter, the fifth image parameter, and the sixth image parameter are different in a color space.

[0496] In a possible implementation, the fourth image parameter, the fifth image parameter, and the sixth image parameter have different brightness.

[0497] In a possible implementation, the display module is specifically configured to alternately display the at least three frames of images on a background image, and a pattern in the background image is related to a pattern in the first graphic code.

[0498] In a possible implementation, the background image is a dynamic image.

[0499] In a possible implementation manner, the size of the pattern in the first graphic code is smaller than or equal to the size of the related pattern in the background pattern.

[0500] In a possible implementation manner, an average value of the first image parameter, the second image parameter, and the third image parameter is related to an image parameter of the background image.

[0501] In a possible implementation, the encoding module is specifically configured to encode the first graphic code based on the fourth image to generate at least three frames of images of the first graphic code.

[0502] In a possible implementation, the display module is specifically configured to alternately display the first image, the second image, and the third image, and to loop the alternate display process n times, where n is a positive integer.

[0503] In a possible implementation, during the n alternating display processes of the at least three frames of images, there are at least two alternating display processes in which the display order of the images is different.

[0504] In one possible embodiment, the at least three frames of images also include at least one fifth image, wherein the first part of the fifth image is encoded with the first image parameters; the display module is specifically used to alternately display the first image, the second image, the third image, and the at least one fifth image, and to loop the alternating display process n times.

[0505] In a possible embodiment, the image display order corresponding to the at least one alternating display process of the at least three frames of images is the first image, the second image, and the third image, and the second image parameter is related to the average of the first image parameter and the third image parameter.

[0506] The effects of the image processing apparatuses of the above embodiments are similar to the processes and effects of the image processing methods executed by the transmitting end of the above embodiments, and are not described in detail here.

[0507] In one possible embodiment, the present application provides an image processing device. The image processing device includes: a shooting module, configured to shoot at least three frames of images alternately displayed on a screen of a transmitting end to obtain at least two frames of images; wherein the at least three frames of images are encoding results of a first graphic code; wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, the at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and the at least one first unit pattern in the first part of the third image is encoded with a third image parameter; a calculation module, configured to calculate the difference in image parameters for at least one group of two frames of the at least two frames of images; and an acquisition module, configured to acquire the first graphic code based on the difference to obtain target information represented by the first graphic code.

[0508] In a possible implementation, each of the at least two frames of images includes a fusion area, wherein the fusion area is a fusion image of two adjacently displayed frames of images among the at least three frames of images displayed on the screen.

[0509] In a possible implementation, the at least two frames of images include a fourth image and a fifth image captured sequentially; and the calculation module is specifically configured to calculate a difference in image parameters between the fourth image and the fifth image.

[0510] In a possible implementation manner, the number of the at least two frames of images is 3 or more.

[0511] In one possible embodiment, the calculation module is specifically used to calculate the difference in image parameters for at least two groups of images among the at least three frames of images captured, thereby obtaining at least two groups of difference values, wherein each group of the at least two groups of images includes two frames of images; and the acquisition module is specifically used to fuse the at least two groups of difference values ​​to obtain the first graphic code.

[0512] In a possible implementation, the acquisition module is specifically configured to: perform binarization processing on the at least two groups of differences to obtain at least two groups of binarization results; and perform an AND operation on the at least two groups of binarization results to obtain the first graphic code.

[0513] In a possible implementation, the acquisition module is specifically configured to: add the at least two groups of difference values ​​to obtain an addition result; and perform binarization processing on the addition result to obtain the first graphic code.

[0514] The effects of the image processing apparatuses of the above embodiments are similar to the processes and effects of the image processing methods executed by the receiving ends of the above embodiments, and are not described in detail here.

[0515] In one possible implementation, FIG9 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. As shown in FIG9 , the device 500 may include: a processor 501 , a transceiver 505 , and optionally a memory 502 .

[0516] The transceiver 505 may be referred to as a transceiver unit, a transceiver, or a transceiver circuit, etc., and is configured to implement transceiver functions. The transceiver 505 may include a receiver and a transmitter. The receiver may be referred to as a receiver or a receiving circuit, etc., and is configured to implement a receiving function; the transmitter may be referred to as a transmitter or a transmitting circuit, etc., and is configured to implement a transmitting function.

[0517] The memory 502 may store a computer program or software code or instruction 504, which may also be referred to as firmware. The processor 501 may implement the image processing methods provided in various embodiments of the present application by running the computer program or software code or instruction 503 therein, or by calling the computer program or software code or instruction 504 stored in the memory 502. The processor 501 may be a central processing unit (CPU), and the memory 502 may be, for example, a read-only memory (ROM) or a random access memory (RAM).

[0518] The processor 501 and transceiver 505 described in this application can be implemented on an integrated circuit (IC), an analog IC, a radio frequency integrated circuit RFIC, a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc.

[0519] The image processing device 500 may further include an antenna 506 . The modules included in the image processing device 500 are merely examples and are not limited in this application.

[0520] The structure of the image processing device may not be limited to FIG9 . The image processing device may be an independent device or may be part of a larger device. For example, the image processing device may be implemented as follows:

[0521] (1) An independent integrated circuit IC, or chip, or chip system or subsystem; (2) A collection of one or more ICs, optionally including a storage component for storing data and instructions; (3) A module that can be embedded in other devices; (4) In-vehicle equipment, etc.; (5) Others, etc.

[0522] If the image processing device is implemented as a chip or a chip system, please refer to the chip structure diagram shown in Figure 10. The chip shown in Figure 10 includes a processor 601 and an interface 602. The number of processors 601 can be one or more, and the number of interfaces 602 can be multiple. Optionally, the chip or chip system can include a memory 603.

[0523] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0524] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to implement the above-mentioned image processing method embodiment.

[0525] Based on the same technical concept, an embodiment of the present application also provides a computer program, which, when executed by a terminal device, is used to implement the above-mentioned image processing method embodiment.

[0526] The program may be stored in whole or in part on a storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor.

[0527] Based on the same technical concept, an embodiment of the present application further provides a chip including a processor. The processor can implement the above-mentioned image processing method embodiment.

[0528] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0529] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0530] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first graphic code, wherein the first graphic code includes a first part for representing target information; Encoding the first graphic code to generate at least three frames of images of the first graphic code, wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and at least one first unit pattern in the first part of the third image is encoded with a third image parameter; The at least three frames of images are displayed alternately to transmit the target information.

2. The method according to claim 1, characterized in that: The first image parameter, the second image parameter, and the third image parameter are different in color space.

3. The method according to claim 1 or 2, characterized in that: The first image parameter, the second image parameter, and the third image parameter have different brightness.

4. The method according to any one of claims 1 to 3, characterized in that The first graphic code also includes a second part; The second parts of the at least three frames of images are encoded using the same image parameters.

5. The method according to any one of claims 1 to 3, characterized in that: The first graphic code also includes a second part; At least one second unit pattern in the second part of the first image is encoded with a fourth image parameter, at least one second unit pattern in the second part of the second image is encoded with a fifth image parameter, and at least one second unit pattern in the second part of the third image is encoded with a sixth image parameter.

6. The method according to claim 5, characterized in that The fourth image parameter, the fifth image parameter, and the sixth image parameter are different in color space.

7. The method according to claim 5 or 6, characterized in that: The fourth image parameter, the fifth image parameter, and the sixth image parameter have different brightness.

8. The method according to any one of claims 1 to 7, characterized in that The alternately displaying the at least three frames of images to transmit the target information comprises: The at least three frames of images are alternately displayed on a background image, and a pattern in the background image is related to a pattern in the first graphic code.

9. The method according to claim 8, characterized in that The background image is a dynamic image.

10. The method according to claim 8 or 9, characterized in that: The size of the pattern in the first graphic code is smaller than or equal to the size of the related pattern in the background pattern.

11. The method according to any one of claims 8 to 10, characterized in that An average value of the first image parameter, the second image parameter, and the third image parameter is related to the image parameter of the background image.

12. The method according to any one of claims 1 to 11, characterized in that The step of encoding the first graphic code to generate at least three frames of images of the first graphic code includes: The first graphic code is encoded based on the fourth image to generate at least three frames of images of the first graphic code.

13. The method according to any one of claims 1 to 12, characterized in that The alternately displaying the at least three frames of images to transmit the target information comprises: The first image, the second image, and the third image are displayed alternately, and the process of the alternate display is repeated n times, where n is a positive integer.

14. The method according to claim 13, characterized in that In the n alternating display processes of the at least three frames of images, there are at least two alternating display processes in which the display order of the images is different.

15. The method according to claim 13 or 14, characterized in that The at least three frames of images further include at least one fifth frame of image, wherein the first part of the fifth image is encoded with the first image parameter; The alternately displaying the at least three frames of images to transmit the target information comprises: The first image, the second image, the third image, and the at least one frame of the fifth image are displayed alternately, and the process of the alternate display is repeated n times.

16. The method according to any one of claims 13 to 15, characterized in that The image display order corresponding to the at least one process of alternating display of the at least three frames of images is the first image, the second image, and the third image, and the second image parameter is related to the average of the first image parameter and the third image parameter.

17. An image processing method, characterized in that: The method comprises: Shooting at least three frames of images alternately displayed on a screen of a transmitting end to obtain at least two frames of images; Wherein, the at least three frames of images are encoding results of the first graphic code; Wherein, the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and at least one first unit pattern in the first part of the third image is encoded with a third image parameter; For at least one group of two frames of images among the at least two frames of images, calculating a difference in image parameters; Based on the difference, the first graphic code is acquired to obtain target information represented by the first graphic code.

18. The method according to claim 17, characterized in that Each of the at least two frames of images includes a fusion area, wherein the fusion area is a fusion image of two adjacently displayed frames of images among the at least three frames of images displayed on the screen.

19. The method according to claim 17 or 18, characterized in that The at least two frames of images include a fourth image and a fifth image captured sequentially; and calculating the difference of image parameters for at least one group of two frames of images among the at least two frames of images includes: For the fourth image and the fifth image, a difference in image parameters is calculated.

20. The method according to claim 17 or 18, characterized in that The number of the at least two frames of images is 3 or more.

21. The method according to claim 20, characterized in that The calculating the difference of the image parameters for at least one group of two frames of the at least two frames of images comprises: For at least two groups of images among the at least three frames of images obtained by shooting, respectively calculate differences in image parameters to obtain at least two groups of differences, wherein each group of images in the at least two groups of images includes two frames of images; The acquiring the first graphic code based on the difference to obtain the target information represented by the first graphic code includes: The at least two sets of difference values ​​are merged to obtain the first graphic code.

22. The method according to claim 21, characterized in that The step of fusing the at least two sets of difference values ​​to obtain the first graphic code includes: Binarization is performed on the at least two groups of difference values ​​to obtain at least two groups of binarization results; An AND operation is performed on the at least two groups of binarization results to obtain the first graphic code.

23. The method according to claim 21, characterized in that The step of fusing the at least two sets of difference values ​​to obtain the first graphic code includes: Adding the at least two groups of difference values ​​to obtain an addition result; Binarization is performed on the addition result to obtain the first graphic code.

24. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a first graphic code, wherein the first graphic code includes a first part for representing target information; An encoding module, used for encoding the first graphic code to generate at least three frames of images of the first graphic code, wherein the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and at least one first unit pattern in the first part of the third image is encoded with a third image parameter; The display module is used to alternately display the at least three frames of images to transmit the target information.

25. An image processing device, characterized in that: The device comprises: A shooting module, used for shooting at least three frames of images alternately displayed on the screen of the transmitting end to obtain at least two frames of images; Wherein, the at least three frames of images are the encoding results of the first graphic code; wherein, the at least three frames of images include a first image, a second image, and a third image; wherein at least one first unit pattern in the first part of the first image is encoded with a first image parameter, at least one first unit pattern in the first part of the second image is encoded with a second image parameter, and at least one first unit pattern in the first part of the third image is encoded with a third image parameter; A calculation module, used for calculating the difference of image parameters for at least one group of two frames of images among the at least two frames of images; An acquisition module is used to acquire the first graphic code based on the difference to obtain target information represented by the first graphic code.

26. A computer-readable storage medium, characterized in that: The method comprises a computer program, which, when running on a computer or a processor, enables the computer or the processor to execute the method according to any one of claims 1 to 16 or claims 17 to 23.

27. An image processing device, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory and send the signal to the processor, the signal comprising a computer instruction stored in the memory; when the processor executes the computer instruction, the processor is used to execute the method as claimed in any one of claims 1 to 16, or claims 17 to 23.

28. A computer program product, characterized in that The computer program product comprises a software program, and when the software program is executed by a computer or a processor, the steps of the method of any one of claims 1 to 16, or claims 17 to 23 are performed.

Images