A pointing interaction method and system based on hidden embedding of QR codes

Abstract

This invention belongs to the field of display interaction technology and provides a pointing interaction method and system based on hidden QR code embedding. Several QR code images are hidden and embedded in the display screen of the target screen. These QR code images utilize the human eye's flicker fusion effect, displaying frames alternately on the target screen according to a preset refresh rate, making the QR code images invisible to the human eye. In response to scanning and capturing by a smartphone camera, the QR code images are extracted and recognized. Based on the extracted and recognized information, the coordinates of the smartphone camera's optical center on the screen are calculated, obtaining the corresponding pointing position of the user on the screen. A cursor pattern that moves with the smartphone is displayed at the corresponding coordinate position, realizing pointing positioning of the smartphone on the display screen. This invention realizes pointing positioning function based on a smartphone on the display screen.

Description

Technical Field

[0001] This invention belongs to the field of display interaction technology, specifically relating to a pointing interaction method and system based on hidden embedding of QR codes. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Currently, an increasing number of public venues such as exhibition halls and museums are using large screens to display information. Because these screens are very large, viewers often need to maintain a certain distance from them, making close-range interactive technologies such as touch screens often unsuitable. Currently, these displays typically only show information in a fixed sequence, or the guide switches between screens using a control panel or handheld computer, rather than allowing users to freely select different content based on their interests and needs.

[0004] Currently, there are also public display screen interaction technologies based on infrared camera devices. Users project infrared target images onto the area of ​​interest on the screen using an infrared laser head on a dedicated handheld interactive device. The system then captures the infrared target images in real time using an infrared camera installed on the opposite side of the screen and determines the target's position using computer vision algorithms, thus achieving a pointing function. Additionally, there are methods that provide interactive support to viewers through gesture and eye-tracking technologies by installing motion capture devices or eye trackers in the exhibition space. However, these methods require additional equipment to be installed in the exhibition space, which presents significant limitations in terms of equipment installation and deployment, especially when multiple people are freely moving around the large screen and interacting with the displayed content. Furthermore, most of these technologies lack the ability to handle complex information input.

[0005] There is already research on technologies that enable interaction between smartphones and public displays. A typical approach is to geometrically map the public display screen to the phone screen. For example, by mapping the four corners of a large rectangular screen to the phone screen, users can use their phones to swipe and navigate on the large screen. However, this approach can lead to a significant reduction in positioning resolution when the physical size of the public display screen is large. Furthermore, users need to constantly swipe while moving around and watching movies, resulting in a poor user experience. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a pointing interaction method and system based on QR code hiding and embedding. This invention hides and embeds a set of QR codes within the screen display. The QR codes are extracted and recognized by scanning with a mobile phone camera, and the corresponding position of the camera's optical center on the screen is calculated. This allows the user to determine their pointing position on the screen. To avoid affecting the display and viewing experience, this invention utilizes the flicker fusion effect of the human visual system to hide the QR codes, making them invisible to the human eye. On the mobile phone, a liquid crystal shutter lens attached to the front of the camera ensures that the camera can correctly capture the hidden QR code image. This achieves the effect of the QR code being invisible to the human eye but visible to the smartphone. Pointing positioning calculations are performed based on the QR code image captured by the mobile phone to obtain the precise coordinates of the phone's optical center mapped onto the screen. A cursor pattern that moves with the user's phone is then displayed at the corresponding position, realizing a pointing positioning function on the smartphone display screen.

[0007] According to some embodiments, the present invention adopts the following technical solution:

[0008] A pointing interaction method based on hidden embedding of QR codes includes the following steps:

[0009] Several QR code images are hidden and embedded in the display of the target screen. The QR code images utilize the human eye flicker fusion effect and are displayed on the target screen in alternating frames according to a preset refresh rate, so that the QR code images are invisible to the human eye.

[0010] In response to scanning and capturing images with a smartphone camera, the system extracts and recognizes pre-hidden embedded QR code images in the display of the target screen.

[0011] Based on the extracted and recognized QR code image information, the coordinates of the smartphone camera's optical center on the screen are calculated, the corresponding pointing position of the user on the screen is obtained, and a cursor pattern that moves with the smartphone is displayed at the corresponding coordinate position, thus realizing the smartphone's pointing positioning on the display screen.

[0012] As an alternative implementation, the process of hiding and embedding several QR code images in the display of the target screen includes: subtracting the original image to be displayed from the QR code image with matching brightness to obtain a difference frame image that matches the screen device's refresh rate; and alternating the display of the QR code image and the difference frame image on the target screen according to the screen refresh rate.

[0013] As an alternative implementation, the QR code is a QR code, which is binarized before being hidden and embedded to obtain a QR code image containing only black and white pixels.

[0014] As a further defined implementation, when the screen refresh rate is 120Hz, the original image to be displayed is subtracted from the QR code image, and then displayed alternately in a manner that consists of one QR code image and one difference frame image.

[0015] As a further defined implementation, when the screen refresh rate is 240Hz, the original image to be displayed is subtracted from the QR code image at 1 / 3 brightness, and then displayed alternately in a manner that consists of one QR code image and three difference frame images as a group.

[0016] As a further defined implementation, the process of subtracting the original image to be displayed from the QR code image to obtain a difference frame image that matches the screen device's refresh rate includes: performing subtraction operations on the red, green, and blue channels of the image respectively, normalizing the values ​​of each channel to a set range, and when performing subtraction operations on each channel, first performing gamma correction on the values ​​of their respective red, green, and blue channels before performing subtraction, then performing inverse gamma correction, and finally obtaining the correct difference value and outputting it for display.

[0017] As a further refined implementation, during the subtraction operation on the red, green, and blue channels of the image, underflow detection is performed on the values ​​of the red, green, and blue channels of each pixel in the original image. Pixel areas where the minimum value of the red, green, and blue channels is lower than a set threshold are set to black and marked as non-embedding areas, while those with a minimum value are set to white and marked as embedding areas. A mask image of the original image is generated. When deploying the embedded QR code, the non-embedding areas marked in the mask image are avoided to ensure that the embedding position of the QR code is located in the embedding area.

[0018] As an alternative implementation, the process of calculating the coordinate position of the smartphone camera's optical center on the screen based on the extracted and identified corresponding QR code image information includes: performing adaptive binarization processing on the QR code image captured by the mobile phone to remove stray light and noise interference to the QR code structure; identifying and decoding each complete QR code structure contained in the image to obtain the unique ID code information corresponding to each QR code structure, as well as its spatial pose transformation data relative to the smartphone's camera coordinate system; calculating a set of candidate projection points of the smartphone's camera optical center on the screen plane; and then performing interpolation calculation on the candidate projection points to finally obtain the accurate coordinates of the smartphone's optical center mapped on the screen.

[0019] A directional interaction system based on hidden QR code embedding, comprising:

[0020] The QR code image embedding display and hiding module is configured to hide and embed several QR code images in the display screen of the target screen. The QR code images utilize the human eye flicker fusion effect and are displayed on the target screen in alternating frames according to a preset refresh rate, so that the QR code images are invisible to the human eye.

[0021] The image extraction and recognition module is configured to extract and recognize a pre-hidden embedded QR code image in response to scanning and capturing by a smartphone camera;

[0022] The pointing and positioning calculation module is configured to calculate the coordinate position of the smartphone camera's optical center on the screen based on the extracted and recognized corresponding QR code image information, obtain the corresponding pointing position of the user on the screen, and display a cursor pattern that moves with the smartphone at the corresponding coordinate position, thereby realizing the pointing and positioning of the smartphone on the display screen.

[0023] As an alternative implementation, the smartphone has a liquid crystal shutter lens module in front of its camera to acquire screen frame synchronization pulse signals and adapt the lens opening and closing sequence according to the screen refresh rate, so that the lens only transmits light in one direction during the opening period, allowing only the QR code frame image to enter the camera lens, while completely blocking light during the rest of the time.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention enables real-time pointing and positioning functionality and interaction in large-space public display environments, overcoming the limitations of existing technologies. By using a smartphone paired with a dedicated shutter liquid crystal lens, this invention can scan and identify hidden QR codes on the screen, enabling pointing, positioning, and selection of displayed objects without requiring additional dedicated interactive devices in the environment. Compared to existing technologies, it is more suitable for scenarios where multiple users simultaneously operate screen objects in large public display environments while freely moving around, improving the convenience and scalability of public screen interaction.

[0026] This invention achieves both visual hiding and seamless embedding of QR codes, balancing directional positioning functionality with optimal screen visual display. Utilizing the human eye's flicker fusion effect, this invention employs a high refresh rate frame alternation display strategy to visually hide the QR code, ensuring the human eye perceives only the original screen display, without affecting the normal viewing experience. Simultaneously, through refresh rate adaptation design, the area where the QR code cannot be embedded is significantly reduced, further enhancing the adaptability and stability of the QR code hiding solution.

[0027] The method of this invention is a content-independent pointing and positioning method, which effectively improves the versatility and practicality of the technology. In this invention, the QR code used for positioning is an independent display frame, which is completely unrelated to the original content displayed on the screen. It eliminates the need for feature extraction and registration of the original image beforehand, and also eliminates the need to download the feature information of the display image to the mobile device in advance. This simplifies the interaction process, breaks through the dependence of traditional positioning methods on the content of the display image, and expands the applicability of the pointing and positioning function.

[0028] The QR code used for positioning in this invention can adopt various existing QR code planar marking patterns for spatial pose estimation in the field of computer vision. The technical implementation of this invention does not depend on any specific QR code pattern.

[0029] This invention features a multimedia interactive guide based on a large public display screen, showcasing diverse content formats, including animated videos and audio narration. The diversified content presentation attracts user interest, enhancing user engagement and immersion. Simultaneously, the exhibition content is highly flexible, adaptable to various application scenarios. It is not only suitable for museum and art gallery exhibitions but can also be widely applied to various venues such as school education, community cultural activities, and commercial displays, thus significantly expanding the system's application scope.

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0032] Figure 1 This is a schematic diagram illustrating an application of a smartphone-based pointing method according to one embodiment;

[0033] Figure 2 This is a schematic diagram illustrating the generation of a difference frame image based on a 120Hz refresh rate, according to one embodiment.

[0034] Figure 3 This is a frame alternation display diagram based on a 120Hz refresh rate, according to one embodiment.

[0035] Figure 4 This is a schematic diagram illustrating an embodiment of underflow detection of the original image based on a 120Hz refresh rate to generate a mask image.

[0036] Figure 5 This is a schematic diagram illustrating the generation of a difference frame image based on a 240Hz refresh rate, according to one embodiment.

[0037] Figure 6 This is a schematic diagram illustrating frame alternation display based on a 240Hz refresh rate, according to one embodiment.

[0038] Figure 7 This is a schematic diagram illustrating an embodiment of underflow detection of the original image based on a 240Hz refresh rate to generate a mask image.

[0039] Figure 8 This is a schematic diagram of the opening and closing control of a liquid crystal shutter lens based on a 120Hz refresh rate, according to one embodiment.

[0040] Figure 9 This is a schematic diagram of liquid crystal shutter lens opening and closing control based on a 240Hz refresh rate according to one embodiment;

[0041] Figure 10 This is a schematic diagram illustrating an embodiment of a mobile phone camera independently capturing a QR code frame image based on a 120Hz refresh rate.

[0042] Figure 11 This is a schematic diagram illustrating an embodiment of a mobile phone camera independently capturing a QR code frame image based on a 240Hz refresh rate.

[0043] Figure 12 This is a flowchart of a method according to one embodiment. Detailed Implementation

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0047] Where there is no conflict, the embodiments and features described in this application may be combined with each other.

[0048] Example 1

[0049] A pointing interaction method based on hidden embedding of QR codes is described in this embodiment, which uses QR codes as an example.

[0050] like Figure 12As shown, it includes the following steps:

[0051] Several QR code images are hidden and embedded in the display of the target screen. The QR code images utilize the human eye flicker fusion effect and are displayed on the target screen in alternating frames according to a preset refresh rate, so that the QR code images are invisible to the human eye.

[0052] In response to scanning and capturing images with a smartphone camera, the system extracts and recognizes pre-hidden embedded QR code images in the display of the target screen.

[0053] Based on the extracted and recognized QR code image information, the coordinates of the smartphone camera's optical center on the screen are calculated, the corresponding pointing position of the user on the screen is obtained, and a cursor pattern that moves with the smartphone is displayed at the corresponding coordinate position, thus realizing the smartphone's pointing positioning on the display screen.

[0054] Specifically, such as Figure 1 As shown, a QR code is hidden and embedded in the screen display beforehand. The QR code is extracted and recognized by scanning and capturing it with a smartphone camera (or camera), and the corresponding position of the optical center of the smartphone camera (hereinafter referred to as the phone) on the screen is calculated, thereby obtaining the user's pointing position on the screen.

[0055] To avoid affecting the display and viewing experience, this invention utilizes the flicker fusion effect of the human visual system to hide the QR code, making it invisible to the human eye. On the mobile phone, a liquid crystal shutter lens attached to the front of the phone's camera ensures that the camera can correctly capture the hidden QR code image, achieving the effect of the QR code being invisible to the human eye but visible to the smartphone. Finally, based on the QR code image captured by the mobile phone, a pointing and positioning calculation is performed to obtain the precise coordinates of the phone's optical center mapped onto the screen. A cursor pattern that moves with the user's phone is then displayed at the corresponding position, realizing the pointing and positioning function on the smartphone display. This embodiment allows users to easily point to and select targets of interest on the screen using a smartphone.

[0056] Example 2

[0057] To implement the method of Embodiment 1, this embodiment provides a system for directional interaction based on hidden QR code embedding.

[0058] To facilitate the description of the specific implementation of this embodiment, the four required modules will be described in the following order: the QR code embedding, display and hiding module, the liquid crystal shutter lens module, the mobile phone pointing and positioning calculation module, and the human-computer interaction module.

[0059] The detailed design of each part is as follows:

[0060] Embedded display and hidden modules for QR codes:

[0061] In this embodiment, based on the human eye flicker fusion effect, a frame alternation display strategy of a high refresh rate screen is used to hide the QR code from the human eye while making it visible to the mobile phone camera. The steps are as follows:

[0062] Research in visual psychology and neuroscience has confirmed that when intermittent flickering light reaches a certain frequency, the human eye perceives it as a continuous and stable light signal; this phenomenon is called the flicker fusion effect. For most ordinary people, the critical value for the flicker fusion frequency is 50Hz-60Hz. Based on this characteristic, if the refresh rate of the display screen exceeds 60Hz (such as the existing commercially available 120Hz and 240Hz high refresh rate screens), special display effects can be achieved through special control of frame alternation.

[0063] A typical application is 3D stereoscopic display devices (such as 3D projectors and LED stereoscopic displays): Existing commercial 3D display devices typically have a refresh rate of 120Hz. Their working principle is as follows: First, left-eye and right-eye images with parallax shift are generated. Then, a 3D video stream is formed by alternating between "left-eye image - right-eye image - left-eye image," and displayed on a 120Hz refresh rate screen. For naked-eye users, the screen displays overlapping ghosting. The opening and closing sequence of the liquid crystal shutter glasses is strictly synchronized with the screen frame display sequence. For users wearing liquid crystal shutter glasses, when the screen displays the left-eye image, the left lens of the shutter glasses is open and the right lens is closed, so the user's left eye only receives the left-eye image, and the user's right eye perceives a completely black image. When the screen displays the right-eye image, the right lens of the shutter glasses is open and the left lens is closed, so the user's right eye only receives the right-eye image, and the user's left eye perceives a completely black image. Through this alternating control, the left and right eyes acquire images from their corresponding perspectives, ultimately forming a stereoscopic image effect in the visual system.

[0064] In the aforementioned 3D display process, the flicker fusion effect of the human eye plays an important role. For users wearing LCD shutter glasses, although the user's left and right eyes perceive a completely black image during the period when the lenses are closed, at a refresh rate of 120Hz, the refresh frequency of the left and right eye images is 60Hz (each frame is 8.33ms), which reaches the critical value of the human eye flicker fusion frequency. Therefore, the user's left and right eyes will not perceive the screen flicker and form a correct stereoscopic image perception effect.

[0065] As a typical implementation, a QR code hiding scheme based on a 120Hz refresh rate:

[0066] Based on the principles of 3D display described above, this embodiment designs the following basic embodiment based on a 120Hz refresh rate display to achieve embedded display and hiding of QR code images.

[0067] Step S101: Image preprocessing

[0068] like Figure 2 As shown, the QR code used for positioning is binarized to obtain a QR code image containing only black (background) and white (foreground) pixels, which is called a QR code frame. Simultaneously, the original image to be displayed is subtracted from the QR code image to obtain a difference image between the original display image and the QR code image, which is called a difference frame.

[0069] Step S102: Alternate frame display

[0070] The binarized QR code frame is used as the left-eye image, and the difference frame is used as the right-eye image. They are displayed alternately at a 120Hz refresh rate in the following order: "left-eye image (QR code frame) - right-eye image (difference frame) - left-eye image (QR code frame) - right-eye image (difference frame)". Each frame is displayed for 8.33 minutes (1 / 120 seconds), and the display periods of the QR code frame and the difference frame do not overlap. Figure 3 As shown.

[0071] According to the above-described frame alternation display method, due to the flicker fusion effect of human vision, what the human eye sees is the brightness fusion effect of the left-eye image (QR code frame) and the right-eye image (difference frame), that is, the human eye perceives the original image. Its equivalent refresh rate is 60Hz, and it does not produce a flickering or stuttering effect for the human eye. Due to the flicker fusion effect of the human eye, the QR code image is invisible to the human eye during display, achieving a hiding effect, such as... Figure 3 As shown. Simultaneously, in the video stream displayed on the screen, the QR code frame is displayed independently, with no overlap with the display period of the difference frame. Therefore, by controlling the opening and closing of the liquid crystal shutter lens attached to the front of the smartphone camera, the liquid crystal shutter lens can be opened during the QR code frame display period and closed during the difference frame display period. This allows the smartphone camera to capture an independent QR code image while filtering out the difference frame image. Consequently, the QR code image is visible to the phone camera, and positioning and pointing calculations can be performed through QR code extraction and recognition.

[0072] Step S103: The specific implementation process of subtracting the original image from the QR code image to generate a difference frame is explained below:

[0073] Step S1031: Channel processing rules for subtraction operation

[0074] The subtraction operation is performed on the three channels of the image: red (R), green (G), and blue (B). The values ​​of the R, G, and B channels are normalized to 0 to 1.

[0075] Step S1032: Subtraction operation process combined with gamma correction

[0076] The subtraction operation should be performed in conjunction with the gamma correction function of the display device. Specifically, when subtracting the R, G, and B channels of corresponding pixels of the original image and the QR code image, gamma correction should be performed on the R, G, and B channel values ​​before subtraction, and then inverse gamma correction should be performed to obtain the correct difference value and output it for display.

[0077] To accommodate the non-linear nature of human brightness perception, display devices perform gamma correction on the R, G, and B channel values ​​of the input image. Gamma correction establishes the non-linear relationship between the actual displayed brightness and the input R, G, and B channel values. The formula for gamma correction is: if the display brightness range of the screen device is normalized to 0 to 1, the value of a certain channel among the R, G, and B channels of the input image... , The actual display brightness after gamma correction is , where the index For fixed values ​​of display devices, such as those used for indoor displays, common values ​​are... The value is either 2.2 or 2.4. For outdoor displays, the common values ​​are... The value is 2.8, and the screen... The values ​​can be read from the monitor's configuration file. If you require the actual display brightness of a specific channel (R, G, B) of a pixel in the image to be L, then the value of that channel... The actual display brightness should be used. L It is obtained by performing an inverse gamma transform, i.e. .

[0078] Since the actual display brightness of the screen is the result of non-linear gamma correction on the R, G, and B channel values ​​of the input image, gamma correction should be applied when subtracting corresponding pixels from the original image and the QR code image. The specific implementation method is as follows:

[0079] A QR code image is a binary, pure grayscale image where the R, G, and B channel values ​​of its pixels are all equal. For the black pixels of the QR code For the white pixels of the QR code After gamma correction of the display device, the RGB channel values ​​are... The actual display brightness of the pixels is Let the RGB values ​​of the corresponding pixels in the original image be... After gamma correction by the display device, the actual display brightness of this pixel is When alternating between QR code images and difference images at a 120Hz refresh rate, to achieve complementarity between the QR code images and difference images under the flickering effect of the human eye, the QR code images are canceled out and hidden. The actual display brightness of the pixel in the difference image should be [value missing]. Then, the corresponding R, G, and B channel values ​​are obtained through inverse gamma transformation. Taking the R channel as an example, the R channel values ​​of the pixels in the difference frame obtained after subtraction satisfy Formula 1:

[0080] (1)

[0081] Similarly, the values ​​of the G and B channels of the pixels in the difference frame obtained after subtraction satisfy Equations 2 and 3, respectively:

[0082] (2)

[0083] (3)

[0084] Based on formulas 1-3 above, we can analyze the situation where QR code frames and difference frames are displayed alternately at a refresh rate of 120Hz. For the pixels in the QR code image... After gamma correction of the display device, the actual display brightness is The corresponding difference image pixels are After gamma correction of the display device, the actual display brightness is At a 120Hz refresh rate, the display duration of the alternating QR code image and the difference image is 8.33ms, reaching the flicker fusion frequency of the human eye. During the 16.66ms duration of the two image displays, the brightness perceived by the human eye is the time average of the actual display brightness of the QR code image and the difference image, i.e.:

[0085] (4)

[0086] When only the original image is displayed, without the QR code image, the brightness perceived by the human eye during the 16.66ms duration of displaying the two images is the time average of the brightness of the two original images, that is:

[0087] (5)

[0088] Comparing Formula 4 and Formula 5 above, when the QR code image and the difference image are displayed alternately at a refresh rate of 120Hz, the QR code image is canceled out due to the flicker fusion effect of the human eye. The human eye perceives the original image, but the brightness is reduced by 50%.

[0089] Step S1033: Measures to prevent underflow

[0090] During the subtraction process, the minimum brightness of the display is 0, and negative brightness is impossible. Subtracting the black pixels from the original image and the binarized QR code image results in the brightness value of that pixel in the original image. However, when subtracting from the white pixels in the QR code image, the brightness value of that pixel in the original image must be greater than the brightness value of the white pixels in the QR code image to avoid underflow and ensure that the display effect of the original image remains unchanged after fusion. Therefore, the values ​​of the R, G, and B channels of the white pixels in the binarized QR code image... It should not be set to the maximum value of 1, but rather to a medium brightness value, for example... =0.3, which ensures sufficient differentiation between black pixels (background) and white pixels (foreground) in the QR code image, while also ensuring that the brightness of the original image is greater than 0.3. The part can perform correct subtraction operations, avoiding underflow. And for the original image where the RGB channel values ​​are lower than... The regions that are not embedded are marked as non-embedded regions. Therefore, this invention introduces masking image technology, such as... Figure 4 As shown, underflow detection is performed on the R, G, and B channels of each pixel in the original image, and the minimum value of the R, G, and B channels is lower than the specified value. The pixel areas are set to black and marked as non-embedding areas, while the areas with visible pixels are set to white and marked as embedding areas, thus generating a mask image of the original image. When deploying QR codes, designers need to avoid the non-embedding areas marked by this mask image and ensure that the QR code is embedded within the embedding area.

[0091] S1034: GPU-based real-time computing

[0092] In the aforementioned subtraction operation, both gamma correction and inverse gamma correction involve power function operations. To ensure the real-time performance of the image rendering, the specific implementation accelerates the operation through the graphics card GPU. Specifically, after normalizing the RGB channel values ​​of the input image to [0, 1], a lookup table for gamma correction and inverse gamma correction is pre-generated based on the actual gamma value of the display device. The table data is then used to generate a texture image through pseudo-color encoding for the GPU to read. The relevant power function operations are directly completed through the lookup table data, improving computational efficiency and ensuring real-time performance.

[0093] The above method can achieve the effect of hiding and displaying QR code images, but it has the following problems at a 120Hz refresh rate. First, it requires two frames of images to be merged into one original image, which is equivalent to alternating between a completely black image and an original image. Therefore, the actual display brightness of the image is reduced to 50%, and the brightness attenuation is relatively large. Second, in order to avoid underflow when the images are subtracted, the non-embedded area in the original image is relatively large.

[0094] As another typical embodiment, a QR code hiding scheme based on a 240Hz refresh rate will be described in detail.

[0095] To further improve the brightness of the display and reduce the area where the QR code cannot be embedded in the original image, this invention uses an LED screen with a refresh rate of 240Hz as the display device, and designs the following optimized embodiment.

[0096] At a refresh rate of 240Hz, the display duration of each frame is 4.167ms. To hide the QR code image by utilizing the flicker fusion effect of the human eye, the display process uses four frames as one display cycle, employing a combination of "1 QR code frame + 3 difference frames". The total display duration of a single cycle is 16.66ms, corresponding to an equivalent screen refresh rate of 60Hz, which meets the flicker fusion frequency requirements of the human eye.

[0097] Compared to the 120Hz implementation set, where a single interpolation frame needs to completely cancel the QR code brightness, this implementation evenly distributes the QR code brightness cancellation task among three interpolation frames. Each interpolation frame only needs to bear one-third of the brightness cancellation amount, effectively reducing the brightness cancellation load of a single interpolation frame. This also provides core support for improving the overall display brightness of the screen and reducing the area in the original image where the QR code cannot be embedded.

[0098] Based on this core design of brightness distribution, the 240Hz refresh rate implementation achieves dual optimization in terms of image display effect and embedding area adaptability. Regarding brightness performance, relying on the human eye's flicker fusion effect, the human eye forms a time-averaged brightness perception of multiple frames within a single display cycle. In this implementation, the average perceived brightness can be increased to 75% of the original image, compared to only 50% for the 120Hz solution, effectively reducing brightness decay and significantly improving overall display brightness. In terms of optimizing areas where QR codes cannot be embedded, because the brightness cancellation amount of a single-frame difference frame is greatly reduced, the original image pixel brightness only needs to be higher than one-third of the QR code brightness to meet the requirement of no underflow in the subtraction operation. The brightness constraint is greatly relaxed, allowing more low-brightness areas in the original image to support QR code embedding, thus significantly reducing the range of areas that cannot be embedded. Furthermore, in this implementation, the QR code frame is still displayed independently as a single frame, and its display time does not overlap with the difference frame. Through camera front LCD shutter control synchronized with the screen timing, QR code frames can be accurately captured and identified, fully retaining the QR code positioning function of the basic solution.

[0099] The specific implementation steps of this embodiment are as follows:

[0100] Step S201: Image preprocessing

[0101] like Figure 5 As shown, the QR code used for positioning is binarized to obtain a QR code image containing only black (background) and white (foreground) pixels, which is called a QR code frame. Simultaneously, the original image to be displayed is subtracted from the QR code image at 1 / 3 brightness to obtain a difference image, called a difference frame.

[0102] Step S202: Frame loop display

[0103] like Figure 6 As shown, at a refresh rate of 240Hz, the images are displayed in a cyclical pattern of "QR code frame - difference frame - difference frame - difference frame", with four frames constituting one display cycle. Each frame lasts for 4.167ms (1 / 240s), and the total duration of a single display cycle is 16.66ms. The display periods of the QR code frames and difference frames do not overlap.

[0104] According to the above display method, relying on the flicker fusion effect of human vision, the human eye will form a brightness fusion effect between the 1 QR code frame and 3 difference frames within a single display cycle, and finally perceive the original image. This display method has an equivalent refresh rate of 60Hz, which will not cause the human eye to experience flickering or stuttering. The QR code image is also thus hidden from the human eye, such as... Figure 6 As shown. Simultaneously, the QR code frame is displayed as an independent single frame on the screen, with no overlap with the display period of the difference frame. By synchronizing the liquid crystal shutter lens attached to the front of the smartphone camera with the screen display timing, the lens is opened during the QR code frame display period and closed during the difference frame display period. This allows the phone camera to accurately capture the independent QR code image and filter out the difference frame image, ensuring the QR code image remains visible to the phone camera. Then, the positioning and pointing calculations are completed through QR code extraction and recognition.

[0105] Step S203: The following describes the specific implementation process of subtracting the original image from the QR code image at a refresh rate of 240Hz to generate a difference frame:

[0106] Step S2031: Channel Processing Rules

[0107] The subtraction operation is performed separately on the red (R), green (G), and blue (B) channels of the image. The values ​​of the R, G, and B channels are normalized to 0 to 1.

[0108] Step S2032: Subtraction operation process combined with gamma correction

[0109] The subtraction operation should be combined with the display device's gamma correction function: first, gamma correction is performed on the RGB channel values ​​of the corresponding pixels of the original image and the 1 / 3 brightness QR code image; then, the subtraction is performed, followed by inverse gamma correction, finally obtaining the difference frame for output and display. The specific implementation is as follows:

[0110] A QR code image is a binary, pure grayscale image, where the R, G, and B channels of its pixels take values ​​of _____. For the black pixels of the QR code For the white pixels of the QR code After gamma correction, the actual displayed brightness is: The original image is an RGB format color image, with the RGB values ​​of the pixels being... After gamma correction, the actual displayed brightness is γ is a fixed gamma value for the display device, which can be read from the display configuration file. At a 240Hz refresh rate, to achieve complementary brightness cancellation under flicker fusion, each difference frame only needs to cancel 1 / 3 of the brightness of the QR code image, meaning the actual display brightness of the difference frame is... Then, after inverse gamma transformation, the R, G, and B channel values ​​of the pixels in the difference frame are obtained. Taking the R channel as an example, the R channel values ​​of the pixels in the difference frame are as follows:

[0111] (6)

[0112] Similarly, the values ​​of the G and B channels of the pixels in the difference frame are as follows:

[0113] (7)

[0114] (8)

[0115] Based on formulas 6-8 above, we can analyze the situation where one QR code frame + three difference frames are displayed alternately at a refresh rate of 240Hz. The display duration of a single frame is 4.167ms, and the total duration of four frames is 16.66ms. The brightness perceived by the human eye is the average of the actual display brightness over the four frames.

[0116] (9)

[0117] When only the original images are displayed, without the QR code images, the brightness perceived by the human eye during the 16.66ms duration of displaying four frames is the time average of the brightness of the four original images, that is:

[0118] (10)

[0119] Compared to Formulas 9 and 10 above, when displaying the QR code image alternately with 1 QR code frame + 3 difference frames at a refresh rate of 240Hz, the QR code image is canceled out due to the flicker fusion effect of the human eye, and the brightness perceived by the human eye is 75% of the original image.

[0120] Step S2033: Measures to prevent underflow

[0121] To avoid underflow during image subtraction, the values ​​of the R, G, and B channels of white pixels in the binarized QR code image are... It should not be set to the maximum value of 1, but rather to a medium brightness value, for example... =0.3. At a 240Hz refresh rate, each frame difference only needs to cancel 1 / 3 of the QR code brightness, so in the original image, all frames where the minimum value of the R, G, and B channels is not lower than... The pixel regions that can avoid underflow when subtracted from the QR code image are marked as QR code embeddable areas (white) and non-embedded areas (black), resulting in a mask image. This mask image allows designers to avoid non-embedded areas when deploying QR codes. Figure 7 As shown. In the 120Hz refresh rate embodiment, only the minimum values ​​of the R, G, and B channels in the original image are not lower than... Only by defining the pixel area can underflow be avoided when subtracting from the QR code image. As can be seen from the comparison, the constraints are greatly relaxed, and more low-brightness areas in the original image can be included in the QR code embedding range. The range of non-embedded areas is thus reduced, effectively improving the flexibility of QR code deployment and image adaptability.

[0122] Step S2034: GPU-based real-time computation

[0123] In image subtraction operations, both gamma correction and inverse gamma correction involve power function operations. To ensure the real-time performance of image rendering and display, the operation can be accelerated through the graphics card GPU. The specific implementation method is the same as the basic implementation based on a 120Hz refresh rate.

[0124] The optimized embodiment based on a 240Hz refresh rate employs a display cycle of 1 QR code frame + 3 difference frames. The QR code brightness cancellation is evenly distributed across the 3 difference frames, and gamma correction is used to complete the difference frame calculation, effectively hiding the QR code from the human eye. This solution increases the perceived brightness of the image to 75% of the original image, significantly reducing brightness decay. Simultaneously, it relaxes the underflow constraint of the subtraction operation, significantly reducing the non-embedded area and improving the flexibility of QR code deployment. This embodiment retains the independent display characteristic of the QR code frame, and precise capture and recognition of the QR code for positioning can be achieved through synchronous control of the LCD shutter. Compared to the 120Hz embodiment, it achieves dual optimization in display effect and adaptability.

[0125] Matching LCD shutter lens module:

[0126] By using the aforementioned QR code embedding and hiding module, based on the flicker fusion effect of the human eye, the QR code image can be hidden from the human eye, meaning it is invisible. Furthermore, to ensure the phone camera can correctly capture the hidden QR code image, thus making the QR code visible to the smartphone, a liquid crystal shutter lens attached to the front of the phone camera is used to ensure the camera can correctly capture the hidden QR code image.

[0127] By using the aforementioned QR code embedding and hiding module, and leveraging the flicker fusion effect of the human eye, the QR code image can be hidden from the human eye, meaning it is invisible. However, to enable location-based interaction on a mobile phone, the phone's camera must be able to independently capture the hidden QR code image, meaning the QR code must be "visible" to the smartphone.

[0128] Therefore, an ideal solution is to directly implement frame synchronization between the display device and the smartphone camera. This involves extracting the frame synchronization pulse signal from the display driver or graphics card, and then sending the frame synchronization pulse signal to the smartphone via wireless broadcast communication. Each time the smartphone receives a frame synchronization pulse, it immediately activates the camera and sets the shutter time to the display duration of the QR code image (e.g., 8.333ms in the 120Hz refresh rate embodiment and 4.166ms in the 240Hz refresh rate embodiment). This ensures that the smartphone camera can independently capture and photograph the QR code image from the video stream displayed on the screen.

[0129] However, this ideal solution is difficult to achieve in actual engineering because it is constrained by multiple inherent limitations of mobile phone system characteristics, hardware control permissions, and wireless transmission, making it impossible to achieve precise matching between mobile phone shooting and screen display timing. Firstly, mobile phones running Android and iOS are non-real-time operating systems, lacking millisecond / microsecond-level precise task scheduling capabilities. The command to trigger the camera after receiving the frame synchronization pulse has unpredictable system scheduling delays, easily missing the brief display window of the QR code frame or causing overlap with the difference frame image. Secondly, ordinary mobile phone apps lack hardware control permissions for the camera shutter, making it impossible to precisely define the shutter opening time and millisecond-level precise duration. Shutter triggering is scheduled at the system level, resulting in uncontrollable execution deviations. Thirdly, random signal jitter in wireless broadcast transmission and the parsing delay on the mobile phone end cause a shift in the timing reference of the synchronization pulse, further amplifying the shooting timing error.

[0130] Therefore, this embodiment adopts a hardware implementation method with a matching liquid crystal shutter lens module. By using hardware-level timing synchronization control, the timing deviation and aliasing problems of the above-mentioned software control are solved, and accurate capture of QR code frames is achieved.

[0131] The core logic of this module is to extract the screen's frame synchronization pulse signal from the display driver or graphics card and broadcast it via a 2.4G wireless communication module. This transfers the core aspects of timing response and light control from the mobile phone to the hardware lens module itself. The lens module is equipped with a matching 2.4G wireless receiver module, which acquires the frame synchronization pulse signal in real time and transmits it to the microcontroller built into the module. This microcontroller is a hardware real-time control core with microsecond-level precise timing analysis and command output capabilities, completely circumventing the scheduling limitations of the mobile phone's non-real-time operating system.

[0132] The microcontroller, based on the received frame synchronization pulse signal, precisely matches the frame display timing of different refresh rates on the screen, and generates opening and closing control commands for the liquid crystal shutter lens, such as... Figure 8 As shown, in the 120Hz refresh rate embodiment, the control lens is turned on during the 8.333ms display period of the QR code frame and turned off during the same length display period of the difference frame, achieving precise switching with equal duty cycles; as Figure 9 As shown, in the 240Hz refresh rate embodiment, the control lens is turned on only during the display period of the QR code frame 4.166ms, and is turned off continuously during the display period of the 3 difference frames totaling 12.5ms, thereby achieving unequal duty cycle control of 1:3.

[0133] The liquid crystal shutter lens is hardware-mounted to the front of the mobile phone camera lens, allowing light to pass through the camera only when it is open and completely blocking light when it is closed. By completely shielding the light interference of the difference frame through hardware-level light filtering, the mobile phone camera does not need any software-level shutter timing control. It can capture clear, aliased independent QR code frame images simply by shooting normally, providing a stable and reliable image source for subsequent QR code extraction, recognition and positioning interactive calculations.

[0134] Meanwhile, the phone's own automatic exposure (AE) system can be adapted to the liquid crystal shutter lens module, becoming an important auxiliary link to ensure clear imaging of QR code frames. Its core function is to dynamically and stably adjust the exposure parameters according to the hardware light control characteristics of the lens module, so as to ensure that the phone camera achieves accurate exposure of QR code frames during the brief opening period of the lens.

[0135] Because the liquid crystal shutter lens only transmits light unidirectionally during the QR code frame display period and is completely blocked during the rest of the time, the phone's automatic exposure (AE) system monitors the effective light intake when the lens is open in real time, filtering out interference from dark light during the light-blocking phase, using the actual light intensity of the QR code frame as the metering reference; and automatically adjusts the electronic shutter duration for each frame capture so that the electronic shutter opening period includes the fixed opening period of the lens, adapted to 8.333ms in the 120Hz embodiment. Figure 10 As shown, in the 240Hz embodiment, the adaptation time is 4.166ms, as... Figure 11 As shown, this ensures that a complete exposure is completed within the limited effective light window, avoiding insufficient light or aliasing caused by mismatched shutter speeds.

[0136] When ambient light is low and the shutter speed alone is insufficient to meet the exposure requirements, the AE system will appropriately increase the ISO to supplement the brightness. At the same time, it will rely on its own smooth adjustment algorithm to control the ISO increase, reduce noise, and ensure the contrast and detail clarity of the QR code image. Furthermore, because the lens has shielded the drastic fluctuations in brightness between frames caused by the difference frames, the AE system does not need to frequently adjust parameters, resulting in a more stable exposure state and avoiding overexposure or underexposure of QR code frames due to sudden parameter changes.

[0137] In summary, the adaptive adjustment of the mobile phone's AE system enables the device to achieve accurate exposure of QR code frames under different ambient lighting conditions, such as low indoor light and strong outdoor light, ensuring the accuracy of subsequent QR code extraction and recognition. Moreover, no manual intervention is required throughout the process. It works seamlessly with the hardware control logic of the LCD shutter lens module to achieve seamless QR code frame shooting.

[0138] The mobile device points to the location processing module:

[0139] This module serves as the core data processing component of the entire positioning and interaction system. Its core objective is to process and recognize clear QR code frame images captured by the mobile phone camera, extract each QR code structure from the image, and use mature open-source QR code solving algorithms to parse the ID number of each QR code structure and its corresponding spatial pose transformation relationship. Finally, by combining the spatial pose transformation relationships of multiple QR code structures, the module optimizes the calculation of the corresponding mapping position of the mobile phone camera's optical center on the screen, thereby achieving precise pointing and positioning of the screen.

[0140] The specific technical implementation of this module consists of the following steps:

[0141] Step S401: Adaptive binarization processing of QR code image

[0142] QR code images captured by mobile phones are affected by factors such as ambient light intensity, lens transmittance, and screen brightness, resulting in uneven local brightness and slight noise. Direct recognition can easily lead to feature point loss and parsing errors. Therefore, it is necessary to perform binarization preprocessing on the original QR code image. This solution employs an adaptive binarization algorithm, which differs from the traditional single-threshold division method of global binarization. This algorithm dynamically calculates and adapts the threshold based on the local brightness characteristics of different regions of the image, accurately and efficiently distinguishing the white foreground and black background areas in the QR code image. It effectively eliminates the interference of local lighting deviations and slight noise on the QR code structure, completely extracting the QR code's contour, feature points, and coding structure. This provides a clear and well-organized image foundation for subsequent recognition operations, avoiding recognition failures caused by foreground-background confusion from the outset. The adaptive binarization algorithm can be implemented by calling corresponding functions in computer vision algorithm libraries such as OpenCV.

[0143] Step S402: QR code recognition and information extraction based on open-source algorithms

[0144] Based on the binarization preprocessing, this embodiment uses a mature open-source QR code recognition algorithm (such as AprilTag) to realize QR code recognition and information parsing. This type of algorithm has the characteristics of high recognition accuracy, strong real-time performance, and adaptability to the limited computing power of mobile devices, which fully meets the needs of real-time location interaction on mobile devices.

[0145] Specifically, the algorithm inputs the QR code image after adaptive binarization into the open-source recognition algorithm. The algorithm sequentially performs QR code feature point detection, encoding structure matching, and data parsing. On the one hand, it can accurately extract the unique ID number of each QR code, enabling precise identification of QR codes at different locations on the screen. On the other hand, it can calculate the complete coordinate system transformation relationship (including rotation matrix and translation vector) between the world coordinate system and the mobile phone camera coordinate system, establishing a spatial relationship between the world coordinates on the screen and the camera coordinates on the mobile phone, providing a core coordinate transformation basis for subsequent spatial position calculation of the camera's optical center.

[0146] Step S403: Directional positioning calculation based on QR code recognition results

[0147] First, let's explain the calculation principle of pointing positioning. The principle of pointing positioning in this embodiment is essentially to reasonably transplant and adapt the mature "planar marker pose estimation" technology in computer vision to the pointing positioning application scenario of "large screen + mobile phone". By visually embedding and hiding the QR code image, it is made invisible to the human eye, ensuring that the QR code image as a planar marker has no impact on the viewing effect. At the same time, the liquid crystal shutter lens module ensures that the mobile phone can capture the hidden QR code image.

[0148] To enable users to point and locate themselves when scanning the screen with their handheld mobile phones, the world coordinate system is defined as follows: the lower left corner of the screen is taken as the origin of the world coordinate system, the X-axis is the horizontal direction of the screen, the Y-axis is the vertical direction of the screen, and the Z-axis is perpendicular to the screen plane and points outward.

[0149] This invention utilizes the flicker fusion effect of the human eye to embed a binarized QR code image into the screen for display. The QR code image is invisible to the human eye, but on the mobile phone, a liquid crystal shutter lens attached to the front of the phone's camera ensures that the camera can correctly capture the hidden QR code image, achieving the effect of making the QR code "visible" to the smartphone. Subsequently, the captured QR code image is extracted and recognized on the mobile phone, and the spatial transformation relationship between the phone's camera coordinate system and the world coordinate system is obtained. This determines the mapped position of the phone's camera's optical center on the screen, and a cursor icon is displayed at this mapped position. When the user holds the phone to scan the screen, the movement trajectory of the cursor displayed on the screen allows them to determine the phone's pointing position on the screen, serving as the basis for interactive operations such as target selection.

[0150] The aforementioned QR codes can employ various positioning icon patterns commonly used in the field of computer vision, such as AprilTag, ArUco, and ARTag. When a mobile phone captures such a QR code, the corresponding open-source code can be used to identify the QR code's ID number and calculate the transformation relationship between the QR code coordinate system and the mobile phone camera coordinate system. Here, the QR code coordinate system refers to a spatial rectangular coordinate system with the center of the QR code as the origin, and the horizontal and vertical axes of the QR code structure forming the XY plane. Since the pixel coordinates of the QR code center on the screen and the physical spacing of the screen pixels are known, and the X and Y axes of the QR code are parallel to the X and Y axes of the world coordinate system, there is a translation relationship between the QR code coordinate system and the world coordinate system. Therefore, based on the calculated transformation relationship between the mobile phone camera coordinate system and the QR code coordinate system, the transformation relationship between the mobile phone camera coordinate system and the world coordinate system can be further obtained, thereby calculating the mapped position of the mobile phone camera's optical center on the screen.

[0151] In large-screen display environments, multiple QR code structures need to be embedded in the screen image to ensure that at least one complete QR code structure can be captured when the user is walking freely and taking a picture of the screen. The size and spacing of the multiple QR code structures can be specifically designed based on the actual physical size of the screen and the user's typical viewing distance in front of the screen. When the image captured by the mobile phone contains more than one complete QR code structure, a candidate mapping position of the mobile phone camera's optical center on the screen can be calculated based on each QR code structure. Then, interpolation calculations are performed on the obtained multiple candidate mapping positions to finally obtain the precise mapping coordinates of the mobile phone camera's optical center on the screen.

[0152] Finally, it also includes a human-computer interaction module, used to detect the user's finger touch operation on the smartphone screen, obtain the corresponding touch coordinates, and then realize the selection and playback control of the content displayed on the large public display screen; when the interactive icon moves on the screen and a user long-presses the phone screen is detected, the movement trajectory can be drawn on the screen to achieve a trajectory retention effect similar to electronic graffiti. In addition, the human-computer interaction module supports multi-point collaborative interaction function, which can simultaneously recognize the touch commands of multiple smartphones, and realize the parallel selection, playback and other interactive operations of different images or video content by multiple users.

[0153] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can be implemented in one or more computer-usable storage media (including, but not limited to, disk storage, etc.) containing computer-usable program code. CD - ROM It takes the form of a computer program product implemented on (such as optical memory, etc.).

[0154] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A directional interaction method based on hidden embedding of QR codes, characterized in that, Includes the following steps: Several QR code images are hidden and embedded in the display of the target screen. The QR code images utilize the human eye flicker fusion effect and are displayed on the target screen in alternating frames according to a preset refresh rate, so that the QR code images are invisible to the human eye. The process of hiding and embedding several QR code images in the display of the target screen includes: subtracting the original image to be displayed from the QR code image with matching brightness to obtain a difference frame image that matches the screen device's refresh rate; and alternating the display of the QR code image and the difference frame image on the target screen according to the screen refresh rate. When the screen refresh rate is 240Hz, the original image to be displayed is subtracted from the QR code image at 1 / 3 brightness, and then displayed alternately in a group of one QR code image and three difference frame images. The process of subtracting the original image to be displayed from the QR code image to obtain a difference frame image that matches the screen refresh rate includes: performing subtraction operations on the red, green, and blue channels of the image respectively, normalizing the values ​​of each channel to a set range, and when performing subtraction operations on each channel, first performing gamma correction on the values ​​of their respective red, green, and blue channels before performing subtraction, then performing inverse gamma correction, and finally obtaining the correct difference value and outputting it for display. During the subtraction operation on the red, green, and blue channels of the image, underflow detection is performed on the values ​​of the red, green, and blue channels of each pixel in the original image. Pixel areas where the minimum value of the red, green, and blue channels is lower than a set threshold are set to black and marked as non-embedding areas, while those with a minimum value are set to white and marked as embedding areas. A mask image of the original image is generated. When deploying the embedded QR code, the non-embedding areas marked by the mask image are avoided to ensure that the embedding position of the QR code is located in the embedding area. In response to scanning and capturing images with a smartphone camera, the system extracts and recognizes pre-hidden embedded QR code images in the display of the target screen. Based on the extracted and recognized QR code image information, the coordinates of the smartphone camera's optical center on the screen are calculated, the corresponding pointing position of the user on the screen is obtained, and a cursor pattern that moves with the smartphone is displayed at the corresponding coordinate position, thus realizing the smartphone's pointing positioning on the display screen.

2. The pointing interaction method based on hidden embedding of QR codes as described in claim 1, characterized in that, The QR code is a QR code. Before being hidden and embedded, it is pre-binarized to obtain a QR code image containing only black and white pixels.

3. The pointing interaction method based on hidden embedding of QR codes as described in claim 1, characterized in that, When the screen refresh rate is 120Hz, the original image to be displayed is subtracted from the QR code image, and then displayed alternately in a group of one QR code image and one difference frame image.

4. The pointing interaction method based on hidden embedding of QR codes as described in claim 1, characterized in that, The process of calculating the coordinates of the smartphone camera's optical center on the screen based on the extracted and identified QR code image information includes: adaptive binarization processing of the QR code image captured by the phone to remove stray light and noise interference to the QR code structure; identification and decoding of each complete QR code structure in the image to obtain the unique ID code information corresponding to each QR code structure, as well as its spatial pose transformation data relative to the smartphone's camera coordinate system; calculation of a set of candidate projection points of the smartphone's camera optical center on the screen plane; interpolation calculation of the candidate projection points; and finally obtaining the precise coordinates of the smartphone's optical center mapped on the screen.

5. A directional interaction system based on hidden QR code embedding, employing the directional interaction method based on hidden QR code embedding as described in any one of claims 1-4, characterized in that, include: The QR code image embedding display and hiding module is configured to hide and embed several QR code images in the display screen of the target screen. The QR code images utilize the human eye flicker fusion effect and are displayed on the target screen in alternating frames according to a preset refresh rate, so that the QR code images are invisible to the human eye. The image extraction and recognition module is configured to extract and recognize a pre-hidden embedded QR code image in response to scanning and capturing by a smartphone camera; The pointing and positioning calculation module is configured to calculate the coordinate position of the smartphone camera's optical center on the screen based on the extracted and recognized corresponding QR code image information, obtain the corresponding pointing position of the user on the screen, and display a cursor pattern that moves with the smartphone at the corresponding coordinate position, thereby realizing the pointing and positioning of the smartphone on the display screen.

6. The directional interaction system based on hidden QR code embedding as described in claim 5, characterized in that, The smartphone has a liquid crystal shutter lens module in front of its camera to acquire screen frame synchronization pulse signals and adapt the lens opening and closing sequence according to the screen refresh rate, so that the lens only transmits light in one direction during the opening period, allowing only the QR code frame image to enter the camera lens, while completely blocking light during the rest of the time.

Images