Screen-camera communication method for resisting light interference and imaging distortion

Abstract

The invention discloses a screen-camera communication method for resisting light interference and imaging distortion. A system involved in the method comprises a transmitting terminal screen, a transmitting terminal positioning sensor and a receiving terminal camera. The method can obtain the relative position of the camera and the screen and the position of the interference light source through the positioning sensor. Under the condition of light source interference, an illumination distortion model and a position transformation model are constructed. Through pre-transformation of a coding pattern on a screen, the coding pattern captured by a camera can avoid illumination interference, and angle distortion is compensated. According to the method, ambient light interference can be resisted, distortion of a camera-screen channel is resisted, the communication robustness and reliability in a complex environment are remarkably improved, and the method is suitable for the screen shooting communication requirement in the environment with complex illumination interference and inclination angles.

Description

Technical Field

[0001] The invention relates to a screen-camera communication method for resisting light interference and imaging distortion, and belongs to the intersection field of optical communication, optical positioning and computer vision. Background Art

[0002] Visible light communication technology is an emerging wireless communication method. Its principle is to use light sources such as light-emitting diodes to transmit information. It can communicate in any occasion with sufficient illumination. Nowadays, due to the widespread use of various displays and mobile phone cameras, screen camera communication has gained wide attention. As a type of visible light communication, it is a new communication method developed to address the limitations of traditional communication technology.

[0003] Screen camera communication uses low-resolution displays or LED arrays to transmit information by changing the brightness and color of each pixel. The receiving end extracts information from the captured pictures or videos through image processing technology and decodes them. Screen camera communication has the advantages of simple structure, fast transmission speed, and low implementation cost. In addition, visible light communication has sufficient unlicensed spectrum resources, especially screen camera communication, which has good communication performance in close-range and unobstructed scenarios, and will not encounter problems such as poor network or signal leakage. It is completely harmless to the human body, so it has rich application scenarios and broad exploration space.

[0004] The existing screen-camera communication system has obvious shortcomings under complex lighting conditions and shooting angles. Angle distortion and light distortion will cause the shooting effect of the coded pattern to be impaired, especially when the light source interference is serious, the information on the screen may be lost due to excessive light or uneven reflection. The geometric distortion caused by the shooting angle will also affect the recognition and decoding of the coded pattern. When there are too many distortions and distortions, the possibility of decoding failure is greatly increased, which seriously affects the reliability of communication. In order to overcome these problems, existing technologies often rely on complex post-processing algorithms and ways to increase coding redundancy, which cannot fundamentally solve the problem. Therefore, there are still many bottlenecks in the existing technologies in dealing with robustness, decoding reliability and cost-effectiveness in complex environments. Summary of the invention

[0005] The purpose of the present invention is to provide a screen-camera communication method that can combat light interference and imaging distortion in order to address the problems of angle distortion and illumination distortion existing in existing screen-camera communication systems. By pre-transforming the coding pattern on the screen, the coding pattern captured by the camera can avoid illumination interference and compensate for angle distortion.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is: Step 1: Input screen; Step 2: Measure the screen size , ; Calculate the physical size of the screen pixels , ; Use positioning sensors to detect the position of the camera 、Light source i The position of the corner points ; Step 3: Calculate the area of ​​brightness distortion on the screen based on the detection data in step 2; Step 4: Calculate the perspective transformation relationship based on the detection data in step 2 to determine the angle distortion; Step 5: Simulate the distorted image captured by the camera based on the angle distortion and brightness distortion ; Step 6: Simulating the Distortion Pattern On the original coding pattern Pan and zoom to get distortion-free images ,Will Perform inverse transformation to obtain pre-transformed screen pattern ; Step 7: Use your camera to capture the pre-transformed screen pattern on the screen , get the undistorted captured pattern .

[0007] Preferably, the brightness distortion area described in step 3 is: .

[0008] Preferably, the perspective transformation relationship in step 4 is: ,in, Is the screen pattern The pixel coordinates of It is a distorted image taken by a simulated camera The pixel coordinates, K is the camera internal parameter, R is the camera rotation matrix, is the scaling factor, , , is an intermediate variable.

[0009] Preferably, the pre-transformation relationship in step 6 is: ,in, , , represents the displacement, s represents the scale, and represents the translation and scaling in step 6. Represents the pre-transformed screen pixels, is the scaling factor, , , is an intermediate variable.

[0010] Preferably, the calculation step of the camera rotation matrix R includes: adding constraints to calculate the camera angle, the constraints are that the camera center is aligned with the screen center, and the camera rotation around the z axis is 0, and the camera rotation angle around the x, y, and z axes is , , It is expressed as: .

[0011] Preferably, , where the rotation angle , , Provided by other methods, such as the mobile phone's inertial measurement unit IMU.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses a positioning sensor to capture the relative position of the camera and the screen, as well as the position of the interfering light source, to construct a light distortion model and a position transformation model. By pre-transforming the coded pattern on the screen, the coded pattern captured by the camera can avoid light interference and compensate for angle distortion. This method can resist ambient light interference and distortion of the camera-screen channel, significantly improving the robustness and reliability of communication in complex environments, and is suitable for screen-shot communication needs in environments with complex light interference and tilt angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a block diagram of a screen-camera communication method for combating light interference and imaging distortion provided by an embodiment of the present invention; Figure 2 is the original coding pattern provided by the embodiment of the present invention ; Figure 3 The photographic image with illumination distortion and angle distortion provided by the embodiment of the present invention is ; Figure 4 The screen display image after using the pre-conversion method provided by the embodiment of the present invention is ; Figure 5 The image captured using the pre-transformation method provided in the embodiment of the present invention is . DETAILED DESCRIPTION

[0014] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0015] A screen-camera communication method that resists light interference and imaging distortion. Figure 1 ,In the figure, a universal display screen and a camera are set up, there is an interfering light source in the environment, and a pattern (taking a QR code as an example) is displayed on the screen. Figure 2 The original image provided by the example of the present invention is shown , Figure 3 It's shooting Figure 2 The resulting distorted image , Figure 4 is the pre-transformed screen display image , Figure 5 It's shooting Figure 4 The resulting undistorted image .

[0016] Measuring screen size , ; Calculate the physical size of the screen pixels , , using the positioning sensor to detect the position of the camera , the position of the i-th corner point of the light source ; Take the upper left corner of the display as the origin of the space coordinate system, and the plane where the screen is located Z=0 to establish the screen world coordinate system , the spatial coordinates of screen pixels are denoted by .

[0017] The illumination distortion model is established according to the position of the screen and the camera. The illumination distortion area is: .

[0018] The constraints are: the center of the camera is aligned with the center of the screen, and the rotation of the camera around the z axis is 0. The rotation angle of the camera around the x, y, and z axes , , It can be expressed as:

[0019] So the camera's rotation matrix R : , the camera’s intrinsic parameter matrix K can be obtained through camera calibration.

[0020] From the screen pattern Pixels on To simulate the distorted image captured The pixel coordinates The perspective transformation relationship is:

[0021] in, is the scaling factor.

[0022] Through the above model, simulate the picture captured by the image sensor ,get Figure 3 In order to obtain the undistorted Figure 5 , calculate the scaling and displacement matrix of the screen pattern and record it as

[0023] in , represents the displacement, and s represents the scale. Figure 5 The undistorted coding pattern placed in a simulated shot is shown.

[0024] So we can get the pixels from the screen to pre-transformed screen pixels Pre-transformation model ,in, is the scaling factor.

[0025] According to the inverse transformation model, the simulated distortion-free coding information after shooting is transformed to the screen image, ensuring that the captured coding information has no angle distortion and is not affected by light distortion. The final screen display is shown in Figure 4 , you can ensure that the captured image is Figure 5 , so that the captured image is not affected by lighting distortion and angle distortion.

[0026] The following experiment is used to verify the effect of the present invention: the experimental device includes a laptop, a camera and a light source. The coding types tested in the experiment include QR code, Aztec code, data matrix code and artistic QR code. The experiment was carried out under two conditions, normal condition and in the presence of light interference. In the experiment, a three-dimensional space model with the center of the screen as the coordinate origin was constructed. The light source was placed 1 meter above the screen. The camera was placed on a hemispherical surface with a radius of 0.5 meters and the origin as the center. There are 100 camera positions evenly distributed on the hemispherical surface, and the decoding performance is judged according to the number of positions successfully decoded. The purpose of the experiment is to achieve a comprehensive analysis of the screen display content by capturing screen images from these different perspectives.

[0027] Experimental results show that illumination distortion reduces the decoding success rate of the original image by 2-5 positions, but has almost no effect on the pre-transformed image. The pre-transformed algorithm consistently outperforms the original image, achieving 20-50 additional successful decodings in all tested coding types. In summary, the proposed pre-transformed algorithm significantly improves decoding performance and is highly resilient to distortion. These results validate the practical benefits of the proposed algorithm in improving decoding reliability for a variety of coding types, especially under challenging conditions.

Claims

1. A screen-camera communication method for resisting light interference and imaging distortion, characterized in that: The steps include: Step 1: Convert the original coding pattern Input screen; Step 2: Measure the screen size , ; Calculate the physical size of the screen in pixels , ; Use positioning sensors to detect the position of the camera 、Light source i The position of the corner points ; Step 3: Calculate the area of ​​brightness distortion on the screen based on the detection data in step 2; Step 4: Calculate the perspective transformation relationship based on the detection data in step 2 to determine the angle distortion; Step 5: Simulate the distorted image captured by the camera based on the angle distortion and brightness distortion ; Step 6: Simulating the Distortion Pattern On the original coding pattern Pan and zoom to get distortion-free images ,Will Perform inverse transformation to obtain pre-transformed screen pattern ; Step 6: Use your camera to capture the pre-transformed screen pattern on the screen , get the undistorted captured pattern .

2. A screen-camera communication method for resisting light interference and imaging distortion according to claim 1, characterized in that: The brightness distortion area described in step 3 is: .

3. A screen-camera communication method for resisting light interference and imaging distortion according to claim 1, characterized in that: The perspective transformation relationship in step 4 is: ,in, Is the screen pattern The pixel coordinates of It is a distorted image taken by a simulated camera The pixel coordinates, K is the camera internal parameter, R is the camera rotation matrix, is the scaling factor, , , is an intermediate variable.

4. The screen-camera communication method for resisting light interference and imaging distortion according to claim 1, characterized in that: The pre-transformation relationship in step 6 is: ,in, , , represents the displacement, s represents the scale, Represents the pre-transformed screen pixels, is the scaling factor, , , is an intermediate variable.

5. The screen-camera communication method for resisting light interference and imaging distortion according to claim 3, characterized in that: The calculation steps of the camera rotation matrix R include: adding constraints to calculate the camera angle, the constraints are that the camera center is aligned with the center of the screen, and the camera rotation around the z axis is 0, the camera rotation angle around the x, y, and z axes , , It is expressed as: 。 6. The screen-camera communication method for resisting light interference and imaging distortion according to claim 3, characterized in that: , where the rotation angle , , Measured directly by the sensor.

Images