A screen-camera communication method against light interference and imaging distortion

Abstract

The application discloses a screen-camera communication method for resisting light interference and imaging distortion, and relates to a system comprising a transmitting end screen, a transmitting end positioning sensor and a receiving end camera. The method can obtain the relative position of the camera and the screen and the position of the interference light source through the positioning sensor. Under the condition of light source interference, a light distortion model and a position transformation model are constructed. The coded pattern on the screen is pre-transformed, so that the coded pattern captured by the camera can avoid light interference and compensate for angle distortion. The method can resist environmental light interference and resist distortion of the camera-screen channel, significantly improves the communication robustness and reliability in a complex environment, and is suitable for screen-camera communication requirements in an environment with complex light interference and an inclined angle.

Description

TECHNICAL FIELD

[0001] The present application relates to a screen-camera communication method against light interference and imaging distortion, belonging to the cross field of optical communication, optical positioning and computer vision. BACKGROUND

[0002] Visible light communication technology is a new wireless communication method, which uses light sources such as light-emitting diodes to transmit information, and can communicate in any environment with sufficient light intensity. Nowadays, due to the wide use of various displays and mobile phone cameras, screen camera communication has attracted widespread attention, and as a kind of visible light communication, it is a new communication method developed to overcome the limitations of traditional communication technology.

[0003] Screen camera communication uses low-resolution displays or LED arrays to transmit information by changing the brightness and color of each pixel. The receiving end extracts information from the captured pictures or videos through image processing technology and decodes it. Screen camera communication has the advantages of simple structure, fast transmission speed, and low implementation cost. In addition, visible light communication has sufficient unlicensed spectrum resources, especially screen camera communication, which has good communication performance in close-range, unobstructed scenarios, and does not suffer from network problems or signal leakage. It is completely harmless to the human body, so it has rich application scenarios and broad exploration space.

[0004] The existing screen-camera communication system has obvious shortcomings under complex lighting conditions and shooting angles. Angle distortion and light distortion can cause the shooting effect of the encoded pattern to be damaged, especially when the light source interference is serious, the information on the screen may be lost due to excessive or uneven reflection of light. The geometric distortion caused by the shooting angle also affects the recognition and decoding of the encoded pattern. When distortion and distortion are too much, the possibility of decoding failure increases greatly, which seriously affects the reliability of communication. In order to overcome these problems, existing technologies often rely on complex post-processing algorithms and increase the encoding redundancy, which cannot fundamentally solve the problem. Therefore, the existing technology still has many bottlenecks in terms of robustness, decoding reliability and cost-effectiveness in complex environments. SUMMARY

[0005] The purpose of the present application is to solve the problems of angle distortion and light distortion in the existing screen camera communication system, and to provide a screen-camera communication method against light interference and imaging distortion. By pre-transforming the encoded pattern on the screen, the encoded pattern captured by the camera can avoid light interference and compensate for angle distortion.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is as follows: Step 1: the original encoded pattern is input into the screen;

[0007] Step 2: Measure the screen size 、 ; Calculate the physical size of the screen pixels 、 ; Detect the position of the camera using the positioning sensor , the position of the light source i corner point ;

[0008] Step 3: Calculate the area of luminance distortion on the screen according to the detection data in Step 2;

[0009] Step 4: Calculate the perspective transformation relationship according to the detection data in Step 2 to determine the angle distortion;

[0010] Step 5: Simulate the distorted image captured by the camera according to the angle distortion and luminance distortion ;

[0011] Step 6: On the simulated distorted pattern , translate and scale the original encoded pattern to obtain the non-distorted captured pattern , and perform inverse transformation to obtain the pre-transformed screen pattern ;

[0012] Step 7: Capture the pre-transformed screen pattern on the screen using the camera to obtain the non-distorted captured pattern .

[0013] Preferably, the luminance distortion area in Step 3 is: .

[0014] Preferably, the perspective transformation relationship in Step 4 is: wherein is the pixel coordinate of the screen pattern , is the pixel coordinate of the simulated distorted image captured by the camera , K is the camera intrinsic parameter, R is the camera rotation matrix, is the scale transformation factor, , , , is an intermediate variable.

[0015] Preferably, the pre-transformed relationship in Step 6 is: wherein , , represents the displacement amount, s represents the scale, and represents the translation and scaling in Step 6 Represents the pre-transformed screen pixels, is the scaling factor, 、 、 is an intermediate variable.

[0016] Preferably, the steps of calculating the camera rotation matrix R include: adding constraints to calculate the camera angle, the constraints are that the camera center is aligned with the screen center, and the camera rotation around the z axis is 0, the camera rotation angle around the x, y, and z axes is 、 、 Expressed as:

[0017] .

[0018] Preferably, , where the rotation angle 、 、 Provided by other methods, such as the mobile phone's inertial measurement unit IMU.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention uses a positioning sensor to capture the relative position of the camera and screen, as well as the position of the interfering light source, to construct a light distortion model and a position transformation model. By pre-transforming the coded pattern on the screen, the coded pattern captured by the camera can avoid light interference and compensate for angular distortion. This method can combat ambient light interference and distortion in the camera-screen channel, significantly improving the robustness and reliability of communications in complex environments. It is suitable for screen-capture communication needs in environments with complex light interference and tilt angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a block diagram of a screen-camera communication method for combating light interference and imaging distortion provided by an embodiment of the present invention;

[0022] Figure 2 The original coding pattern provided by the embodiment of the present invention ;

[0023] Figure 3 This is a photographic image with illumination distortion and angle distortion provided by an embodiment of the present invention. ;

[0024] Figure 4 This is the screen display image after using the pre-conversion method provided by the embodiment of the present invention ;

[0025] Figure 5 The image captured using the pre-transformation method provided by the embodiment of the present invention is . DETAILED DESCRIPTION

[0026] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0027] A screen-camera communication method that resists light interference and imaging distortion. Figure 1 ,In the figure, a universal display screen and a camera are set up, there is an interfering light source in the environment, and a pattern (taking a QR code as an example) is displayed on the screen. Figure 2 The original image provided by the example of the present invention is shown , Figure 3 It's shooting Figure 2 The resulting distorted image , Figure 4 is the pre-transformed screen display image , Figure 5 It's shooting Figure 4 The resulting undistorted image .

[0028] Measuring screen size 、 ; Calculate the physical size of the screen pixels 、 , using positioning sensors to detect the position of the camera , the position of the i-th corner point of the light source ; Take the upper left corner of the display as the origin of the space coordinate system, and the screen plane Z=0 to establish the screen world coordinate system , the spatial coordinates of screen pixels are denoted as .

[0029] A lighting distortion model is established based on the positions of the screen and camera. The area of ​​lighting distortion is: .

[0030] The constraints are: the camera center is aligned with the screen center, and the camera rotation around the z axis is 0. The camera rotation angle around the x, y, and z axes 、 、 It can be expressed as:

[0031] So the camera's rotation matrix R : , the camera's intrinsic parameter matrix K can be obtained through camera calibration.

[0032] From the screen pattern Pixels on To simulate the distorted image captured Pixel coordinates The perspective transformation relationship is:

[0033] in, is the scaling factor.

[0034] Through the above model, simulate the picture captured by the image sensor ,get Figure 3 In order to obtain undistorted Figure 5 , calculate the scaling and displacement matrix of the screen pattern as

[0035] in 、 represents the displacement, and s represents the scale. Figure 5 The undistorted coding pattern is shown placed in a simulated photograph.

[0036] So you can get the pixels from the screen to pre-transformed screen pixels Pre-transformation model ,in, is the scaling factor.

[0037] According to the inverse transformation model, the simulated undistorted coding information after shooting is transformed into the screen image, ensuring that the captured coding information has no angle distortion and is not affected by light distortion. The final screen display is shown in Figure 4 , you can ensure that the captured image is Figure 5 , so that the shooting picture is not affected by lighting distortion and angle distortion.

[0038] The following experiment was used to verify the effect of the present invention: the experimental device includes a laptop computer, a camera and a light source. The coding types tested in the experiment include QR code, Aztec code, data matrix code and art QR code. The experiment was carried out under two conditions, normal condition and in the presence of light interference. In the experiment, a three-dimensional space model with the center of the screen as the coordinate origin was constructed. The light source was placed 1 meter above the screen. The camera was placed on a hemisphere with a radius of 0.5 meters and the origin as the center. There are 100 camera positions evenly distributed on the hemisphere, and the decoding performance is judged according to the number of positions successfully decoded. The purpose of the experiment is to achieve a comprehensive analysis of the content displayed on the screen by capturing screen images from these different perspectives.

[0039] Experimental results show that illumination distortion reduces the decoding success rate of original images by 2–5 positions but has little effect on pre-transformed images. The pre-transformed algorithm consistently outperforms the original images, achieving 20–50 additional successful decodes across all tested codecs. In summary, the proposed pre-transformed algorithm significantly improves decoding performance and is robust to distortion. These results validate the practical benefits of the proposed algorithm in improving decoding reliability across various codecs, particularly under challenging conditions.

Claims

1. A screen-camera communication method for resisting light interference and imaging distortion, characterized in that: The steps include: Step 1: Convert the original coding pattern Q s Input screen; Step 2: Measure the screen size U0, V0; calculate the physical size of the screen pixel Δu, Δv; use the positioning sensor to detect the position of the camera (x0, y0, z0), the position of the i-th corner point of the light source (X Li , Y Li , Z Li ); Step 3: Based on the detection data from step 2, calculate the area of ​​brightness distortion on the screen, expressed as: Step 4: Calculate the perspective transformation relationship based on the detection data in step 2 to determine the angle distortion. The transformation relationship is: Among them, (u s , v s ) is the screen pattern Q s The pixel coordinates, (u c , v c ) is the distorted image Q taken by the simulated camera c The pixel coordinates, K is the camera internal parameter, R is the camera rotation matrix, z c is the scaling factor, U s 、V s 、W s is an intermediate variable; Step 5: Simulate the distorted image Q captured by the camera based on the angle distortion and brightness distortion c ; Step 6: Simulating the Distortion Pattern Q c On the original coding pattern Q s Perform translation and scaling to obtain the undistorted photographic pattern Q′ c , Q′ c Perform inverse transformation to obtain the pre-transformed screen pattern Q p , the pre-transformation relationship is expressed as: in, t1 and t2 represent the displacement, s represents the scale, and represents the translation and scaling in step 5. (u p , v p ) represents the pre-transformed screen pixel, h′ 33 is the scaling factor, U p 、V p 、W p is an intermediate variable; Step 7: Use your camera to capture the pre-transformed screen pattern Q on the screen p , get the undistorted photographic pattern Q′ c .

2. The screen-camera communication method for resisting light interference and imaging distortion according to claim 1, characterized in that: The calculation formula of the camera rotation matrix R is: Among them, α x , α y , α z To add constraints to calculate the camera angle, the constraints are that the camera center is aligned with the center of the screen and the camera rotation around the z axis is 0. The camera rotation angle around the x, y, and z axes is calculated as follows:

3. The screen-camera communication method for resisting light interference and imaging distortion according to claim 1, characterized in that: The camera rotation matrix is ​​calculated as: Among them, α x , α y , α z Measured directly by the sensor.

Images