Naked eye 3D display surround view method and system based on human eye tracking

Abstract

The invention discloses a naked eye 3D display surround view method and system based on human eye tracking, and the method comprises the steps: S1, obtaining a to-be-displayed 3D image, and dividing the to-be-displayed 3D image into a left eye view and a right eye view which are respectively used for the display of a left viewport and a right viewport; s2, collecting human eye data of a user through a sensor on the display equipment; s3, according to the distance between the left eye and the display device and the distance between the right eye and the display device in the human eye data of the user obtained in the step S2, the scaling ratio of the left eye view and the right eye view is calculated, and the left eye view and the right eye view obtained in the step S1 are scaled according to the scaling ratio; and S4, calculating translation amounts of left and right eye views according to the positions of the left and right eyes in the human eye data of the user obtained in the step S2, and respectively translating the left and right eye views obtained in the step S1 according to the translation amounts of the left and right eye views. According to the invention, zooming and translation of the left eye view and the right eye view can be respectively controlled according to the viewing position and the viewing angle of the user, the size and the display angle of the naked eye 3D picture can be changed in real time, and the viewing angle range is expanded.

Description

Technical Field

[0001] The present invention relates to the field of naked-eye 3D display technology, and in particular to a naked-eye 3D display panoramic viewing method and system based on human eye tracking. Background Art

[0002] In the prior art of naked-eye 3D display, although a stereoscopic visual effect can be provided, there are still deficiencies in user interaction and viewing angle adjustment. Only views of a fixed size and fixed angle can be viewed. This not only causes the distortion of the display screen to be unable to be eliminated, but also makes it impossible to achieve directional and fixed-point observation of local content.

[0003] Although a wide-angle camera can be used, and the images transmitted by it can display 3D images on a naked-eye 3D display, due to the limitations of the wide-angle camera itself, there is a distortion phenomenon, and there are still problems such as a fixed picture size, a fixed image display direction, and fixed displayed content, making it impossible to view the whole picture or magnify a certain detail content for viewing. Summary of the Invention

[0004] Object of the Invention: Aiming at the above deficiencies, the present invention proposes a naked-eye 3D display panoramic viewing method and system based on human eye tracking, which can not only eliminate the distortion phenomenon, but also adjust the display content and display angle of the 3D picture in real time according to the change of the position of the user's human eyes.

[0005] Technical Solution:

[0006] The present invention provides a naked-eye 3D display panoramic viewing method based on human eye tracking, including:

[0007] S1. Obtain the 3D picture to be displayed, and divide it into left and right eye views respectively for display in the left and right viewports;

[0008] S2. Collect the human eye data of the user through the sensors on the display device;

[0009] S3. Calculate the zoom ratios of the left and right eye views according to the distances between the left and right eyes and the display device in the human eye data obtained in S2, and zoom the left and right eye views obtained in S1 accordingly;

[0010] S4. Calculate the translation amounts of the left and right eye views according to the positions of the left and right eyes in the human eye data obtained in S2, and translate the left and right eye views obtained in S1 respectively accordingly.

[0011] Specifically, in S3, the calculation of the zoom ratios of the left and right eye views is as follows:

[0012] Given the minimum value Z of the distance between the left / right eye and the display device min and the maximum value Z max, the distance between the left / right eye and the display device is Z, and the minimum value F of the scaling ratio is set min and the maximum value F max ;

[0013] When the distance between the left / right eye and the display device is a fixed value Z s at this time, set the scaling ratio F = 1;

[0014] Then:

[0015] When the distance Z between the left / right eye and the display device is less than the fixed value, the calculation of the scaling ratio is as follows:

[0016] F′ = F + (Z s - Z) / (Z s - Z min ) × (F max - F);

[0017] When the distance Z between the left / right eye and the display device is greater than the fixed value, the calculation of the scaling ratio is as follows:

[0018] F′ = F - (Z - Z s ) / (Z max - Z s ) × (F - F min ).

[0019] Specifically, in S3, before calculating the scaling ratio, it further includes the step of filtering the distance between the left and right eyes in the user's eye data and the display device, specifically as follows:

[0020] (1) Use exponential weighted moving average filtering to filter the distance between the left / right eye and the display device in the user's eye data;

[0021] (2) Perform hysteresis filtering on the filtered distance between the left / right eye and the display device obtained in step (1).

[0022] More specifically, the filtering exponent of the exponential weighted moving average filtering is taken as 0.1 - 0.4, and the hysteresis threshold of the hysteresis filtering is taken as 10 - 100 mm.

[0023] Specifically, after calculating the scaling ratio, damping filtering can also be performed on the calculated scaling ratio.

[0024] Specifically, in S4, the calculation of the translation amount of the left and right eye views is as follows:

[0025] It is known that the size of the target display window is w × h, where h is the height of the target display window and w is the width of the target display window;

[0026] Based on the positions of the left and right eyes in the human eye data obtained from S2, the x - coordinate value x0 and y - coordinate value y0 of the left / right eye in the plane coordinate system corresponding to the target display window are obtained. Then, the offset values x1 and y1 of the left / right eye views in the x - direction and y - direction are calculated as follows:

[0027] x1 = x0 / (w / k1);

[0028] y1 = y0 / (h / k2);

[0029] Where k1 and k2 are respectively the set offset coefficients in the x - direction and y - direction.

[0030] Specifically, in S4, before calculating the translation amounts of the left and right eye views, it further includes the step of filtering the positions of the left and right eyes in the human eye data of the user, as follows:

[0031] In the plane coordinate system corresponding to the target display window:

[0032] After adding the x - coordinate values of the position coordinates of the left / right eye in the collected human eye data, low - pass filtering is performed;

[0033] After subtracting the set value from the y - coordinate values of the position coordinates of the left / right eye in the collected human eye data, low - pass filtering is performed.

[0034] Specifically, in S1, the 3D scene to be displayed is segmented by OpenGL to obtain the left and right eye views respectively for display in the left and right viewports.

[0035] Specifically, the sensor of the display device is installed on a pan - tilt head. There is a motor in the pan - tilt head that controls the rotation of the pan - tilt head and is communicatively connected to the user terminal. The sensor of the display device rotates with the pan - tilt head;

[0036] When the live image collected by the sensor of the display device does not meet the user's requirements, control information is sent from the user terminal to the motor in the pan - tilt head. The motor controls the rotation of the pan - tilt head, thereby driving the rotation of the sensor of the display device to immediately adjust the viewing angle.

[0037] The present invention also provides a naked - eye 3D display panoramic viewing system for the aforementioned naked - eye 3D display panoramic viewing method based on human eye tracking, including:

[0038] A scene segmentation module for segmenting the 3D scene to be displayed into left and right eye views respectively for display in the left and right viewports;

[0039] A data acquisition module for acquiring the human eye data of the user collected by the sensor on the display device;

[0040] A zoom control module for filtering the distances between the left and right eyes in the user's eye data and the display device, calculating the zoom ratios of the left and right eye views based on this, and respectively zooming the left and right eye views accordingly;

[0041] A pan control module for filtering the positions of the left and right eyes in the user's eye data, calculating the pan amounts of the left and right eye views based on this, and respectively panning the left and right eye views accordingly.

[0042] Advantageous effects: The present invention can respectively control the zooming and panning of the left and right eye views according to the user's viewing position and viewing angle, and can change the size and display angle of the naked-eye 3D image in real time, expanding the viewing angle range. It solves the limitations caused by adaptive zooming. The user can view the whole picture or zoom in on a certain part of the content to achieve a panoramic viewing function; at the same time, it eliminates the image distortion effect caused by a wide-angle camera and improves the user's experience of viewing 3D images. Brief Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of a naked-eye 3D display panoramic viewing method based on eye tracking according to the present invention;

[0045] Figure 2 It is a flowchart of filtering the distances between the left and right eyes in the eye data and the display device and calculating the zoom ratio according to the present invention;

[0046] Figure 3 It is a flowchart of angle adjustment according to the present invention. Detailed Embodiments

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following will further describe the present application in detail with reference to specific embodiments and the accompanying drawings.

[0048] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should have the ordinary meaning understood by those of ordinary skill in the field to which the present invention belongs. Words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "couple" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0049] The naked-eye 3D display panoramic view method based on human eye tracking of the present invention is as Figure 1 shown, including:

[0050] S1. Obtain the 3D picture to be displayed, and divide it into left and right eye views respectively for display in the left and right viewports;

[0051] In the present invention, the left and right viewports can be set according to the target display window on the display device.

[0052] In the present invention, the left and right eye views can be set according to the size of the 3D picture to be displayed. Specifically, the 3D picture to be displayed can be evenly divided into two views from the midpoint of its width, corresponding to the left and right eye views respectively; further, the display range of the left eye view starts from the lower left corner point of the target display window, its width is half of the width of the 3D picture to be displayed, and its height is the same as the height of the 3D picture to be displayed; the display range of the right eye view starts from the lower right corner point of the left eye view, its width is half of the width of the 3D picture to be displayed, and its height is the same as the height of the 3D picture to be displayed.

[0053] In the present invention, the 3D picture to be displayed can be divided by OpenGL to obtain the left and right eye views respectively for display in the left and right viewports. Specifically, the vertex coordinates and texture coordinates of the left and right eye views are extracted, and according to the vertex coordinates of the left and right eye views, the texture coordinates of the left and right eye views are re-assigned, and the specific assignment is as follows:

[0054] If a certain vertex is in the left eye view and the x coordinate value of its corresponding texture coordinate is greater than 0.5, then the x coordinate value of its texture coordinate is re-assigned to x - 0.5, otherwise the x coordinate value of its texture coordinate remains unchanged;

[0055] If a certain vertex is in the right eye view and the x coordinate value of its corresponding texture coordinate is less than 0.5, then the x coordinate value of its texture coordinate is re-assigned to 0.5 - x, otherwise the x coordinate value of its texture coordinate remains unchanged.

[0056] S2. Collect the human eye data of the user through the sensor on the display device;

[0057] S3. Calculate the scaling ratios of the left and right eye views according to the distances between the left and right eyes and the display device in the human eye data obtained in S2, and scale the left and right eye views obtained in S1 accordingly;

[0058] In the present invention, each vertex of the left and right eye views is scaled according to the calculated scaling ratio, and then the texture coordinates of the left and right eye views are re-assigned, and the specific assignment is the same as that in S1, so that the overall scaling of the picture can be realized.

[0059] In the present invention, for the distance between the left / right eye of the user obtained according to S2 and the display device, i.e., the z coordinate value in the world coordinate system, the scaling ratio of the left / right eye view is calculated by interpolation method, specifically as follows:

[0060] Obtain the distance range between the left / right eye and the display device, and set the value range of the scaling ratio; within the said distance range, when the distance between the left / right eye and the display device is a certain value, set the scaling ratio to 1;

[0061] Thus, according to the actual distance between the left / right eye and the display device, calculate the actual scaling ratio.

[0062] More specifically, knowing the minimum value Z min and the maximum value Z max of the distance between the left / right eye and the display device, and the distance between the left / right eye and the display device is Z, set the minimum value F min and the maximum value F max of the scaling ratio;

[0063] When the distance between the left / right eye and the display device is the fixed value Z s , set the scaling ratio F = 1 at this time; in this embodiment, the fixed value is taken as 1000 mm;

[0064] Then:

[0065] When the distance between the left / right eye and the display device is Z less than the fixed value, the scaling ratio becomes larger at this time, and the calculation of the scaling ratio is as follows:

[0066] F′ = F + (Z s - Z) / (Z s - Z min ) × (F max - F);

[0067] When the distance between the left / right eye and the display device is Z greater than the fixed value, the scaling ratio becomes smaller at this time, and the calculation of the scaling ratio is as follows:

[0068] F′ = F - (Z - Z s ) / (Z max - Z s ) × (F - F min );

[0069] In the present invention, before calculating the scaling ratio, it further includes the step of filtering the distance between the left and right eyes of the user in the eye data with respect to the display device, so as to eliminate the noise and jitter of the eye data.

[0070] Specifically, the filtering of the distance between the left / right eye of the user in the eye data with respect to the display device adopts step-by-step filtering, as Figure 2 shown, specifically including the steps:

[0071] (1) Filter the distance between the left / right eye of the user and the display device in the user's eye data by using Exponential Moving Average (EMA) filtering; in this embodiment, the filtering exponent of the Exponential Moving Average filtering can be taken as 0.1 - 0.4, preferably, the filtering exponent of the Exponential Moving Average filtering can be taken as 0.3;

[0072] (2) Perform hysteresis filtering on the distance between the left / right eye and the display device obtained by the filtering in step (1) to reduce the influence of system noise; in this embodiment, the hysteresis threshold of the hysteresis filtering can be taken as 10 - 100 mm, preferably, the hysteresis threshold of the hysteresis filtering is taken as 30 mm;

[0073] In the present invention, after calculating the scaling ratio, the calculated scaling ratio can also be subjected to damping filtering; in this embodiment, to achieve the balance between fast response and smoothness, the damping coefficient of the damping filtering can be taken as 0.05 - 0.3, preferably, the damping coefficient of the damping filtering is taken as 0.2. The present invention performs damping filtering on the calculated scaling ratio to ensure the stability of the output naked-eye 3D image display and the smoothness of the change.

[0074] The present invention independently scales the left and right eye views according to the distance between the human eye and the display device, so as to keep the visual feeling of the 3D image consistent with the feeling of "objects appear larger when closer and smaller when farther" in the real world when viewing objects, and improve the viewing experience of the 3D image.

[0075] S4. Calculate the translation amounts of the left and right eye views according to the positions of the left and right eyes in the human eye data obtained in S2, and translate the left and right eye views obtained in S1 accordingly;

[0076] Specifically, given the size of the target display window, according to the positions of the left / right eyes in the human eye data obtained in S2, calculate the offset values of the left / right eye views in the x-direction and y-direction of the plane coordinate system corresponding to the target display window, and thus translate the left and right eye views obtained in S1 to simulate the images of different viewing angles observed when the user's head moves, and increase the viewing angle to achieve a panoramic viewing effect.

[0077] In a specific embodiment of the present invention, it is known that the size of the target display window is w×h, where h is the height of the target display window and w is the width of the target display window;

[0078] According to the positions of the left / right eyes in the human eye data obtained in S2, obtain the x coordinate value x0 and y coordinate value y0 of the left / right eyes, then according to the requirements, the offset values x1 and y1 of the left / right eye views in the x-direction and y-direction can be calculated as follows:

[0079] x1 = x0 / (w / k1)

[0080] y1 = y0 / (h / k2)

[0081] Wherein, k1 and k2 are respectively the set offset coefficients in the x - direction and y - direction;

[0082] In this embodiment, k1 can take values from 2 to 100, and k2 can take values from 4 to 100. Preferably, k1 takes 2 and k2 takes 4, as follows:

[0083] x1 = x0 / (w / 2)

[0084] y1 = y0 / (h / 4)

[0085] In this embodiment, the size of the target display window is 3840×2160, that is, the width of the target display window is 3840 and the height of the target display window is 2160.

[0086] In the present invention, before calculating the translation amounts of the left and right eye views, it further includes the step of filtering the positions of the left and right eyes in the user's eye data to eliminate the 3D picture jitter phenomenon caused by the movement of the user's left and right eyes, specifically as follows:

[0087] Filter the positions of the left and right eyes in the collected eye data by using low - pass filtering.

[0088] Specifically, after adding the x - coordinate values of the position coordinates of the left / right eye in the collected eye data, perform low - pass filtering; after subtracting the set value from the y - coordinate values of the position coordinates of the left / right eye in the collected eye data, perform low - pass filtering.

[0089] In this embodiment, the filtering index of the low - pass filtering can take values from 0.1 to 0.3. Preferably, the filtering index of the low - pass filtering takes 0.2.

[0090] In the present invention, it further includes an adaptive angle adjustment step. Install the sensors of the display device, such as binocular cameras, on a pan - tilt head. There is a motor in the pan - tilt head that controls the rotation of the pan - tilt head and is communicatively connected to the user terminal. The sensors of the display device can rotate with the pan - tilt head. When the on - site image collected by the sensors of the display device does not meet the user's requirements, that is, due to the angle limitation of the sensors of the display device resulting in the problem that the user cannot observe the whole scene, the user can send control information to the motor in the pan - tilt head through the user terminal, and the motor controls the rotation of the pan - tilt head, thereby driving the rotation of the sensors of the display device, and then realizing the immediate adjustment of the viewing angle.

[0091] The present invention realizes the eye - controlled pan - tilt head through eye positioning and viewing angle judgment. It can track the position of the human eye through sensors such as binocular cameras, and judge the visual direction of the human eye. Transmit the direction information to the remote pan - tilt head according to the visual direction of the human, and then the viewing direction of the pan - tilt head can be adjusted, thereby realizing the immediate adjustment of the viewing angle.

[0092] Specifically, the general angle range collected by the sensors of the display device is 150°. If the data collected by the sensors of the display device within this angle range still cannot meet the image angle required by the customer, adaptive angle adjustment is needed.

[0093] In this embodiment, as Figure 3 shown, the user terminal, i.e., the on-site device, can also be communicatively connected to the remote device through communication. When there is also a problem that the user at the remote device cannot fully observe due to the angle limitation of the sensors of the display device, the on-site device returns the current position of the pan-tilt to the remote device, and the rotation direction of the pan-tilt can be controlled at the remote device. Specifically, the user can control the pan-tilt at the remote device. The remote device sends control data to the user terminal, and the user terminal sends the control data to the motor in the pan-tilt. The motor controls the rotation of the pan-tilt, thereby driving the rotation of the sensors of the display device, and then realizing instant adjustment of the viewing angle, so as to ensure that the user can also view the on-site image of the application within the maximum range at the remote device.

[0094] In this embodiment, the user terminal and the remote device are communicatively connected through a 485 serial port.

[0095] Through the adaptive angle adjustment step, the present invention can instantaneously adjust the acquisition angle of the sensors of the display device and acquire more on-site images.

[0096] The present invention can respectively control the zooming and panning of the left / right eye views according to the viewing position and viewing angle of the user, and can instantaneously change the size and display angle of the naked-eye 3D image, expanding the viewing angle range. It solves the limitation caused by adaptive zooming. The user can view the whole picture and can also zoom in on a certain part of the content to achieve the panoramic viewing function. At the same time, it eliminates the image distortion effect caused by the wide-angle camera and improves the user experience of viewing the 3D image.

[0097] The present invention also provides a naked-eye 3D display panoramic viewing system based on the foregoing naked-eye 3D display panoramic viewing method based on human eye tracking, including:

[0098] A picture segmentation module for segmenting the 3D picture to be displayed into left and right eye views respectively for left and right viewport displays;

[0099] A data acquisition module for acquiring the human eye data of the user collected by the sensors on the display device;

[0100] A zoom control module for calculating the zoom ratios of the left and right eye views according to the distances between the left and right eyes and the display device in the human eye data of the user, and respectively zooming the left and right eye views accordingly;

[0101] The translation control module is used to calculate the translation amounts of the left and right eye views based on the positions of the left and right eyes in the user's eye data, and respectively translate the left and right eye views accordingly.

[0102] In the present invention, before calculating the zoom ratios of the left / right eye views, the zoom control module can also filter the distances between the left and right eyes of the user in the eye data and the display device to eliminate the noise and jitter of the eye data. The specific filtering process is as described above.

[0103] In the present invention, after calculating the zoom ratios, the zoom control module can also perform damping filtering on the calculated zoom ratios. By filtering the calculated magnification factors in the present invention, the stability of the output naked-eye 3D image display and the smoothness of the change are ensured.

[0104] In the present invention, before calculating the translation amounts of the left and right eye views, the translation control module can also filter the positions of the left and right eyes in the user's eye data to eliminate the jitter phenomenon of the 3D image caused by the movement of the user's left / right eyes. The specific filtering process is as described above.

[0105] The present invention further includes an angle adjustment module, which is used to, when the on-site image collected by the sensor of the display device does not meet the user's requirements, that is, when the user cannot observe the whole scene due to the angle limitation of the sensor of the display device, send control information to the motor in the pan-tilt head, control the pan-tilt head to rotate through the motor, and then drive the sensor of the display device to rotate, so as to realize the immediate adjustment of the viewing angle.

[0106] The present invention first crops the full-screen image left and right, and then adjusts the zoom and translation of the 3D image in real time by collecting the change data of the user's eye positions. Through dynamic decision-making, it can change the size and display angle of the naked-eye 3D image in real time according to the user's viewing position and viewing angle. The user can either zoom in on a certain part of the content to view it, or move to view all the content, improving the 3D viewing experience and effect. At the same time, the present invention can eliminate the distortion effect caused by the camera. The present invention processes the left and right eye views separately, which can simplify the operation of processing the 3D image subsequently and quickly realize the panoramic viewing function. In addition, the present invention utilizes the pan-tilt head function. When the content with a fixed angle of the camera cannot meet the user's requirements, it can adjust the acquisition angle of the front-end camera in real time, enabling the system to have a larger viewing angle and adjust the display angle of the 3D image in real time, improving the user's viewing experience in the dynamic viewing angle.

[0107] Those of ordinary skill in the art should understand that any discussion of the above embodiments is merely exemplary and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples; under the concept of the present invention, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of brevity.

[0108] Embodiments of the present invention are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention shall be included within the protection scope of the present invention.

Claims

1. A naked-eye 3D display panoramic view method based on human eye tracking, characterized in that, Including: S1. Obtain the 3D picture to be displayed, and divide it into left and right eye views respectively for display in the left and right viewports; S2. Collect the user's eye data through the sensors on the display device; S3. Calculate the scaling ratios of the left and right eye views according to the distances between the left and right eyes and the display device in the user's eye data obtained in S2, and scale the left and right eye views obtained in S1 accordingly; S4. Calculate the translation amounts of the left and right eye views according to the positions of the left and right eyes in the user's eye data obtained in S2, and translate the left and right eye views obtained in S1 respectively accordingly.

2. The naked-eye 3D display panoramic view method according to claim 1, wherein In S3, the calculation of the scaling ratios of the left and right eye views is as follows: The minimum value Z of the distance between the left / right eye and the display device is known min and the maximum value Z max , the distance between the left / right eye and the display device is Z, and the minimum value F of the set scaling ratio is set min and the maximum value F max ; When the distance between the left / right eye and the display device is a fixed value Z s the zoom ratio F is set to 1 at this time; Then: When the distance Z between the left / right eye and the display device is less than the fixed value, the calculation of the scaling ratio is as follows: F′ = F + (Z s - Z) / (Z s - Z min ) × (F max - F); When the distance Z between the left / right eye and the display device is greater than the fixed value, the calculation of the scaling ratio is as follows: F′ = F - (Z - Z s ) / (Z max - Z s ) × (F - F min )。 3. The naked-eye 3D display panoramic viewing method according to claim 1, wherein In S3, before calculating the scaling ratio, it further includes the step of filtering the distances between the left and right eyes and the display device in the user's eye data, which is as follows: (1) Use the exponentially weighted moving average filter to filter the distances between the left and right eyes and the display device in the user's eye data; (2) Perform hysteresis filtering on the distances between the left / right eye and the display device obtained by the filtering in step (1).

4. The naked-eye 3D display panoramic view method according to claim 3, characterized in that, The filtering exponent of the exponentially weighted moving average filter is taken as 0.1 - 0.4, and the hysteresis threshold of the hysteresis filter is taken as 10 - 100mm.

5. The naked-eye 3D display panoramic view method according to claim 1, characterized in that, After calculating the scaling ratio, damping filtering can also be performed on the calculated scaling ratio.

6. The naked-eye 3D display panoramic view method according to claim 1, wherein In S4, the calculation of the translation amounts of the left and right eye views is as follows: It is known that the size of the target display window is w×h, where h is the height of the target display window and w is the width of the target display window; According to the positions of the left / right eyes in the eye data obtained in S2, obtain the x coordinate value x0 and y coordinate value y0 of the left / right eyes in the plane coordinate system corresponding to the target display window, then the offset values x1 and y1 of the left / right eye views in the x direction and y direction are calculated as follows: x1 = x0 / (w / k1); y1 = y0 / (h / k2); Where k1 and k2 are the set offset coefficients in the x direction and y direction respectively.

7. The naked-eye 3D display panoramic view method according to claim 1, characterized in that In S4, before calculating the translation amounts of the left and right eye views, it further includes the step of filtering the positions of the left and right eyes in the user's eye data, which is as follows: In the plane coordinate system corresponding to the target display window: After adding the x coordinate values of the position coordinates of the left / right eyes in the collected eye data, perform low-pass filtering; After subtracting the set value from the y coordinate values of the position coordinates of the left / right eyes in the collected eye data, perform low-pass filtering.

8. The naked-eye 3D display panoramic viewing method according to claim 1, wherein In S1, the 3D picture to be displayed is divided through OpenGL to obtain left and right eye views respectively for display in the left and right viewports.

9. The naked-eye 3D display panoramic view method according to claim 1, characterized in that The sensors of the display device are installed on the pan-tilt, and there is a motor in the pan-tilt that controls the rotation of the pan-tilt and is communicatively connected to the user terminal, and the sensors of the display device rotate with the pan-tilt; When the on-site image collected by the sensor of the display device does not meet the user's requirements, control information is sent from the user terminal to the motor in the pan-tilt head, and the motor controls the pan-tilt head to rotate, thereby driving the sensor of the display device to rotate and instantly adjusting the viewing angle.

10. A naked-eye 3D display panoramic view system for the naked-eye 3D display panoramic view method based on human eye tracking according to any one of claims 1-9, characterized in that, It includes: A picture segmentation module for segmenting the 3D picture to be displayed into left and right eye views respectively for display on the left and right viewports; A data acquisition module for acquiring the human eye data of the user collected by the sensor on the display device; A zoom control module for filtering the distances between the left and right eyes of the user in the human eye data and the display device, and calculating the zoom ratios of the left and right eye views accordingly, thereby zooming the left and right eye views respectively; A pan control module for filtering the positions of the left and right eyes of the user in the human eye data and calculating the pan amounts of the left and right eye views accordingly, thereby panning the left and right eye views respectively.

Images