A Real-time Naked-eye 3D Rendering Method and System Based on a Folding Screen

Abstract

The present invention provides a real-time naked-eye 3D rendering method and system based on a foldable screen, which relates to the field of display imaging technology of foldable screen devices, and includes calculating the relative position between the human eye and the camera based on the images collected in real time by a depth perception camera; constructing a coordinate system with the human eye as the origin, and calculating the position coordinates of the four corners of each display area of the foldable screen according to the relative position; performing rendering in a 3D virtual environment based on the position coordinates to obtain virtual environment images corresponding to each display area of the foldable screen. In the interaction process between the user and the foldable screen, the present invention calculates the relative position between the human eye and each display area of the foldable screen in real time, and performs rendering on each display area of the foldable screen according to the relative position. Even if the relative position changes, the foldable screen can correctly render the 3D virtual image seen by the virtual camera simulated by the human eye in the 3D virtual environment, so that the user can use the portable mobile device related to the foldable screen to experience the naked-eye 3D effect in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of display imaging of foldable screen devices. Specifically, it relates to a real-time naked-eye 3D rendering method and system based on a foldable screen. Background Art

[0002] Naked-eye 3D visual illusion refers to a form of expression that enables viewers to perceive three-dimensional stereoscopic images without wearing any auxiliary devices through specific visual effects and technologies. Its core lies in leveraging the characteristics of the human visual system and creating a sense of depth and three-dimensionality through ingenious image design and optical technologies.

[0003] Traditional naked-eye 3D rendering has fixed viewing angles and rendering devices, resulting in problems such as low popularity and poor portability. Users need to go to specific locations, use specific devices, and view from specific angles to experience the naked-eye 3D effect. And in some cases, the viewing angles of the naked-eye 3D effect are limited, and the content of the rendered 3D image cannot be adjusted in real time according to the user's position and viewing angle. Although mobile 3D experience devices such as VR glasses do not have the above problems, during the experience process, users need to wear designated devices and instruments, which can easily cause burdens and discomfort to users during the experience. Summary of the Invention

[0004] The purpose of the present invention is to provide a real-time naked-eye 3D rendering method and system based on a foldable screen to improve the above problems. To achieve the above purpose, the technical solutions adopted by the present invention are as follows:

[0005] In a first aspect, the present application provides a real-time naked-eye 3D rendering method based on a foldable screen, including:

[0006] Obtain first information and a first image, where the first information includes the size information and spatial position information of each display area of the foldable screen, and the first image is an image collected in real time by a depth perception camera;

[0007] Calculate second information based on the first image to obtain the second information, where the second information includes the distance and angle between the user's eyes and the depth perception camera in the first image;

[0008] Construct a target coordinate system with the user's eyes as the origin, and calculate a first position coordinate based on the first information and the second information to obtain the first position coordinate, where the first position coordinate is the position coordinates of the four corners of each display area of the foldable screen in the target coordinate system;

[0009] Render in the 3D virtual environment to be displayed based on the first position coordinates to obtain a first virtual environment image, where the first virtual environment image is a virtual environment image that simulates the user's human eyes as a virtual camera observing in the 3D virtual environment to be displayed. The first virtual environment image is composed of multiple second virtual environment images, and one second virtual environment image corresponds to one display area of the folding screen.

[0010] In a second aspect, the present application also provides a real-time naked-eye 3D rendering system based on a folding screen, including:

[0011] A first acquisition module, configured to acquire first information and a first image, where the first information includes the size information and spatial position information of each display area of the folding screen, and the first image is an image collected in real time by a depth perception camera;

[0012] A first processing module, configured to calculate second information based on the first image to obtain the second information, where the second information includes the distance and angle between the user's human eyes and the depth perception camera in the first image;

[0013] A second processing module, configured to construct a target coordinate system with the user's human eyes as the origin, and calculate first position coordinates based on the first information and the second information to obtain the first position coordinates, where the first position coordinates are the position coordinates of the four corners of each display area of the folding screen in the target coordinate system;

[0014] A third processing module, configured to render in the 3D virtual environment to be displayed based on the first position coordinates to obtain a first virtual environment image, where the first virtual environment image is a virtual environment image that simulates the user's human eyes as a virtual camera observing in the 3D virtual environment to be displayed. The first virtual environment image is composed of multiple second virtual environment images, and one second virtual environment image corresponds to one display area of the folding screen.

[0015] The beneficial effects of the present invention are as follows:

[0016] In the interaction process between the user and the folding screen, the present invention calculates the relative positions of the human eyes and each display area of the folding screen in real time, and performs rendering on each display area of the folding screen according to the relative positions. Even if the relative positions change, the folding screen can correctly render the 3D virtual image seen by the virtual camera simulated by the human eyes in the 3D virtual environment, so that the user can use the portable mobile device related to the folding screen to experience the naked-eye 3D effect in real time.

[0017] Other features and advantages of the present invention will be described in the subsequent specification, and part of them will become obvious from the specification or be understood by implementing the embodiments of the present invention. Description of the Drawings

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0019] Figure 1 Schematic diagram of the process of a real-time naked-eye 3D rendering method based on a foldable screen according to an embodiment of the present invention;

[0020] Figure 2 Schematic diagram of the structure of a real-time naked-eye 3D rendering system based on a foldable screen according to an embodiment of the present invention;

[0021] Figure 3 Schematic diagram of the position of the first coordinate system according to an embodiment of the present invention;

[0022] Figure 4 Schematic diagram of the naked-eye 3D effect according to an embodiment of the present invention.

[0023] Reference numerals in the figure: 901, first acquisition module; 902, first processing module; 903, second processing module; 904, third processing module; 905, second acquisition module; 906, fourth processing module; 9021, first processing unit; 9022, second processing unit; 9031, third processing unit; 9032, fourth processing unit; 9033, fifth processing unit; 9034, sixth processing unit; 9035, seventh processing unit; 9041, eighth processing unit; 9042, ninth processing unit; 9043, tenth processing unit; 9044, eleventh processing unit; 90221, first processing sub-module; 90222, second processing sub-module; 90223, third processing sub-module. Detailed implementation manners

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and illustrated in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0025] It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present invention, terms such as "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance. Embodiment 1:

[0026] This embodiment provides a real-time naked-eye 3D rendering method based on a foldable screen.

[0027] See Figure 1 , the figure shows that this method includes steps S1, S2, S3, and S4.

[0028] S1. Obtain first information and a first image. The first information includes the size information and spatial position information of each display area of the foldable screen, and the first image is an image captured in real time by a depth perception camera.

[0029] Specifically, the first image is obtained in real time by a front depth perception camera on the same side as a certain display area of the foldable screen. The depth perception camera can not only provide ordinary two-dimensional RGB images but also provide the depth value of each pixel, that is, the distance from this point to the camera. Therefore, the depth perception camera can complete the acquisition of the distance and angle between the user's human eye and the camera, thereby realizing the real-time tracking of the user's human eye.

[0030] Then, according to the foldable screen device, relevant API interfaces are provided to obtain the size information and spatial position information of each display area of the device in real time, such as the screen folding state, display area size, angle and distance between each display area. The acquisition of the first information and the first image has prepared sufficient data for calculating the relative spatial position between the user's human eye and each display area of the foldable screen.

[0031] S2. Calculate second information based on the first image to obtain the second information, where the second information includes the distance and angle between the user's human eye and the depth perception camera in the first image;

[0032] Specifically, step S2 includes:

[0033] S21. Detect the user's human eye area in the first image and mark the user's human eye area in the first image to obtain a second image, where the second image is the first image after marking the user's human eye area;

[0034] Preferably, in this embodiment, first, the user's face region in the first image is recognized by a pre-trained face detection model, and then within the recognized face region, the user's eye region is further located and marked according to a pre-trained eye detection model to obtain an image with the user's eye region marked.

[0035] S22. Extract the second information from the second image based on the depth perception camera to obtain the second information.

[0036] Specifically, after obtaining the image with the user's eye region marked, a depth map containing the depth values of each pixel of the image is generated by the structured light technology, and then the depth values of the pixels corresponding to the user's eye region are extracted according to the marked user's eye region. The specific steps include:

[0037] S221. Project a preset light pattern onto the user's eye region in the second image based on the depth perception camera to obtain a reflected light pattern, where the reflected light pattern is the pattern of the preset light pattern collected by the depth perception camera reflected in the user's eye region in the second image;

[0038] S222. Calculate the depth information of each pixel in the user's eye region in the second image according to the degree of deformation of the reflected light pattern to obtain the depth information;

[0039] S223. Generate a depth image based on the depth information and calculate the second information according to the depth image. Through the depth map, the three-dimensional space coordinates corresponding to each pixel relative to the depth perception camera can be accurately calculated, so as to deduce the distance and angle of the user's eyes relative to the depth perception camera.

[0040] S3. Construct a target coordinate system with the user's eyes as the origin, and calculate the first position coordinates based on the first information and the second information to obtain the first position coordinates, where the first position coordinates are the position coordinates of the four corners of each display area of the folding screen in the target coordinate system;

[0041] Specifically, in this embodiment, with the eyes as the origin, the front of the eyes as the z-axis, the top as the y-axis, and the right as the z-axis, a target coordinate system is established. According to the relative spatial position relationship between the user's eyes and each display area of the folding screen, the accurate coordinates of the four corners of each display area of the folding screen in the target coordinate system are calculated, so as to simulate the user's eyes as a virtual camera to observe in the 3D virtual environment to be displayed, and render the observed virtual environment image on the folding screen according to the accurate coordinates of the four corners of each display area of the folding screen in the target coordinate system to present a real-time naked-eye 3D effect.

[0042] Further, step S3 includes:

[0043] S31. Construct a first coordinate system with the depth perception camera as the origin, and calculate the second position coordinates based on the first information to obtain the second position coordinates, where the second position coordinates are the position coordinates of the four corners of each display area of the folding screen in the first coordinate system;

[0044] Specifically, as Figure 3 shown, with the depth perception camera as the origin and the plane of the display area of the folding screen where the depth perception camera is located as the xy plane, a left-handed space coordinate system is established to obtain the first coordinate system.

[0045] S32. Calculate the third position coordinates based on the second information, where the third position coordinates are the position coordinates of the user's eyes in the first coordinate system. Constructing the coordinate system with the depth perception camera as the origin is convenient for calculating the coordinates of the four corners of each display area of the folding screen according to the size information and spatial position information of each display area of the folding screen on the one hand, and convenient for directly using the three-dimensional space coordinates of each pixel in the user's eye area calculated from the depth map corresponding to the depth perception camera on the other hand. In this way, the second position coordinates and the third position coordinates can accurately store the relative spatial position information between the user's eyes and each display area of the folding screen, laying a foundation for subsequent coordinate conversion and further realizing the naked-eye 3D effect.

[0046] S33. Rotate the first coordinate system based on a preset rotation matrix to obtain the rotated first coordinate system, and the direction of the rotated first coordinate system is aligned with the target coordinate system;

[0047] Specifically, calculate the unit vectors corresponding to the x, y, and z coordinate axes of the first coordinate system 、 、 , and then calculate the unit vectors 、 、 corresponding to the x, y, and z coordinate axes of the target coordinate system in the first coordinate system based on the third position coordinates, and then calculate the rotation matrix according to the dot product of the unit vectors in the corresponding coordinate axis directions to obtain the rotation matrix as:

[0048] ;

[0049] Among them, 、 、 are the unit vectors corresponding to the x, y, and z coordinate axes of the first coordinate system respectively, 、 、 They are unit vectors in the x, y, and z axis directions corresponding to the target coordinate system in the first coordinate system respectively.

[0050] S34. Translate the rotated first coordinate system based on a preset translation vector to obtain the translated first coordinate system, where the origin of the translated first coordinate system coincides with the origin of the target coordinate system.

[0051] Specifically, the coordinate position of the user's human eye in the rotated first coordinate system changes. If the coordinate point of the user's human eye in the first coordinate system before rotation, that is, the third position coordinate, is , then the calculation formula for the coordinate of the user's human eye in the rotated first coordinate system is:

[0052] ;

[0053] Among them, is the rotation matrix, and are the coordinates of the user's human eye in the first coordinate system before and after rotation respectively;

[0054] Furthermore, according to the coordinate of the user's human eye in the rotated first coordinate system, the translation vector can be obtained, which represents the displacement from the coordinate origin of the first coordinate system to the coordinate origin of the target coordinate system in the rotated first coordinate system.

[0055] S35. Perform coordinate transformation on the second position coordinate based on the preset rotation matrix and the preset translation vector to obtain the first position coordinate. After obtaining the rotation matrix and the translation vector , perform coordinate transformation on the second position coordinate in the first coordinate system.

[0056] S4. Render based on the first position coordinate in the 3D virtual environment to be displayed to obtain a first virtual environment image. The first virtual environment image is a virtual environment image simulating the view of the user's human eye as a virtual camera in the 3D virtual environment to be displayed. The first virtual environment image is composed of multiple second virtual environment images, and one second virtual environment image corresponds to one display area of the folding screen.

[0057] Specifically, step S4 includes:

[0058] S41. Transform the target coordinate system and the first position coordinate into the 3D virtual environment to be displayed based on a preset scaling factor to obtain a virtual environment coordinate system and virtual environment coordinates. The position of the origin of the virtual environment coordinate system is the position of the human eye in the 3D virtual environment to be displayed.

[0059] Further, since the target coordinate system is still the coordinate system of the real world, it is also necessary to determine the scaling factor according to the proportional relationship between the real world unit and the virtual environment unit, and transform the target coordinate system and the first position coordinate into the 3D virtual environment to be displayed, so as to obtain the virtual coordinate system and the virtual environment coordinate in the 3D virtual environment to be displayed. The origin of the virtual coordinate system is the position of the human eye in the 3D virtual environment to be displayed, and the direction of the virtual coordinate system is the same as that of the target coordinate system, so as to ensure that the rendered image of the folding screen is the image seen by the human eye in the 3D virtual environment to be displayed. The formula for obtaining the virtual environment coordinate by transforming the second position coordinate is:

[0060] ;

[0061] Wherein, is the scaling factor, is the rotation matrix, is the translation vector, are the coordinates of the four corners of each display area of the folding screen in the first coordinate system, are the coordinates of the four corners of each display area of the folding screen in the virtual coordinate system in the 3D virtual environment to be displayed. In this way, the virtual environment coordinate can be obtained.

[0062] S42. Create a plurality of virtual cameras at the origin of the virtual environment coordinate system. The number of virtual cameras is the same as the number of display areas of the folding screen, and one virtual camera corresponds to one display area of the folding screen. After obtaining the virtual environment coordinate system and the virtual environment coordinate, it is necessary to create virtual cameras at the position of the human eye in the 3D virtual environment to be displayed. The number of virtual cameras is the same as the number of display areas of the folding screen. Since there are certain angles and distances between the display areas of the folding screen, if only one virtual camera is created, there is a problem that the viewing angle of the virtual camera cannot cover all the display areas of the folding screen. Therefore, in order to ensure the integrity of the images rendered for each display area of the folding screen, a virtual camera is created for each display area. The viewing angle of each virtual camera is adjusted according to the coordinates of the four corners of the corresponding display area of the folding screen to ensure that the viewing angle of the virtual camera can just cover the entire display area, and then the images captured by the virtual camera are cropped according to the coordinates of the four corners of the folding screen, so as to ensure the correct rendering effect of the folding screen.

[0063] S43. Generate a plurality of initial virtual images in the 3D virtual environment to be displayed based on the plurality of virtual cameras. One initial virtual image corresponds to one display area of the folding screen;

[0064] S44. Crop each of the initial virtual images based on the virtual environment coordinates corresponding to each of the initial virtual images, and render each of the cropped initial virtual images onto the display area corresponding to the folding screen to obtain a plurality of the second virtual environment images.

[0065] For the virtual cameras set in the manner of this embodiment, the images captured by each virtual camera need to cover the corresponding display area, so there will be some parts of the images that exceed the display area. To ensure the correct rendering effect, it is necessary to crop the images captured by the virtual cameras according to the coordinates of the four corners of each display area to obtain the cropped virtual images.

[0066] S5. Obtain the screen resolution of each display area of the folding screen;

[0067] S6. Adjust the image allocation rate of the second virtual environment image corresponding to each display area of the folding screen according to the screen resolution of each display area of the folding screen to obtain a plurality of third virtual images, and the image allocation rate of the third virtual images matches the corresponding screen allocation rate.

[0068] Specifically, after obtaining the rendering images corresponding to each display area, it is necessary to process the images according to the screen resolution of each display area of the folding screen so that the image resolution of each image matches the screen resolution of the corresponding display area, so as to be correctly rendered onto the folding screen and finally present the visual effect of naked-eye 3D.

[0069] As Figure 4 shown, the folding screen finally renders the images of the objects observed in the human eye's perspective in the virtual scene through the folding screen according to the relative position between the human eye and the folding screen, enabling the user to directly experience the real-time naked-eye 3D effect. Embodiment 2:

[0070] As Figure 2 shown, this embodiment provides a real-time naked-eye 3D rendering system based on a folding screen, and the system includes a first acquisition module 901, a first processing module 902, a second processing module 903, and a third processing module 904:

[0071] The first acquisition module 901 is configured to acquire first information and a first image, where the first information includes the size information and spatial position information of each display area of the folding screen, and the first image is an image captured in real time by a depth perception camera;

[0072] The first processing module 902 is configured to calculate second information based on the first image to obtain the second information, where the second information includes the distance and angle between the user's human eye and the depth perception camera in the first image;

[0073] The second processing module 903 is configured to construct a target coordinate system with the user's human eye as the origin, and calculate the first position coordinates based on the first information and the second information, so as to obtain the first position coordinates, where the first position coordinates are the position coordinates of the four corners of each display area of the folding screen in the target coordinate system;

[0074] The third processing module 904 is configured to perform rendering in the to-be-displayed 3D virtual environment based on the first position coordinates, so as to obtain a first virtual environment image, where the first virtual environment image is a virtual environment image simulated by the user's human eye as a virtual camera observing in the to-be-displayed 3D virtual environment, and the first virtual environment image is composed of multiple second virtual environment images, and one second virtual environment image corresponds to one display area of the folding screen.

[0075] The first processing module 902 includes a first processing unit 9021 and a second processing unit 9022:

[0076] The first processing unit 9021 is configured to detect the user's human eye area in the first image, and mark the user's human eye area in the first image, so as to obtain a second image, where the second image is the first image after marking the user's human eye area;

[0077] The second processing unit 9022 is configured to extract the second information from the second image based on the depth perception camera, so as to obtain the second information.

[0078] The second processing module 903 includes a third processing unit 9031, a fourth processing unit 9032, a fifth processing unit 9033, a sixth processing unit 9034, and a seventh processing unit 9035:

[0079] The third processing unit 9031 is configured to construct a first coordinate system with the depth perception camera as the origin, and calculate the second position coordinates based on the first information, so as to obtain the second position coordinates, where the second position coordinates are the position coordinates of the four corners of each display area of the folding screen in the first coordinate system;

[0080] The fourth processing unit 9032 is configured to calculate the third position coordinates based on the second information, where the third position coordinates are the position coordinates of the user's human eye in the first coordinate system;

[0081] The fifth processing unit 9033 is configured to rotate the first coordinate system based on a preset rotation matrix, so as to obtain the rotated first coordinate system, and the direction of the rotated first coordinate system is aligned with the target coordinate system;

[0082] The sixth processing unit 9034 is configured to translate the rotated first coordinate system based on a preset translation vector to obtain the translated first coordinate system, and the origin of the translated first coordinate system coincides with the origin of the target coordinate system;

[0083] The seventh processing unit 9035 is configured to perform coordinate transformation on the second position coordinate based on the preset rotation matrix and the preset translation vector to obtain the first position coordinate.

[0084] The third processing module 904 includes an eighth processing unit 9041, a ninth processing unit 9042, a tenth processing unit 9043, and an eleventh processing unit 9044:

[0085] The eighth processing unit 9041 is configured to transform the target coordinate system and the first position coordinate into a 3D virtual environment to be displayed based on a preset scaling factor to obtain a virtual environment coordinate system and virtual environment coordinates, and the position of the origin of the virtual environment coordinate system is the position of the human eye in the 3D virtual environment to be displayed;

[0086] The ninth processing unit 9042 is configured to create a plurality of virtual cameras at the origin of the virtual environment coordinate system, the number of the virtual cameras is the same as the number of display areas of the folding screen, and one virtual camera corresponds to one display area of the folding screen;

[0087] The tenth processing unit 9043 is configured to generate a plurality of initial virtual images in the 3D virtual environment to be displayed based on the plurality of virtual cameras, and one initial virtual image corresponds to one display area of the folding screen;

[0088] The eleventh processing unit 9044 is configured to crop each initial virtual image based on the virtual environment coordinates corresponding to each initial virtual image, and render each cropped initial virtual image onto the corresponding display area of the folding screen to obtain a plurality of second virtual environment images

[0089] A real-time naked-eye 3D rendering system based on a folding screen further includes a second acquisition module 905 and a fourth processing module 906:

[0090] The second acquisition module 905 is configured to acquire the screen resolution of each display area of the folding screen;

[0091] The fourth processing module 906 is configured to adjust the image allocation rate of the second virtual environment image corresponding to each display area of the folding screen according to the screen resolution of each display area of the folding screen to obtain a plurality of third virtual images, and the image allocation rate of the third virtual image matches the corresponding screen allocation rate.

[0092] The second processing unit 9022 includes a first processing sub-module 90221, a second processing sub-module 90222, and a third processing sub-module 90223:

[0093] The first processing sub-module 90221 is configured to project a preset light pattern onto the user's eye region in the second image based on a depth perception camera, and obtain a reflected light pattern, where the reflected light pattern is the pattern of the preset light pattern collected by the depth perception camera reflected in the user's eye region in the second image;

[0094] The second processing sub-module 90222 is configured to calculate the depth information of each pixel in the user's eye region in the second image according to the degree of deformation of the reflected light pattern, and obtain the depth information;

[0095] The third processing sub-module 90223 is configured to generate a depth image based on the depth information, and calculate the second information according to the depth image.

[0096] It should be noted that regarding the system in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0097] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0098] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention.

Claims

1. A real-time naked-eye 3D rendering method based on a folding screen, characterized in that: include: Acquire first information and a first image, wherein the first information includes size information and spatial position information of each display area of ​​the folding screen, and the first image is an image captured in real time by a depth perception camera; Calculate second information based on the first image to obtain the second information, where the second information includes a distance and an angle between a user's eyes and a depth perception camera in the first image; Constructing a target coordinate system with the user's eyes as the origin, and calculating first position coordinates based on the first information and the second information to obtain the first position coordinates, where the first position coordinates are position coordinates of four corners of each display area of ​​the folding screen in the target coordinate system; Based on the first position coordinates, rendering is performed in the 3D virtual environment to be displayed to obtain a first virtual environment image, wherein the first virtual environment image simulates the virtual environment image observed by the user's human eyes as a virtual camera in the 3D virtual environment to be displayed, and the first virtual environment image is composed of multiple second virtual environment images, and one of the second virtual environment images corresponds to a display area of ​​the folding screen.

2. A real-time naked-eye 3D rendering method based on a folding screen according to claim 1, characterized in that , the calculating the second information based on the first image comprises: Detecting a user's eye region in the first image, and marking the user's eye region in the first image to obtain a second image, where the second image is the first image after the user's eye region is marked; The second information is extracted from the second image based on a depth perception camera to obtain the second information.

3. The real-time naked eye 3D rendering method based on a folding screen according to claim 1, characterized in that , the calculating the first position coordinates based on the first information and the second information includes: Constructing a first coordinate system with the depth perception camera as the origin, and calculating second position coordinates based on the first information to obtain the second position coordinates, where the second position coordinates are position coordinates of four corners of each display area of ​​the folding screen in the first coordinate system; Calculate a third position coordinate based on the second information, where the third position coordinate is the position coordinate of the user's eyes in the first coordinate system; Rotating the first coordinate system based on a preset rotation matrix to obtain a rotated first coordinate system, wherein a direction of the rotated first coordinate system is aligned with the target coordinate system; The rotated first coordinate system is translated based on a preset translation vector to obtain the translated first coordinate system, wherein the origin of the translated first coordinate system coincides with the origin of the target coordinate system; The second position coordinates are transformed based on the preset rotation matrix and the preset translation vector to obtain the first position coordinates.

4. The real-time naked-eye 3D rendering method based on a folding screen according to claim 1, characterized in that , the rendering in the 3D virtual environment to be displayed based on the first position coordinates includes: transforming the target coordinate system and the first position coordinates into the 3D virtual environment to be displayed based on a preset scaling factor to obtain a virtual environment coordinate system and virtual environment coordinates, wherein the position of the origin of the virtual environment coordinate system is the position of the human eye in the 3D virtual environment to be displayed; Creating a plurality of virtual cameras at the origin of the virtual environment coordinate system, wherein the number of the virtual cameras is the same as the number of display areas of the folding screen, and one virtual camera corresponds to one display area of ​​the folding screen; generating a plurality of initial virtual images in the 3D virtual environment to be displayed based on the plurality of virtual cameras, wherein one of the initial virtual images corresponds to a display area of ​​the folding screen; Each of the initial virtual images is cropped based on the virtual environment coordinates corresponding to each of the initial virtual images, and each of the cropped initial virtual images is rendered onto a display area corresponding to the folding screen to obtain a plurality of the second virtual environment images.

5. The method for real-time naked-eye 3D rendering based on a folding screen according to claim 1, characterized in that After obtaining the first virtual environment image, the method further includes: Get the screen resolution of each display area of ​​the folding screen; The image allocation ratio of the second virtual environment image corresponding to each display area of ​​the folding screen is adjusted according to the screen resolution of each display area of ​​the folding screen to obtain multiple third virtual images, and the image allocation ratio of the third virtual images matches the screen resolution of each display area of ​​the corresponding folding screen.

6. A real-time naked-eye 3D rendering system based on a folding screen, characterized in that: include: A first acquisition module, configured to acquire first information and a first image, wherein the first information includes size information and spatial position information of each display area of ​​the folding screen, and the first image is an image acquired in real time by a depth perception camera; A first processing module, configured to calculate second information based on the first image to obtain the second information, wherein the second information includes a distance and an angle between a user's eyes and a depth perception camera in the first image; a second processing module, configured to construct a target coordinate system with the user's eyes as the origin, and calculate first position coordinates based on the first information and the second information to obtain the first position coordinates, where the first position coordinates are position coordinates of four corners of each display area of ​​the folding screen in the target coordinate system; The third processing module is used to render in the 3D virtual environment to be displayed based on the first position coordinates to obtain a first virtual environment image, wherein the first virtual environment image simulates the virtual environment image observed by the user's human eyes as a virtual camera in the 3D virtual environment to be displayed, and the first virtual environment image is composed of multiple second virtual environment images, and one of the second virtual environment images corresponds to a display area of ​​the folding screen.

7. The real-time naked-eye 3D rendering system based on a folding screen according to claim 6, characterized in that: The first processing module comprises: a first processing unit, configured to detect a user eye region in the first image, and mark the user eye region in the first image to obtain a second image, where the second image is the first image after the user eye region is marked; The second processing unit is used to extract the second information from the second image based on the depth perception camera to obtain the second information.

8. The real-time naked-eye 3D rendering system based on a folding screen according to claim 6, characterized in that: The second processing module comprises: a third processing unit, configured to construct a first coordinate system with the depth perception camera as an origin, and calculate second position coordinates based on the first information to obtain the second position coordinates, where the second position coordinates are position coordinates of four corners of each display area of ​​the folding screen in the first coordinate system; a fourth processing unit, configured to calculate third position coordinates based on the second information, wherein the third position coordinates are position coordinates of the user's eyes in the first coordinate system; a fifth processing unit, configured to rotate the first coordinate system based on a preset rotation matrix to obtain a rotated first coordinate system, wherein a direction of the rotated first coordinate system is aligned with the target coordinate system; a sixth processing unit, configured to translate the rotated first coordinate system based on a preset translation vector to obtain the translated first coordinate system, wherein the origin of the translated first coordinate system coincides with the origin of the target coordinate system; The seventh processing unit is used to perform coordinate transformation on the second position coordinates based on the preset rotation matrix and the preset translation vector to obtain the first position coordinates.

9. The real-time naked-eye 3D rendering system based on a folding screen according to claim 6, characterized in that: The third processing module comprises: an eighth processing unit, configured to transform the target coordinate system and the first position coordinates into a 3D virtual environment to be displayed based on a preset scaling factor, to obtain a virtual environment coordinate system and virtual environment coordinates, wherein the position of the origin of the virtual environment coordinate system is the position of the human eye in the 3D virtual environment to be displayed; a ninth processing unit, configured to create a plurality of virtual cameras at the origin of the virtual environment coordinate system, wherein the number of the virtual cameras is the same as the number of display areas of the folding screen, and one of the virtual cameras corresponds to one display area of ​​the folding screen; a tenth processing unit, configured to generate a plurality of initial virtual images in the 3D virtual environment to be displayed based on the plurality of virtual cameras, wherein one initial virtual image corresponds to one display area of ​​the folding screen; The eleventh processing unit is used to crop each of the initial virtual images based on the virtual environment coordinates corresponding to each of the initial virtual images, and render each of the cropped initial virtual images onto the display area corresponding to the folding screen to obtain a plurality of the second virtual environment images.

10. The real-time naked-eye 3D rendering system based on a folding screen according to claim 6, characterized in that: Also includes: A second acquisition module is used to acquire the screen resolution of each display area of ​​the folding screen; The fourth processing module is used to adjust the image allocation rate of the second virtual environment image corresponding to each display area of ​​the folding screen according to the screen resolution of each display area of ​​the folding screen, so as to obtain multiple third virtual images, and the image allocation rate of the third virtual images matches the screen resolution of each display area of ​​the corresponding folding screen.

Images