Three-dimensional display method, device, display module and readable storage medium of image

Abstract

The application provides a three-dimensional display method and device of an image, a display module and a readable storage medium. The method comprises the following steps: obtaining an eye position of a user at a current time as a first eye position, and obtaining an eye position of the user at a previous time as a second eye position; determining a position deviation amount of the first eye position and the second eye position; correcting the first eye position by using the position deviation amount to obtain a current observation position; and simultaneously updating a target image displayed in a left half region and a target image displayed in a right half region of a display region by using the current observation position, so that the user can observe a three-dimensional target image according to the content displayed in the left half region and the right half region at the current observation position. In this way, the display module can adjust the image in the display region according to the changed current observation position after the change of the eye position of the user, so that the user can continue to observe the three-dimensional image by using the naked eye.

Description

Technical Field

[0001] This application relates to the field of three-dimensional technology, and more specifically, to a three-dimensional display method, apparatus, display module, and readable storage medium for an image. Background Technology

[0002] In related technologies, glasses-free 3D requires users to maintain a fixed eye position to ensure continuous viewing of 3D images. If a user moves their head, causing a change in eye position, they must return to the previous position to continue viewing the 3D images, resulting in a poor viewing experience. Summary of the Invention

[0003] This application provides a method, apparatus, display module, and readable storage medium for displaying three-dimensional images.

[0004] This application provides a method for displaying a three-dimensional image, comprising:

[0005] The user's current eye position is obtained as the first eye position, and the user's previous eye position is obtained as the second eye position.

[0006] Based on the first eye position and the second eye position, determine the positional deviation.

[0007] The position of the first eye is corrected based on the positional deviation to obtain the current observation position;

[0008] Based on the current observation position, the target images in the left half and right half of the display area are updated accordingly, so that a three-dimensional image can be observed at the current observation position.

[0009] In the three-dimensional image display method provided in this application, after the display module receives / acquires the user's eye position at the current moment, it takes the current eye position as the first eye position and the user's eye position at the previous moment as the second eye position. Then, it determines the deviation between the first eye position and the second eye position, i.e., the position deviation. After that, it corrects the first eye position using the position deviation so that the corrected current observation position can correctly reflect the user's eye position at the current moment. Finally, it updates the target images of the left and right halves of the display area simultaneously using the current observation position so that the user can observe the three-dimensional image at the current observation position.

[0010] Thus, by determining the current observation position, this application enables the display module to adjust the image in the display area accordingly after the user's eye position changes, ensuring that the user can continue to observe the 3D image with the naked eye, thereby guaranteeing the user's observation experience. Furthermore, this application also obtains the current observation position by correcting the first eye position, avoiding the errors introduced by directly using the first eye position, and thus reducing the negative impact caused by hardware costs and hardware limitations.

[0011] This application also provides a three-dimensional image display device, the device comprising:

[0012] The acquisition module is used to acquire the user's eye position at the current moment as the first eye position, and acquire the user's eye position at the previous moment as the second eye position.

[0013] The determining module is used to determine the positional deviation based on the first eye position and the second eye position;

[0014] The correction module is used to correct the first eye position based on the position deviation to obtain the current observation position;

[0015] The update module is used to update the target image in the left half of the display area and the target image in the right half of the display area based on the current observation position, so that a three-dimensional image can be observed at the current observation position.

[0016] This application provides a display module, including a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the above-described three-dimensional image display method.

[0017] The computer-readable storage medium of this application stores a computer program that, when executed by one or more processors, implements the above-described three-dimensional image display method.

[0018] The three-dimensional display device, display module, and computer-readable storage medium provided in this application, by determining the current observation position, enable the display module to adjust the image in the display area accordingly after the user's eye position changes, so as to ensure that the user can continue to observe the 3D image with the naked eye, thereby guaranteeing the user's observation experience; in addition, this application also obtains the current observation position by correcting the first eye position, avoiding the error introduced by directly using the first eye position, and thus reducing the negative impact caused by hardware cost and hardware limitations.

[0019] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0021] Figure 1 This is a flowchart illustrating a three-dimensional image display method according to certain embodiments of this application;

[0022] Figure 2 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0023] Figure 3 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0024] Figure 4 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0025] Figure 5 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0026] Figure 6 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0027] Figure 7 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0028] Figure 8 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0029] Figure 9 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0030] Figure 10 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0031] Figure 11 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0032] Figure 12 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0033] Figure 13 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0034] Figure 14 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0035] Figure 15 This is a schematic diagram illustrating application scenarios for some embodiments of this application;

[0036] Figure 16 This is a schematic diagram of a three-dimensional display device for images according to certain embodiments of this application. Detailed Implementation

[0037] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0038] Please see Figure 1 This application provides a method for displaying a three-dimensional image, comprising:

[0039] 0110, obtain the user's current eye position as the first eye position, and obtain the user's previous eye position as the second eye position.

[0040] It is understood that the method for obtaining eye position in this application is customizable based on actual circumstances. In some embodiments, the display module used to display the screen / image integrates a camera module and a target detection module. Therefore, after the display captures an image through the camera module, the target detection module identifies the user's eyes in the image to obtain the eye position.

[0041] It is also understood that the time difference between the first eye position and the second eye position in this application, that is, the frequency of collecting eye positions in this application, is set according to the actual situation. For example, in some embodiments, the collection frequency of eye positions is 60Hz.

[0042] It is not difficult to imagine that after each eye position is completed, the obtained eye position can be stored in a preset storage space (such as a cache). Then, when a second eye position needs to be obtained, it can be achieved by reading the last stored data in the storage space.

[0043] 0120, based on the first eye position and the second eye position, determine the positional deviation.

[0044] It is understandable that, due to hardware limitations of the acquisition devices (such as the aforementioned camera module and target detection module), it may be impossible to accurately determine changes in the user's eye position when collecting / acquiring eye position data. For example, when the user turns their head, the low shooting frequency of the camera module may result in the camera module capturing images with eye afterimages. Consequently, the target detection module will identify the position of the eye afterimages in the images, leading to inaccurate / unreal eye positions.

[0045] Therefore, in order to correct the eye position deviation caused by hardware limitations, this application calculates the deviation between the previous and current eye positions (i.e., position deviation) after acquiring / collecting any eye position. Then, the position deviation is used to correct the current eye position in subsequent processes.

[0046] It is also understandable that, since this application needs to use the previously obtained eye position to correct the currently obtained eye position, in order to improve the accuracy of the eye position, in some embodiments of this application, the second eye position in the process of calculating the position deviation can be a corrected amount. That is, after correcting the second eye position using the eye position obtained at the previous moment of the second eye position, the corrected amount is used to calculate the position deviation corresponding to the first eye position.

[0047] Furthermore, it should be understood that if the positional deviation is 0, or the positional deviation is less than a preset threshold, the display module of this application will determine that the user's eye position has not changed, and will therefore no longer execute subsequent steps 0130, 0140 and others.

[0048] Furthermore, it should be understood that the specific method for determining / calculating the positional deviation is customizable based on actual circumstances. For example, in some implementations, the aforementioned 0120 specifically includes:

[0049] The positional deviation is obtained based on the difference between the first and second eye positions.

[0050] That is, let the position of the first eye be ΔNew, the position of the second eye be ΔLast, and the positional deviation be ΔVector, then we have:

[0051] ΔVector=ΔBew-ΔLast

[0052] 0130, the position of the first eye is corrected based on the position deviation to obtain the current observation position.

[0053] In other words, this application will correct the first eye position obtained at the current moment (this time) based on the deviation between the first eye position and the second eye position obtained at two different times (two times), thereby eliminating the error / deviation caused by hardware limitations.

[0054] It is understandable that the specific process of correcting / adjusting the position of the first eye based on the positional deviation is something that can be set according to the actual situation. For example, in some embodiments, the above-mentioned 0130 specifically includes:

[0055] The target correction amount is obtained based on the difference between the position deviation and the preset deviation correction amount;

[0056] The current observation position is obtained based on the sum of the first eye position and the target correction value.

[0057] To more clearly illustrate the implementation method provided in this application, let the position deviation be ΔVector, the deviation correction be ΔValue, the target correction be ΔVectorFinal, the current observation position / current viewpoint be P', and the first eye position be expressed as P(x, y, z), then we have:

[0058] ΔVectorFinal=ΔVector-ΔValue

[0059] P'=P(x,y,z)+ΔVectorFinal

[0060] It should be noted that the deviation correction amount ΔValue in the embodiments of this application can be data that has been fitted from multiple samples in advance. That is, firstly, multiple image samples including the eyes are acquired by the camera module; then, the position of the eyes in each image sample is calibrated to obtain the real eye position; subsequently, the eye position in each image sample is identified to determine the eye detection position; finally, the deviation correction amount ΔValue is obtained by fitting the real eye position and the eye detection position of each image sample.

[0061] Thus, in this embodiment, after determining the deviation between the current position of the first eye and the previous position of the second eye (i.e., the position deviation) by the position deviation amount, the position deviation amount can be corrected by the deviation correction amount, thereby obtaining the deviation between the first eye position and the true eye position (i.e., the current observation position) (i.e., the target correction amount). After correcting the first eye position by the target correction amount, a relatively accurate eye position, i.e., the current observation position, can be obtained. This reduces the negative impact of hardware cost and hardware limitations.

[0062] Furthermore, it should be understood that the position deviation ΔVector and deviation correction ΔValue in the above embodiments can both be understood as the directional offset between the first eye position and the second eye position. Therefore, if the user only changes the eye position with a small movement (such as tilting the head), the target correction ΔVectorFinal determined by the above embodiments is relatively accurate. However, if the user changes the eye position with a large movement (such as suddenly squatting or running), due to the aforementioned hardware limitations and the scale setting of the camera module, the position deviation ΔVector and deviation correction ΔValue will be difficult to accurately describe the true distance deviation between the first eye position and the actual eye position. In other words, the accuracy / reliability of the target correction ΔVectorFinal decreases. Therefore, in some embodiments of this application, the target correction is obtained based on the difference between the position deviation and the preset deviation correction, including:

[0063] Candidate correction values ​​are obtained based on the difference between the position deviation and the deviation correction.

[0064] The target correction is obtained by multiplying the candidate correction amount and the deviation correction amount.

[0065] To more clearly illustrate the implementation methods provided in this application, please refer to the following formula:

[0066] ΔVectorFinal=(ΔVector-ΔValue)*ΔValue

[0067] It is easy to understand that (ΔVector-ΔValue) in the formula is the candidate correction amount mentioned above.

[0068] Understandably, the scale refers to the actual distance in real / real space corresponding to a unit distance in an image captured by the camera module, such as how many meters in real space correspond to 1 pixel in an image captured by the camera module.

[0069] It can also be understood that the product of the candidate correction and the deviation correction is interpreted as the distance (ΔVector-ΔValue) that the user's eye has moved in the ΔValue direction. In other words, the target correction ΔVectorFinal is obtained by multiplying (ΔVector-ΔValue) by ΔValue. Thus, the target correction ΔVectorFinal can describe the changes in the user's eye position under large movements.

[0070] Thus, the implementation method of this application enables the display module to determine the relatively accurate deviation (i.e., target correction amount) between the first eye position and the actual eye position even after a significant change in the user's eye position, thereby further reducing the negative impact of hardware costs and hardware limitations.

[0071] 0140, based on the current observation position, simultaneously update the target images in the left and right halves of the display area to ensure that a three-dimensional image can be observed at the current observation position.

[0072] In other words, after the user's eye position changes, this application will use the changed position (i.e., the current observation position) to readjust the images displayed / presented in the left and right halves of the display area of ​​the display module, thereby ensuring that the user can observe a three-dimensional image after receiving the target image in the left half and the target image in the right half of the display area with the naked eye at the current moment.

[0073] It is easy to understand that the target images displayed in the left and right halves of the region are the same object. That is, this application displays two images of the same object generated from different viewing angles in the left and right halves of the region, respectively.

[0074] Thus, by determining the current observation position, this application enables the display module to adjust the image in the display area accordingly after the user's eye position changes, ensuring that the user can continue to observe the 3D image with the naked eye, thereby guaranteeing the user's observation experience. Furthermore, this application also obtains the current observation position by correcting the first eye position, avoiding the errors introduced by directly using the first eye position, and thus reducing the negative impact caused by hardware costs and hardware limitations.

[0075] Furthermore, it is understood that the manner in which the target image is displayed in the left and right halves of the display area of ​​this application is a matter that can be set according to actual conditions. For example, in some embodiments, the resolution of the target image is the same as that of the left half, the resolution height of the left half and the resolution height of the right half are the same as the resolution height of the display area, and the resolution width of the left half and the resolution width of the right half are both half the resolution width of the display area.

[0076] As an example, if the resolution of the display area of ​​the display module of this application is 8K (7680×4320), then the resolution height of the left and right half regions is 4320, and the resolution width of the left and right half regions is half of 7680, that is, 3840. Therefore, the resolution of the target image will be set to 3840×4320.

[0077] It is easy to understand that in the embodiments provided in this application, the display area will be evenly divided into two halves, and both halves will display the complete target image.

[0078] Thus, this application enables the target image to be displayed relatively easily in both halves of the target area, ensuring the display efficiency of the target image.

[0079] Furthermore, it is understood that the display area in this application can be understood as a virtual window used to display the target image, or as a monitor / display panel.

[0080] Furthermore, in some embodiments where the display area is the aforementioned virtual window, the resolution of the display area is always consistent with the resolution of the monitor. For example, assuming the resolution of the monitor in this application's display module is 8K (7680×4320), the resolution of the display area is also 8K, while the resolutions of the left half, right half, and target image can all be 3840×4320.

[0081] It should be noted that if the resolution of the display area does not match the resolution of the monitor, the target image may be displayed poorly, making it impossible for the user to observe the 3D image. For example, please refer to... Figure 2 , 3 and 4, Figure 2 , 3 Figures 4 and 5 are schematic diagrams illustrating application scenarios of certain embodiments of this application.

[0082] in, Figure 2 This illustrates a situation where the resolution of the display area is less than the resolution of the monitor. In other words, because the resolution of the display area is less than the resolution of the monitor, some areas of the display area do not display any content, resulting in black borders. Consequently, it is difficult for users to observe the 3D image.

[0083] Figure 3 This illustrates the case where the resolution of the display area is equal to the resolution of the monitor. That is, because the resolution of the display area is equal to the resolution of the monitor, both the left and right halves of the display area can display the target image completely, thus allowing the user to observe a three-dimensional image.

[0084] Figure 4 This illustrates a situation where the resolution of the display area is greater than the resolution of the monitor. In other words, because the resolution of the display area is greater than the resolution of the monitor, the right half of the area cannot fully display the target image. As a result, the user cannot observe the image in the right half of the area with their right eye, and therefore cannot observe the three-dimensional image.

[0085] Optionally, to make the image displayed in the display area more natural / smooth, in some embodiments, the three-dimensional image display method provided in this application further includes:

[0086] Acquire image samples containing eye positions taken at a preset frequency within a preset time period, wherein the eye positions are different in different image samples;

[0087] Based on all image samples, fit the positional changes of the eye position within a preset time period.

[0088] Furthermore, in some embodiments, the aforementioned 0110 specifically includes:

[0089] Get the user's eye position at the current moment to obtain the first position, and get the user's eye position at the previous moment as the second position;

[0090] Based on the positional change relationship, determine multiple intermediate positions between the previous moment and the current moment;

[0091] The second position, multiple intermediate positions, and the first position are merged in chronological order to obtain a set of positions;

[0092] Extract two adjacent elements from the position set in chronological order, and take the first element as the second eye position and the second element as the first eye position.

[0093] Perform the above 0120 and remove the second eye position from the position set.

[0094] Furthermore, following the aforementioned 0140, it also includes:

[0095] Returning to the steps described above, extract two adjacent elements from the position set in chronological order, and designate the first element as the second eye position and the second element as the first eye position, until the position set no longer contains two elements.

[0096] That is, the embodiments of this application will also utilize image samples collected by the camera module within a preset time period to fit the continuous change relationship of the user's eyes within the preset time period, i.e., the aforementioned position change relationship. Furthermore, after obtaining the first and second positions, the display module can regard the second position as the starting position of the eye and the first position as the ending position of the eye; then, using the position change relationship, multiple intermediate positions between the second position and the first position (i.e., from the previous moment to the current moment) are determined; subsequently, according to the chronological order, the second position, the multiple intermediate positions, and the first position are captured, sorted, and merged to obtain a position set; then, according to the chronological order, the first and second elements of the position set are extracted, and the first element is taken as the second eye position. The process involves taking the second element as the first eye position, determining the positional deviation using the first and second eye positions determined in the previous step, and then removing the first eye position from the position set to ensure that the element extracted from the position set in the next step will not be an element that has already been used or extracted. After updating the image in the display area using the current observation position, the above element extraction steps are performed again until the position set contains only one element, that is, only the aforementioned first position (eye endpoint position), at which point the next round of extraction is no longer performed.

[0097] It is easy to understand that, compared to directly using the current eye position and the previous eye position as the first eye position and the second eye position respectively, when the display module controls the image displayed in the display area according to the first eye position and the second eye position determined according to the above implementation method, the display module will update the image displayed in the display area at a higher frequency.

[0098] Therefore, even if the camera module in the display module fails to capture the user's eye position change from the previous moment to the current moment due to hardware limitations, the display module in this application embodiment can still simulate the eye position change from the previous moment to the current moment through prior knowledge (i.e., continuous change relationship). As a result, the target image in the display area can be updated as the eye position changes continuously.

[0099] Thus, the display module of this application embodiment can complete the adjustment of the target image in the display area in a smoother / natural manner, avoiding the situation that the update may be too drastic when the target image is updated directly using the current time and the previous time, thereby further ensuring the user's observation experience.

[0100] Furthermore, it should be noted that the preset time in the embodiments of this application is a content that can be set according to the actual situation. For example, in some embodiments, the preset time is the time from the "previous moment" to the "current moment"; while in other embodiments, the preset time is any 1 second when the shooting module is working normally.

[0101] It should also be noted that the preset frequency in the embodiments of this application is also a content that can be set according to the actual situation. In some embodiments, the preset frequency is 60Hz.

[0102] Furthermore, in some implementations, the image sample contains only the eye position of one user; while in other implementations, the image sample contains the eye positions of multiple users, each eye position corresponding to / associated with / bound to its respective user.

[0103] Furthermore, the method for obtaining / determining / calculating the positional change relationship in the embodiments of this application is also customizable according to actual circumstances. For example, in some embodiments, the above-mentioned fitting of the positional change relationship of the eye position within a preset time period based on all image samples includes:

[0104] Based on the horizontal axis coordinates of the eye position in each image sample, the first slope and the first intercept are calculated, and the horizontal axis position change relationship is determined based on the first slope and the first intercept.

[0105] Based on the vertical axis coordinates of the eye position in each image sample, the second slope and the second intercept are calculated, and the vertical axis position change relationship is determined based on the second slope and the second intercept.

[0106] The positional change relationship is obtained based on the vertical axis change relationship and the horizontal axis change relationship.

[0107] That is, the embodiments of this application will use the horizontal and vertical coordinates in each image sample to fit a straight line equation of the eye position to describe the changes / relationships of the eye position.

[0108] To more clearly illustrate the embodiments provided in this application, let the number of image samples be 60, and let the 60 image samples be (x1, y1, z2), (x2, y2, z2)...(x60, y60, z60). Let the first slope and second intercept of the horizontal axis be Kx and bx, respectively, and the second slope and second intercept of the vertical axis be Ky and by, respectively. Then we have:

[0109]

[0110] Thus, this application enables the positional changes of the eye position to be determined in a simpler way, thereby reducing the load on the display module.

[0111] In a specific instance, the X-axis, Y-axis, and Z-axis coordinates of the eye coordinates contained in each of the acquired image samples are shown in Table 1.

[0112] Table 1

[0113] 0 2.88 2.56 0.2 2.2576 3.445 0.7 1.9258 4.623 0.9 2.0862 5.253 0.92 2.109 5.663 0.99 2.1979 6.4786 1.2 2.5409 4.2596 1.4 2.9627 7.4458 1.48 3.155 7.9985 1.5 3.2052 8.2568

[0114] Therefore, the values ​​obtained by fitting the data in Table 1 are Kx = 1.0628, bx = 0.082, Ky = 1.2026, and by = 0.852.

[0115] Optionally, to adapt to different scenarios and users, the above 0140 specifically includes the following in some embodiments of this application:

[0116] Based on the current observation position and the received field of view, pupil distance, near-section position and far-section position information, the target three-dimensional model is mapped to two-dimensional space to obtain an updated target image;

[0117] At the same time, the updated target image is displayed in the left and right halves of the region so that a three-dimensional image can be observed at the current observation position.

[0118] In other words, the embodiments of this application provide a parameter customization interface, enabling users to adjust the display module according to their own scene (i.e., near and far section information) or their own physiological characteristics (i.e., field of view and interpupillary distance), so that the display module can control the display of the target image based on the user-defined parameters. Specifically, after the display module receives the user-defined field of view (FOV) information, interpupillary distance (OD) information, near section position information, and far section position information, it will control the display of the target image according to the above four types of information.

[0119] It's easy to understand that field of view information represents the range of the human eye's field of vision. When simulating the human eye using specific devices (such as robots with rotatable cameras), the field of view represents the range of vision for that specific device.

[0120] It's not hard to understand that interpupillary distance (IPD) refers to the distance between a person's left and right eyes.

[0121] It is also easy to understand that the near section and far section represent the sections closest to and farthest from the human eye / specific device, respectively. For a clearer explanation of the meaning of the near section position information and far section position information in the embodiments of this application, please refer to... Figure 5 , Figure 5 This is an illustration of an application scenario for some embodiments of this application. Figure 5In this context, eye represents the current observation position; np represents the near section, N represents the distance from the near section np to the current observation position eye; fp represents the far section, F represents the distance from the far section fp to the current observation position eye; p is the object to be observed, and p' is the position of the object to be observed p (perspective) projected onto the near section np.

[0122] It is not hard to understand that Figure 3 In the scenario shown, the user will observe object p' on the near section np from the current observation position eye.

[0123] It is not difficult to understand that Figure 5 In the scenario shown, after the object to be observed p(x, y, z) is projected onto p' of the near section np, its coordinates will change from (x, y, z) to (-N*x / z, -N*y / z, -N).

[0124] It is conceivable that the process of generating a 2D image that can be observed as a 3D image using the above four types is a setting that can be configured according to actual conditions. For example, in some embodiments of this application, the above-mentioned mapping of the target 3D model to a 2D space to obtain an updated target image based on the current observation position and received field of view information, pupil distance information, near-section position information, and far-section position information includes:

[0125] The first pixel matrix corresponding to the target 3D model is transformed to map the first pixel matrix to the world coordinate system, thus obtaining the second pixel matrix;

[0126] The second pixel matrix is ​​approximated to obtain the third pixel matrix;

[0127] Based on the current observation position and field of view information, pupil distance information, near-section position information and far-section position information, the third pixel matrix is ​​mapped to two-dimensional space to obtain the updated target image.

[0128] It should be noted that the target image in this embodiment is an image obtained by mapping a three-dimensional model (i.e., the target three-dimensional model) to a two-dimensional space. Therefore, this embodiment will perform model transformation, view transformation, and projection transformation on the target three-dimensional model to map the first pixel matrix corresponding to the target three-dimensional model in the local three-dimensional space to the second pixel matrix in the world space, and then map the second pixel matrix from the world space to the view space and the clipping space in sequence.

[0129] It should also be noted that before mapping the second pixel matrix to the target image in the embodiments of this application, the second pixel matrix will be subjected to equalization processing to avoid the field of view distortion problem that may occur when the elements in the second pixel matrix are projected into two-dimensional space, thereby ensuring the correct shape of the target three-dimensional model after projection.

[0130] In some implementations, this application achieves equivalence processing by orthogonally projecting the second pixel matrix.

[0131] Thus, by using the equivalent processing, this application ensures that the shape of the target 3D model remains unchanged after it is mapped to the target image, thereby guaranteeing the authenticity of the target image.

[0132] Optionally, in some embodiments of this application, the above-mentioned mapping of the third pixel matrix to a two-dimensional space based on the current observation position and the received field of view information, pupil distance information, near-section position information, and far-section position information to obtain an updated target image includes:

[0133] Based on the current observation position, field of view information, near-section position information, and far-section position information, the third pixel matrix is ​​mapped to the clipping space to obtain the fourth pixel matrix;

[0134] Based on interpupillary distance information, determine the monocular visual angle deviation value;

[0135] Based on the field of view information and monocular viewing angle deviation, the projection matrix of the fourth pixel matrix corresponding to the left and right half regions is determined.

[0136] The updated first target image is obtained by multiplying the fourth pixel matrix with the projection matrix of the left half region, and the updated second target image is obtained by multiplying the fourth pixel matrix with the projection matrix of the right half region.

[0137] Furthermore, the updated target image is simultaneously displayed in both the left and right halves of the region, so that a three-dimensional image can be observed at the current observation position, including:

[0138] The updated first target image is displayed in the left half of the region, and the updated second target image is displayed in the right half of the region, so that a three-dimensional image can be observed at the current observation position.

[0139] That is, after mapping the target 3D model to the clipping space to obtain the corresponding fourth pixel matrix, in order to reasonably display the fourth pixel matrix in the left and right half of the display area, the embodiments of this application also determine the difference between the left and right half of the user's single eye observation of the left and right half of the display area, i.e., the monocular visual angle deviation value, by using the interpupillary distance and the field of view.

[0140] In some implementations, the monocular visual angle deviation is half the interpupillary distance. For example, assuming an interpupillary distance of 60 mm and the origin at the midpoint between the left and right eyes, the left eye visual angle deviation ΔLoffset is -30 mm, and the right eye visual angle deviation ΔRoffset is +30 mm.

[0141] After determining the monocular viewing angle deviation value, the embodiments of this application will use the monocular viewing angle deviation value and the visible range of the human eye (field of view) to determine the display form of the fourth pixel matrix in the left and right half regions, so as to ensure that the fourth pixel matrix can be displayed completely and reasonably when it is displayed / projected in the left and right half regions.

[0142] In some implementations, the projection matrix TranMatri is:

[0143]

[0144] In the formula, X, Y, and NearZ represent the three-axis visible range of the field of view of a single eye, respectively. NearZ also represents... Figure 5 The distance N from the near-field section np to the current observation position eye; Δoffset represents the monocular visual angle deviation. It's easy to understand that the f in the matrix refers to a single-precision floating-point data type (float).

[0145] Therefore, if the fourth pixel matrix needs to be displayed in the display area, ΔLoffset and ΔRoffset are assigned to Δoffset respectively to obtain the projection matrix TranMatri1 and the projection matrix TranMatri2 corresponding to the left half of the region. Then, the product of the fourth pixel matrix and TranMatri1 (i.e., the updated first target image) is displayed in the left half of the region, and the product of the fourth pixel matrix and TranMatri2 (i.e., the updated second target image) is displayed in the right half of the region, thus completing the display of the target image.

[0146] Thus, the introduction of monocular viewing angle deviation value and projection matrix in this application enables the user's left and right eyes to correctly observe the target image in the left and right halves of the region, thereby further ensuring that the user can observe the three-dimensional target image at the current observation position.

[0147] Furthermore, it is understandable that if the matrix size of the product of the fourth pixel matrix and TranMatri1 (or TranMatri2) is smaller than the resolution of the left half region (or right half region), then the above product result can be interpolated to ensure that the target image can be fully displayed in the left half region (or right half region).

[0148] Additionally, it should be noted that the determination of X and Y in the aforementioned projection matrix TranMatri is a variable that can be set according to actual circumstances. For example, in some implementations, the calculation formulas for X and Y are as follows:

[0149]

[0150] In the formula, FOV represents the field of view; Clamp() indicates that the parameter in parentheses is clamped to limit the parameter within a preset range; Width and Height represent the resolution width and height of the display area, respectively.

[0151] In some implementations, to ensure that different display modules can use the implementations provided in this application, the clamp value range corresponding to the above Clamp() is 100 to 10000, so that display modules with resolutions from 1 to 8K can calculate X and Y using the above formula.

[0152] Optionally, if the embodiments provided in this application are applied to a robot simulation experiment that includes a human eye simulation device (such as a camera), after calculating the current observation position, the deflection angle sigma of the human eye simulation device can be calculated by the following formula so that the human eye simulation device can keep the observed object in the center of the field of vision.

[0153] sigma=acos(dot(PF,(P(x,y,z)+ΔVectorFinal)) / (norm(PF)*norm(P(x,y,z)

[0154] +ΔVectorFinal)))

[0155] In the formula, PF can be understood as a vector pointing from the first eye position to the far section, and P can be understood as the first eye position.

[0156] Optionally, in some embodiments, if one or more of the field of view information, pupil distance information, near-section position information, and far-section position information are not received, this application will use the default value to complete the display of the target image.

[0157] In some implementations, the inventors have experimentally verified that the visible range of the human eye is 60 to 90 degrees. The resulting shooting angle deviation when the field of view (FOV) is 60 to 90 degrees is shown in Table 2, namely:

[0158] Table 2

[0159] 90 1.852 85 1.961 80 2.083 75 2.225 70 2.381 65 2.564 60 2.778 45 3.704

[0160] Therefore, since the field of view is negatively correlated with the shooting angle deviation, in some embodiments, when the display module does not receive the field of view information, the field of view will be assigned a value of 90 degrees. In other embodiments, when the display module does not receive the field of view information, the field of view will be assigned any angle value between 60 and 90 degrees.

[0161] In some embodiments, the inventors' experiments showed that when the interpupillary distance is approximately 60mm, the image displayed in the display area is as follows: Figure 6 , 7 As shown in figures 8, 9, and 10. Among them, Figure 6 , 7 Figures 8, 9, and 10 show the display status of the display area when the interpupillary distance is (60-5.25000005)mm, (60-2.25000005)mm, 60mm, (60+2.25000005)mm, and (60+5.25000005)mm, respectively.

[0162] In other words, experiments have verified that a pupil distance of 60mm ensures a clear three-dimensional effect for the target image in the display area. Therefore, in some implementations, when the display module does not receive pupil distance information, the pupil distance can be assigned a value of 60mm. In other implementations, when the display module does not receive pupil distance information, the pupil distance is assigned any one of (60-5.25000005)mm, (60-2.25000005)mm, 60mm, (60+2.25000005)mm, and (60+5.25000005)mm.

[0163] In some embodiments, the inventors' experiments showed that the distance N between the near-section and the observation point was -0.03 (i.e., ...). Figure 3 When the distance is approximately 0.03 units in the negative Z-axis direction, the display effect of the display area is as follows: Figure 11 , 12 As shown in figures 13, 14, and 15. Among them, Figure 11 , 12 13, 14, and 15 represent the display status of the display area when the distance N between the near section and the observation point is -1.0, -0.05, -0.03, -0.01, and 0.01, respectively.

[0164] In other words, experimental verification has shown that when the distance N between the near-section and the observation point is -0.03, users are less likely to experience 3D dizziness and can observe the three-dimensional target image more clearly. Therefore, in some implementations, when the display module does not receive near-section information, the distance N between the near-section and the observation point will be assigned a value of -0.03. In other implementations, the distance N between the near-section and the observation point will be assigned any one of -1.0, -0.05, -0.03, -0.01, and 0.01.

[0165] refer to Figure 16 This application also provides a three-dimensional image display device 200, comprising:

[0166] The acquisition module 210 is used to acquire the user's eye position at the current moment as the first eye position, and acquire the user's eye position at the previous moment as the second eye position.

[0167] The determining module 220 is used to determine the positional deviation based on the first eye position and the second eye position;

[0168] The correction module 230 is used to correct the position of the first eye based on the position deviation to obtain the current observation position;

[0169] The update module 240 is used to update the target image in the left half of the display area and the target image in the right half of the display area based on the current observation position, so that a three-dimensional image can be observed at the current observation position.

[0170] In some implementations, the resolution of the target image is the same as that of the left half of the region, the resolution height of the left half of the region and the resolution height of the right half of the region are the same as the resolution height of the display area, and the resolution width of the left half of the region and the resolution width of the right half of the region are both half the resolution width of the display area.

[0171] In some implementations, the determining module 220 is further configured to obtain a positional deviation based on the difference between the first eye position and the second eye position.

[0172] In some embodiments, the calibration module 230 includes:

[0173] The first calculation submodule is used to obtain the target correction amount based on the difference between the position deviation and the preset deviation correction amount;

[0174] The second calculation submodule is used to obtain the current observation position based on the sum of the first eye position and the target correction.

[0175] In some implementations, the first computing submodule includes:

[0176] The difference calculation unit is used to obtain candidate correction amounts based on the difference between the position deviation and the deviation correction amount;

[0177] The product calculation unit is used to obtain the target correction amount based on the product of the candidate correction amount and the deviation correction amount.

[0178] In some embodiments, the three-dimensional display device 200 for images of this application further includes:

[0179] The sample acquisition module is used to acquire image samples containing eye positions taken at a preset frequency within a preset time period, wherein the eye positions are different in different image samples;

[0180] The fitting module is used to fit the positional changes of the eye position over a preset time period based on all image samples.

[0181] Furthermore, the acquisition module 210 includes:

[0182] The location acquisition submodule is used to obtain the user's eye position at the current moment to get the first position, and to obtain the user's eye position at the previous moment as the second position;

[0183] The intermediate position submodule is used to determine multiple intermediate positions between the previous time and the current time based on the position change relationship;

[0184] The merge submodule is used to merge the second position, multiple intermediate positions, and the first position in chronological order to obtain a set of positions.

[0185] The extraction submodule is used to extract two adjacent elements from the position set in chronological order, and take the first element as the second eye position and the second element as the first eye position.

[0186] The deletion submodule is used to perform the step of determining the positional deviation based on the first eye position and the second eye position, and then deleting the second eye position from the position set.

[0187] Furthermore, the update module 240 also includes:

[0188] Return to the submodule, which is used to go back to the step of extracting two adjacent elements in the position set in chronological order, and taking the first of the two elements as the second eye position and the second as the first eye position, until the position set no longer contains two elements.

[0189] In some implementations, the fitting submodule includes:

[0190] The first relationship calculation unit is used to calculate the first slope and the first intercept based on the horizontal axis coordinates of the eye position in each image sample, and to determine the horizontal axis position change relationship based on the first slope and the first intercept.

[0191] The second relationship calculation unit is used to calculate the second slope and the second intercept based on the vertical axis coordinate of the eye position in each image sample, and to determine the vertical axis position change relationship based on the second slope and the second intercept.

[0192] The change relationship is obtained as a unit, which is used to obtain the position change relationship based on the vertical axis change relationship and the horizontal axis change relationship.

[0193] In some implementations, the update module 240 includes:

[0194] The mapping submodule is used to map the target 3D model to 2D space based on the current observation position and the received field of view angle information, pupil distance information, near section position information and far section position information to obtain an updated target image;

[0195] The display submodule is used to simultaneously display the updated target image in both the left and right halves of the region, so that a three-dimensional image can be observed at the current observation position.

[0196] In some implementations, the mapped submodule includes:

[0197] The model transformation unit is used to transform the first pixel matrix corresponding to the target 3D model to map the first pixel matrix to the world coordinate system and obtain the second pixel matrix.

[0198] An iso-scaling unit is used to perform iso-scaling on the second pixel matrix to obtain the third pixel matrix;

[0199] The conversion unit is used to map the third pixel matrix to a two-dimensional space to obtain an updated target image based on the current observation position and field of view information, pupil distance information, near-section position information and far-section position information.

[0200] In some embodiments, the conversion unit includes:

[0201] The spatial mapping sub-unit is used to map the third pixel matrix to the clipping space based on the current observation position, field of view information, near-section position information and far-section position information to obtain the fourth pixel matrix;

[0202] The deviation value determination subunit is used to determine the monocular visual angle deviation value based on the pupillary distance information;

[0203] The matrix determination sub-unit is used to determine the projection matrix of the fourth pixel matrix corresponding to the left and right half regions based on the field of view information and monocular viewing angle deviation;

[0204] The image is divided into sub-units, which are used to obtain an updated first target image based on the product of the fourth pixel matrix and the projection matrix of the left half region, and to obtain an updated second target image based on the product of the fourth pixel matrix and the projection matrix of the right half region.

[0205] Furthermore, the display submodule is also used to display the updated first target image in the left half of the region and simultaneously display the updated second target image in the right half of the region, so that a three-dimensional image can be observed at the current observation position.

[0206] The three-dimensional image display device 200 provided in this application embodiment can implement each process of the above-described three-dimensional image display method and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0207] This application also provides a display module, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the three-dimensional display method of the image of this application.

[0208] This application also provides a computer-readable storage medium containing a computer program. When the computer program is executed by one or more processors, it causes the one or more processors to perform a three-dimensional display method for images of this application.

[0209] In the description of this specification, the references to terms such as "some embodiments," "in one example," "exemplarily," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0210] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0211] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for displaying a three-dimensional image, characterized in that, The method includes: The user's current eye position is obtained as the first eye position, and the user's previous eye position is obtained as the second eye position. Based on the first eye position and the second eye position, determine the positional deviation. Based on the difference between the position deviation and the preset deviation correction, a target correction is obtained. The deviation correction is used to indicate the relationship between the detected eye position and the actual eye position. The deviation correction is determined by multiple eye detection positions and the actual eye position corresponding to each eye detection position. The target correction is used to indicate the deviation between the current eye position and the user's actual eye position. The current observation position is obtained based on the sum of the first eye position and the target correction value; Based on the current observation position, the target images in the left half and right half of the display area are updated accordingly, so that a three-dimensional image can be observed at the current observation position. The determination of the positional deviation based on the first eye position and the second eye position includes: The positional deviation is obtained based on the difference between the first eye position and the second eye position.

2. The three-dimensional image display method according to claim 1, characterized in that, The resolution of the target image is the same as that of the left half region. The resolution height of the left half region and the resolution height of the right half region are both the same as the resolution height of the display area. The resolution width of the left half region and the resolution width of the right half region are both half the resolution width of the display area.

3. The three-dimensional display method for images according to claim 1, characterized in that, The method of obtaining the target correction amount based on the difference between the position deviation and the preset deviation correction amount includes: Based on the difference between the position deviation and the deviation correction, a candidate correction amount is obtained; The target correction amount is obtained by multiplying the candidate correction amount and the deviation correction amount.

4. The three-dimensional display method of an image according to claim 1, characterized in that, The method further includes: Acquire image samples containing eye positions taken at a preset frequency within a preset time period, wherein the eye positions are different in different image samples; Based on all the image samples, fit the positional change relationship of the eye position within the preset time period; The step of obtaining the user's eye position at the current moment as the first eye position and obtaining the user's eye position at the previous moment as the second eye position includes: The user's eye position at the current moment is obtained to obtain a first position, and the user's eye position at the previous moment is obtained as a second position. Based on the aforementioned positional change relationship, multiple intermediate positions between the previous moment and the current moment are determined; The second position, the plurality of intermediate positions, and the first position are merged in chronological order to obtain a position set; According to the time sequence, two adjacent elements in the location set are extracted, and the first of the two elements is taken as the second eye position and the second as the first eye position. Perform the step of determining the positional deviation based on the first eye position and the second eye position, and delete the second eye position from the position set; After updating the target image in both the left and right halves of the display area based on the current observation position, so that a three-dimensional target image can be observed at the current observation position, the method further includes: Returning to the step of extracting two adjacent elements from the position set according to the time sequence, and taking the first of the two elements as the second eye position and the second as the first eye position, until the position set no longer contains two elements.

5. The three-dimensional display method for images according to claim 4, characterized in that, The step of fitting the positional change relationship of the eye position within the preset time period based on all the image samples includes: Based on the horizontal axis coordinates of the eye position in each image sample, a first slope and a first intercept are calculated, and the horizontal axis position change relationship is determined based on the first slope and the first intercept. Based on the vertical axis coordinates of the eye position in each image sample, a second slope and a second intercept are calculated, and the vertical axis position change relationship is determined based on the second slope and the second intercept. The position change relationship is obtained based on the vertical axis position change relationship and the horizontal axis position change relationship.

6. The three-dimensional display method of an image according to claim 1, characterized in that, The step of simultaneously updating the target image in the left half and right half of the display area based on the current observation position, so that a three-dimensional image can be observed at the current observation position, includes: Based on the current observation position and the received field of view, pupil distance, near-section position information and far-section position information, the target three-dimensional model is mapped to two-dimensional space to obtain the updated target image; Simultaneously, the updated target image is displayed in the left and right halves of the region, so that a three-dimensional image can be observed at the current observation position.

7. The three-dimensional display method for images according to claim 6, characterized in that, The step of mapping the target 3D model to 2D space to obtain the updated target image based on the current observation position and the received field of view angle information, pupil distance information, near-section position information, and far-section position information includes: The first pixel matrix corresponding to the target 3D model is transformed to map the first pixel matrix to the world coordinate system to obtain the second pixel matrix; The second pixel matrix is ​​subjected to equalization processing to obtain the third pixel matrix; Based on the current observation position, the field of view information, the pupil distance information, the near-section position information, and the far-section position information, the third pixel matrix is ​​mapped to the two-dimensional space to obtain the updated target image.

8. The three-dimensional display method for an image according to claim 7, characterized in that, The step of mapping the third pixel matrix to the two-dimensional space based on the current observation position, the field of view information, the interpupillary distance information, the near-section position information, and the far-section position information to obtain the updated target image includes: Based on the current observation position, the field of view information, the near-section position information, and the far-section position information, the third pixel matrix is ​​mapped to the clipping space to obtain the fourth pixel matrix; Based on the interpupillary distance information, the monocular visual angle deviation value is determined; Based on the field of view information and the monocular viewing angle deviation, the projection matrix of the fourth pixel matrix corresponding to the left half region and the right half region is determined; An updated first target image is obtained by multiplying the fourth pixel matrix with the projection matrix of the left half region, and an updated second target image is obtained by multiplying the fourth pixel matrix with the projection matrix of the right half region. Simultaneously displaying the updated target image in both the left and right halves of the region, so that a three-dimensional image can be observed at the current observation position, includes: The updated first target image is displayed in the left half of the region, and the updated second target image is displayed in the right half of the region, so that a three-dimensional image can be observed at the current observation position.

9. A three-dimensional image display device, characterized in that, The device includes: The acquisition module is used to acquire the user's eye position at the current moment as the first eye position, and acquire the user's eye position at the previous moment as the second eye position. The determining module is used to determine the positional deviation based on the first eye position and the second eye position; The correction module is used to obtain a target correction amount based on the difference between the position deviation amount and the preset deviation correction amount, and to obtain the current observation position based on the sum of the first eye position and the target correction amount; The update module is used to update the target image in the left half of the display area and the target image in the right half of the display area simultaneously based on the current observation position, so that a three-dimensional image can be observed at the current observation position. The determining module is further configured to obtain the position deviation amount based on the difference between the first eye position and the second eye position.

10. A display module, characterized in that, The display module includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the method of any one of claims 1-8.

Images