Light field display method and apparatus

Abstract

The present disclosure provides a light field display method and device. The display method comprises: obtaining light field data; wherein the light field data comprises a plurality of viewpoint images; the viewpoint image comprises a pixel weight parameter; processing the viewpoint image by using an image processing model, determining a target pixel and a non-target pixel according to the processing result; and updating the value of the pixel weight parameter of the target pixel; and determining the calculation order of the light transmittance of each pixel point of each liquid crystal layer of a spatial light modulation array and the value of the light transmittance according to each viewpoint image and the value of the pixel weight parameter of each pixel after being processed by the image processing model. Such a method helps to improve the calculation efficiency and the picture quality of light field display.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of display, in particular to a light field display method and device. BACKGROUND

[0002] Compared with two-dimensional display, three-dimensional display can provide deeper immersion. At present, light field display mainly includes volumetric three-dimensional display, integral imaging, multi-projection array and multi-layer liquid crystal display method, wherein the multi-layer liquid crystal display technology takes into account maturity and cost. SUMMARY

[0003] Therefore, the purpose of the present disclosure is to provide a light field display method and device.

[0004] To achieve the above purpose, in a first aspect, the present disclosure provides a light field display method, which comprises:

[0005] acquiring light field data; wherein the light field data comprises a plurality of viewpoint images; the viewpoint images comprise pixel weight parameters;

[0006] processing the viewpoint images by using an image processing model, determining target pixels and non-target pixels according to the processing results, and updating the values of the pixel weight parameters of the target pixels;

[0007] determining the calculation order of the light transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array and the values of the light transmittance according to the values of the pixel weight parameters corresponding to each pixel of each viewpoint image and each pixel after being processed by the image processing model.

[0008] In a second aspect, the present disclosure provides a light field display device, which comprises a spatial light modulation array and a processor; the spatial light modulation array comprises at least two liquid crystal layers; the liquid crystal layers comprise arrayed pixel points;

[0009] The processor is configured to perform the following steps:

[0010] acquiring light field data; wherein the light field data comprises a plurality of viewpoint images; the viewpoint images comprise pixel weight parameters;

[0011] processing the viewpoint images by using an image processing model, determining target pixels and non-target pixels according to the processing results, and updating the values of the pixel weight parameters of the target pixels;

[0012] determining the calculation order of the light transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array and the values of the light transmittance according to the values of the pixel weight parameters corresponding to each pixel of each viewpoint image and each pixel after being processed by the image processing model.

[0013] As can be seen from the above, the light field display method and device provided by the present disclosure, by image processing, different pixel weights are assigned to the pixels of the view point image, and the calculation order of the light transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array and the numerical value of the light transmittance are determined based on the pixel weights, which helps to improve the calculation efficiency and the picture quality of the light field display. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the present disclosure or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Figure 1 A flowchart of a light field display method provided by an embodiment of the present disclosure is shown in the figure.

[0016] Figure 2 A part of the light field data provided by an embodiment of the present disclosure is shown in the figure.

[0017] Figure 3 A scene diagram for determining a target view image provided by an embodiment of the present disclosure is shown in the figure.

[0018] Figure 4 An effect diagram of a background removal model provided by an embodiment of the present disclosure is shown in the figure.

[0019] Figure 5 An effect diagram of a frame recognition model provided by an embodiment of the present disclosure is shown in the figure.

[0020] Figure 6 A diagram for deriving a pixel according to a light ray provided by an embodiment of the present disclosure is shown in the figure.

[0021] Figure 7 A flowchart of another light field display method provided by an embodiment of the present disclosure is shown in the figure.

[0022] Figure 8 A structure diagram of a light field display device provided by an embodiment of the present disclosure is shown in the figure. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, the present disclosure will be further described in detail below with reference to specific embodiments and drawings.

[0024] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the embodiments of the present disclosure shall have the common meaning understood by one of ordinary skill in the art to which the present disclosure belongs. The terms "first", "second", and similar terms used in the embodiments of the present disclosure do not denote any order, quantity, or importance, but are used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0025] The existing multi-layer liquid crystal display technology solves the problem of light field display by the following scheme: the light rays are modulated by the different light transmittances of each pixel at different angles in each layer of liquid crystal, and the light field is reconstructed by calculating the pixel values of each layer of liquid crystal from the light field data by a data decomposition algorithm.

[0026] For the technical scheme of light field display by multi-layer liquid crystal and algorithmic reconstruction of light field, the more layers of liquid crystal, the lower the overall light transmittance, and the fewer layers of liquid crystal, the higher the algorithmic complexity. Therefore, this scheme has the problem of being difficult to balance image brightness and quality. Specifically, when the number of layers of liquid crystal is small, the light rays are distributed to more pixel points, so it is difficult to optimize and reconcile the light rays at different angles when the light field is reconstructed by the algorithm, resulting in poor display quality.

[0027] Based on this, the present disclosure provides a light field display method. By image processing, different pixel weights are assigned to the pixels of the viewpoint image, the calculation order of the light transmittance of each pixel point of each layer of liquid crystal of the spatial light modulation array and the numerical value of the light transmittance are determined based on the pixel weights, and the light transmittance of the corresponding pixel point is locked when the light transmittance of any pixel point is determined. This way can ensure that the light transmittance of the pixel point through which the light ray corresponding to the pixel with high pixel weight in the viewpoint image is calculated and locked in priority, ensuring that the reconstructed light ray meets its display effect in the viewpoint image, thereby helping to improve the display quality. In addition, if the pixel weight of a pixel is as low as zero, it means that the light field decomposition calculation is not needed for the light ray corresponding to the pixel, thereby reducing the total amount of light field data that needs to be decomposed, thereby helping to improve the calculation efficiency.

[0028] Please refer to Figure 1 and Figure 7 , the light field display method comprises:

[0029] Step S101: acquiring light field data; wherein the light field data comprises a plurality of viewpoint images; the viewpoint image comprises a pixel weight parameter.

[0030] It should be understood that the light field is a complete representation of a set of light rays in space, including psychovisual perception information and physiological visual perception information, and collecting and displaying the light field can visually reproduce the real world.

[0031] Here, using the light field data, the real world can be visually reproduced.

[0032] The viewpoint image refers to the image information of a single angle in the light field. As can be understood by those skilled in the art, a user can view the light field display screen at multiple angles, and therefore the light field data includes multiple viewpoint images V x ×V y , for example, 7x7, and the resolution of each viewpoint image is R x ×R y (as shown in Figure 2 ).

[0033] Assuming that the light field data is 7x7x1080x720 and the liquid crystal layer pixel is 1080x720, the average load of each pixel is initially:

[0034]

[0035] That is, each pixel needs to consider 16.33 light rays at the same time when decomposing the light field data, and therefore it is inevitable that the light rays will conflict with each other when solving each light ray, which is the main reason for the low quality of the reconstructed light field. Wherein, N represents the number of liquid crystal layers.

[0036] Alternatively, the initial weight value of the pixel weight parameter of each pixel is equal, for example, it can be 1. At this time, the pixel weight of each pixel in the viewpoint image is the same, and the importance in the calculation of light field decomposition is also consistent. It should be understood that the values of the pixel weight parameters of each pixel can also be different, which is not specifically limited here.

[0037] In some embodiments, as shown in Figure 3 , the edge perception data is obtained by using the edge perception device 301, the light field data is filtered based on the edge perception data, and the target viewpoint image 302 is determined. It should be noted that the edge perception device 301 is used to determine the position of the user, that is, the specific position of the user relative to the spatial light modulation array can be determined through the edge perception device 301.

[0038] For example, as shown in Figure 3 , the number of original viewpoint images is 7x7, and if there are two users, the number of target viewpoint images after filtering is 2x5. As can be understood by those skilled in the art, the viewpoint images other than the target viewpoint images are invalid viewpoint images and do not participate in the light field decomposition calculation and light field reconstruction.

[0039] Here, the edge sensor 301 can be a camera or a laser radar, which is not specifically limited here. As can be understood by those skilled in the art, the edge sensing data obtained is also correspondingly different depending on the edge sensor 301. For example, the data corresponding to the camera is a camera image; the data corresponding to the laser radar is distance data, etc.

[0040] Such a technical solution determines the target view point image through the edge sensing data, and in subsequent determination of the light transmittance of each pixel point in each liquid crystal layer in the spatial light modulation array, only the light of the target view point image object needs to be considered, which can greatly reduce the calculation amount of light field decomposition. That is, since the invalid view point image is not observed by the user, the corresponding light is not considered when calculating the light transmittance of each pixel point.

[0041] In addition, since the number of view point images loaded by each pixel point is reduced, the light field is reconstructed, which is convenient for optimizing the harmonization of light rays at each angle, and also helps to improve the display quality.

[0042] Further, the edge sensing data includes a camera image; the camera image includes a plurality of regions and each region matches a view point image; and the display method includes:

[0043] Using a face recognition model, a face in the camera image is recognized and the region of the face in the camera image is determined; here, the face recognition model can use a mature face recognition model in the prior art, which is not listed here.

[0044] According to the region, the view point images corresponding to the region and the surrounding regions are determined as target view point images 302.

[0045] For example, please refer to Figure 3 The specific range of the surrounding region can be one view point image above, below, left and right of the region where the face is located. Considering the horizontal movement of the user, the view point images in the left and right directions can also be increased to five.

[0046] Step S103: processing the view point images using an image processing model, determining target pixels and non-target pixels according to the processing result, and updating the value of the pixel weight parameter of the target pixels.

[0047] It should be noted that if the edge sensing data is obtained by using the edge sensor 301, and the target view point images are determined by filtering the light field data based on the edge sensing data, the view point images in this step can be target view point images. Such a way is advantageous to reduce the calculation amount of image processing, and also helps to reduce the light decomposition calculation amount and improve the display quality.

[0048] It should be understood that, after being processed by the image processing model, the pixels in the view point image that meet the image processing model screening are target pixels, and the pixels that do not meet the requirements are non-target pixels.

[0049] For example, the image processing model includes a background removal model. Here, the background removal model is obtained by pre-training. For the specific training method of the background removal model, neural networks, GrabCut algorithms, etc. can be used, which are not limited here.

[0050] As shown in Figure 4 , based on the background removal model, the background pixels in the view point image are determined as the first target pixels; the value of the pixel weight parameter corresponding to the first target pixel is reduced.

[0051] It should be understood that, for the background pixels in the view point image, their importance for realizing light field display is low, so their updated pixel weight parameter can be 0, and at this time the light field data can be represented as L(2, 5, 1080, 480) for example. That is, the light corresponding to the background pixels can not participate in light field decomposition.

[0052] For example, as shown in Figure 5 , the image processing model includes a bounding box recognition model, based on the bounding box recognition model, the bounding box pixels in the view point image are determined as the second target pixels; the value of the pixel weight parameter corresponding to the second target pixel is increased.

[0053] Here, the bounding box recognition model can be a Hough transform, or a model trained by a neural network, a bounding box regression algorithm, which is not limited here.

[0054] Generally, the bounding box of an object plays an important role in the visual effect of the object, and its importance for realizing light field display is higher than that of general pixels, so the value of its pixel weight parameter can be 2, so as to participate in light field decomposition preferentially.

[0055] Since users usually pay more attention to text content, the image processing model can also include a text recognition model. Here, the text recognition model can be obtained by training a neural network, a deep learning model, which is not specifically limited here. Of course, those skilled in the art can understand that the text recognition model here can also be an open source mature text recognition model.

[0056] Based on the text recognition model, the text pixels in the view point image are determined as the third target pixels; the value of the pixel weight parameter corresponding to the third target pixel is increased. It should be noted that the importance of text is usually greater than that of the bounding box, so the pixel weight can be 3.

[0057] As can be seen from the above, for different target pixel types, due to their different importance to the effect of light field display, appropriate pixel weights can be assigned according to the importance to distinguish them from general pixels.

[0058] It should be noted that the specific type of the above image processing model is only an example and is not limited.

[0059] Step S105: According to the values of the pixel weight parameters corresponding to each pixel in each of the viewpoint images and after the image processing model processing, the calculation order and the transmittance value of each pixel point of each liquid crystal layer of the spatial light modulation array are determined; and in response to the transmittance of any of the pixel points being determined, the transmittance of the corresponding pixel point is locked. Here, locking means that during the process of determining the transmittance value of each pixel point of each liquid crystal layer of the spatial light modulation array, the transmittance of the pixel point remains unchanged.

[0060] In this way, the calculation order of the transmittance of a plurality of pixel points is determined based on the values of the pixel weight parameters, which can ensure that the transmittance of the pixel points through which the light corresponding to the pixel with high pixel weight in the viewpoint image is calculated and locked in priority, ensuring that the reconstructed light meets its display effect in the viewpoint image, thereby helping to improve the display quality. In addition, the pixel weight of a pixel being as low as zero indicates that the light corresponding to the pixel does not need to be calculated by light field decomposition, thereby reducing the total amount of light field data that needs to be decomposed by light field, thereby helping to reduce the amount of calculation.

[0061] In some embodiments, the step of determining the calculation order and the transmittance value of each pixel point of each liquid crystal layer of the spatial light modulation array according to the values of the pixel weight parameters corresponding to each pixel in each of the viewpoint images and after the image processing model processing, specifically comprises:

[0062] According to the values of the pixel weight parameters, the light ray weight of the light ray corresponding to each pixel in each of the viewpoint images is determined.

[0063] As can be understood by those skilled in the art, each pixel in the viewpoint image corresponds to a light ray, and the viewpoint image is perceived by the light ray.

[0064] For example, if the value of the pixel weight parameter is 1, the light ray weight of the corresponding light ray is 1; if the value of the pixel weight parameter is 0, the light ray weight of the corresponding light ray is 0; and if the value of the pixel weight parameter is 2, the light ray weight of the corresponding light ray is 2.

[0065] According to the preset light field decomposition algorithm, the light ray and its light ray weight, the calculation order and the transmittance value of each pixel point in each liquid crystal layer are determined; wherein the calculation order is configured to be determined according to the maximum value of the light ray weight of each light ray passing through the pixel point.

[0066] Here, the preset light field decomposition algorithm can be a light field display method based on non-negative matrix decomposition, which is not limited here.

[0067] As shown in Figure 6 , a pixel point will have multiple light rays passing through. The number of light rays of most pixel points can be determined according to the number of viewpoint images. Each light ray passes through each liquid crystal layer, and the final display effect is determined by the synthesis of each pixel point it passes through. It should be noted that due to the limitation of the range of light field display, the number of light rays of some pixel points can be less than the number of viewpoint images.

[0068] Therefore, in order to ensure the display effect of the light field, the calculation order of the light transmittance of the pixel point is determined by the maximum value in the light ray weight of each light ray passing through the pixel point, so that the pixel point through which any important light ray can be processed preferentially, which is beneficial to improve the display effect.

[0069] That is, if the light ray weight of each light ray passing through a pixel point is low, it indicates that the pixel point is not important for light field decomposition, and does not need to be processed preferentially, or even does not participate in light field decomposition.

[0070] In some embodiments, the step of determining the calculation order and the value of the light transmittance of each pixel point in each liquid crystal layer according to the preset light field decomposition algorithm, the light rays and their light ray weights, specifically comprises:

[0071] Comparing the light ray weights of each light ray passing through the pixel point, the maximum light ray weight is determined as the order value of the light field decomposition of the pixel point.

[0072] Figure 3 The selected 10 viewpoints in Figure 6 are displayed in the spatial light modulation array. Exemplarily, the light rays passing through the pixel point a are ten, and their light ray weights are 0, 1, 2, 1, 2, 0, 1, 2, 1, 2 respectively. The corresponding order value is 2.

[0073] Exemplarily, the light rays passing through the pixel point b (not marked in the figure) in the B liquid crystal layer in Figure 6 are ten, and their light ray weights are 0, 0, 1, 1, 1, 3, 0, 1, 1, 1 respectively. The corresponding order value is 3.

[0074] Exemplarily, the light rays passing through the pixel point c (not marked in the figure) in the C liquid crystal layer in Figure 6 are ten, and their light ray weights are 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 respectively. The corresponding order value is 0.

[0075] If the number of light rays passing through a pixel point is zero, it means that the pixel point does not need to be decomposed, and thus the transmittance of the pixel point can be directly set to 100%.

[0076] The light field decomposition sequence values of the pixel points are sorted in descending order, and the corresponding calculation sequences are matched according to the sorting result.

[0077] It should be noted that the pixel points with the sorting result in the front have the calculation sequence in the front, and the pixel points with the sorting result in the back have the calculation sequence in the back.

[0078] It should be understood that the pixel points with the same light field decomposition sequence value have the same bit order. That is, the pixel points with the light field decomposition sequence value of 2 have the same calculation sequence.

[0079] For example, there are 10 pixel points, and the corresponding light field decomposition sequence values are 1, 2, 2, 3, 2, 1, 0, 1, 2, and 2 respectively. The sorting result is 3, (2, 2, 2, 2, 2), (1, 1, 1), and 0. The matched calculation sequence is to calculate the pixel points corresponding to 3 first, then the pixel points corresponding to 2, 2, 2, 2, 2, then the pixel points corresponding to 1, 1, 1, and finally the pixel point corresponding to 0.

[0080] Based on the preset light field decomposition algorithm, the calculation sequence, and the light rays, the transmittances of the pixel points are sequentially determined; wherein the calculation sequence of the transmittances of the pixel points in the same sequence is randomly determined.

[0081] Optionally, in response to the sequence value corresponding to the pixel point in the last sequence being zero, the light field decomposition calculation is directly terminated. Here, according to the sequence value being zero, it can be determined that the light ray weight is zero, which means that each light ray passing through the pixel point is not important for light field decomposition, and thus the light field decomposition is not needed. Optionally, the transmittance of the corresponding pixel point can be directly set to 100%.

[0082] In some embodiments, the number of liquid crystal layers is three, and the preset light field decomposition algorithm includes the calculation formula of formula (1):

[0083]

[0084] wherein L j is the jth light ray passing through the pixel point a of any liquid crystal layer; b j and c j are the intersection pixel points of the light ray L j and the other two liquid crystal layers except the liquid crystal layer where the pixel point a is located; V is the number of light rays passing through the pixel point a; is the jth reconstructed light ray, and a is the pixel transmittance before updating, and a' is the pixel transmittance after updating.

[0085] The transmittance of each pixel point of each liquid crystal layer is randomly initialized, and a' is iteratively calculated according to formula (1) until a predetermined ending rule is met, and the corresponding a' is the transmittance of the pixel point a and is locked.

[0086] Optionally, the predetermined ending rule is that the absolute value of the difference between the transmittance of the pixel before updating and the transmittance of the pixel after updating is minimum.

[0087] It should be understood that in order to calculate the transmittance of a certain pixel point, formula (1) is iteratively calculated.

[0088] For example, the number of view images screened by means of edge perception data can be 10, and the number of un-screened view images can be 49.

[0089] It should be noted that the value of V is usually equal to the number of view images. Due to the limitation of the range of light field display, the number of light rays of some pixel points can be less than the number of view images, for example, the edge part of the liquid crystal layer close to the user side.

[0090] Here, the transmittance of each pixel point is randomly initialized when it is calculated for the first time. When the transmittance of a certain pixel point is determined and locked, the locked transmittance is directly used in the calculation of formula (1) in the process of calculating the transmittance of the remaining pixel points. In this way, the transmittance of the remaining pixel points can be determined under the condition that the display effect of the light emitted by the pixel point is calculated first, so that the transmittance value of the pixel point is more accurate, which is beneficial to obtain better display quality.

[0091] Optionally, in combination with Figure 6 The disclosure also exemplarily illustrates the derivation process of formula (1) as follows:

[0092] The three liquid crystal layers are represented as A layer, B layer and C layer respectively. Taking the A layer as an example, the pixel point a thereon is taken as an example.

[0093]

[0094] The pixel value is updated by differentiation:

[0095]

[0096]

[0097] Substituting into, we get:

[0098]

[0099] L j is the jth light ray passing through the pixel point a; bj and c j is the light ray L j is the intersection pixel point of the light ray L is the jth reconstructed light ray and a is the pixel transmittance before updating, and a' is the pixel transmittance after updating.

[0100] Step S107: Based on the transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array, the spatial light modulation array is controlled to display the reconstructed light field.

[0101] Since a single liquid crystal pixel can correspond to multiple light rays, for example, the reconstructable light rays of a 1080x720 pixel image are within the range of (1080x720) 2 In the actual reconstruction process, in order to reconcile the mutual conflicts between the light rays, the number is often much smaller than this number. According to the present disclosure, the viewpoint image is optimized by using the edge sensor and the image processing model, and the total number of reconstructed light rays after optimization is assumed to be 2x5x720x480, and the average load of each pixel is:

[0102]

[0103] where w represents the number of horizontal pixel points in the liquid crystal layer, and h represents the number of vertical pixel points in the liquid crystal layer.

[0104] It should be noted that the reduction in the number of viewpoint images is achieved by the edge sensor, and the reduction in the number of pixel points in the viewpoint image is achieved by the image processing model, such as a background removal model.

[0105] As can be seen, the optimization of the light field display method of the present disclosure for light field reconstruction can greatly reduce the effective load, thereby improving the output image quality and helping to improve the immersion of the three-dimensional effect.

[0106] In some embodiments, the light field display method obtains light field data, for example, the number of viewpoints is 7x7, and the resolution of the viewpoint image is 1080x720. Optionally, the light field display method can also obtain edge perception data, which is collected by a camera, and the specific effective area is 1000 in width and 700 in height, and the pixels in the two directions are 540 and 360, respectively. Algorithm strategies for processing light field data can also be obtained at the same time, such as a viewpoint selection algorithm, a background removal model algorithm, and a frame recognition model algorithm. When the light field display method obtains the above-mentioned light field data, edge perception data, and processing algorithm strategies, the adaptive strategy configuration is listed as follows:

[0107]

[0108]

[0109] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server. The method of the embodiments can also be applied to a distributed scenario, and be completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present disclosure, and the multiple devices can interact with each other to complete the method.

[0110] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

[0111] The embodiments of the present disclosure also provide a display method. Specifically, the display method comprises:

[0112] obtaining light field data; wherein the light field data comprises a plurality of viewpoint images; the viewpoint images comprise pixel weight parameters;

[0113] processing the viewpoint images by using an image processing model, determining target pixels and non-target pixels according to a processing result, and updating the values of the pixel weight parameters of the target pixels;

[0114] determining the calculation order of the transmittance of each pixel point of each liquid crystal layer of a spatial light modulation array and the values of the transmittance according to each viewpoint image and the values of the pixel weight parameters of each pixel after being processed by the image processing model.

[0115] In some embodiments, the image processing model comprises at least one of a background removal model, a frame recognition model, and a character recognition model; and the method comprises at least one of the following steps:

[0116] determining, based on the background removal model, background pixels in the viewpoint images as first target pixels; and reducing the values of the pixel weight parameters corresponding to the first target pixels;

[0117] determining, based on the frame recognition model, frame pixels in the viewpoint images as second target pixels; and increasing the values of the pixel weight parameters corresponding to the second target pixels; and

[0118] determining, based on the character recognition model, character pixels in the viewpoint images as third target pixels; and increasing the values of the pixel weight parameters corresponding to the third target pixels.

[0119] In some embodiments, the step of determining the calculation order and the transmittance value of each pixel point of each liquid crystal layer of the spatial light modulation array according to the values of the pixel weight parameters corresponding to each pixel of each view image and each pixel after being processed by the image processing model, specifically comprises:

[0120] determining the ray weight of the light ray corresponding to each pixel in each view image according to the value of the pixel weight parameter;

[0121] determining the calculation order and the transmittance value of each pixel point in each liquid crystal layer according to the preset light field decomposition algorithm, the light ray and the ray weight thereof, and locking the transmittance of the corresponding pixel point in response to the transmittance of any pixel point being determined;

[0122] wherein the calculation order is configured to be determined according to the maximum value of the ray weight of each light ray passing through the pixel point.

[0123] In some embodiments, the step of determining the calculation order and the transmittance value of each pixel point in each liquid crystal layer according to the preset light field decomposition algorithm, the light ray and the ray weight thereof, specifically comprises:

[0124] comparing the ray weight of the light ray passing through the pixel point, and determining the maximum ray weight as the light field decomposition order value of the pixel point;

[0125] sorting the light field decomposition order values of each pixel point from large to small, and matching the corresponding calculation order according to the sorting result;

[0126] sequentially determining the transmittance of each pixel point based on the preset light field decomposition algorithm, the calculation order and the light ray, wherein the calculation order of the transmittance of each pixel point in the same order is randomly determined.

[0127] In some embodiments, the method further comprises:

[0128] in response to the order value corresponding to the pixel point of the last order being zero, directly terminating the light field decomposition calculation.

[0129] In some embodiments, the number of liquid crystal layers is three, and the preset light field decomposition algorithm comprises the calculation formula of formula (1):

[0130]

[0131] wherein L j is the jth light ray of the pixel point a passing through any liquid crystal layer; b j and c j is the light ray L jThe intersection pixel point of the two liquid crystal layers other than the liquid crystal layer in which the pixel point a is located; V is the number of light rays passing through the pixel point a; is the jth reconstructed light ray and a is the pixel transmittance before updating, a' is the pixel transmittance after updating;

[0132] Randomly initialize the transmittance of each pixel point of each liquid crystal layer, and iteratively calculate a' according to formula (1) until the predetermined ending rule is met, then the corresponding a' is the transmittance of the pixel point a and is locked.

[0133] In some embodiments, before the step of processing the view images by using the image processing model, the method further comprises:

[0134] Obtaining edge perception data by using an edge perceiver, filtering the light field data based on the edge perception data, and determining the target view image.

[0135] In some embodiments, the edge perception data includes a camera image; the camera image includes a plurality of regions and each region matches a view image;

[0136] The method comprises:

[0137] Using a face recognition model to recognize the face in the camera image and determine the region of the face in the camera image;

[0138] According to the region, the corresponding view image of the region and the surrounding region is determined as the target view image.

[0139] Based on the same inventive concept, the disclosure also provides a light field display device corresponding to the method of any of the above embodiments.

[0140] Reference Figure 8 , the light field display device comprises a spatial light modulation array 81 and a processor 82; the spatial light modulation array comprises at least two liquid crystal layers (811, 812, 813); the liquid crystal layer comprises arrayed pixel points;

[0141] The processor 82 is configured to perform the following steps:

[0142] Obtaining light field data; wherein the light field data comprises a plurality of view images; the view image comprises pixel weight parameters;

[0143] Processing the view images by using an image processing model, determining target pixels and non-target pixels according to the processing result, and updating the value of the pixel weight parameters of the target pixels;

[0144] According to the values of the pixel weight parameters corresponding to each pixel in each of the view images and processed by the image processing model, a calculation sequence of the light transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array and the values of the light transmittance are determined.

[0145] In some embodiments, the light field display device includes a backlight 84 for providing backlight.

[0146] In some embodiments, the image processing model includes at least one of a background removal model, a frame recognition model, and a character recognition model; and the processor 82 is configured to perform at least one of the following steps:

[0147] Based on the background removal model, the background pixels in the view image are determined as first target pixels; the values of the pixel weight parameters corresponding to the first target pixels are reduced;

[0148] Based on the frame recognition model, the frame pixels in the view image are determined as second target pixels; the values of the pixel weight parameters corresponding to the second target pixels are increased; and

[0149] Based on the character recognition model, the character pixels in the view image are determined as third target pixels; the values of the pixel weight parameters corresponding to the third target pixels are increased.

[0150] In some embodiments, the processor 82 is configured to perform the following steps:

[0151] According to the values of the pixel weight parameters, the light ray weight of each light ray corresponding to each pixel in each of the view images is determined;

[0152] According to a preset light field decomposition algorithm, the light rays and the light ray weights thereof, the calculation sequence of the light transmittance of each pixel point in each liquid crystal layer and the values of the light transmittance are determined; and in response to the light transmittance of any of the pixel points being determined, the light transmittance of the corresponding pixel point is locked;

[0153] The calculation sequence is configured to be determined according to the maximum value of the light ray weights of the light rays passing through the pixel point.

[0154] In some embodiments, the processor 82 is configured to perform the following steps:

[0155] The light ray weights of the light rays passing through the pixel point are compared, and the maximum light ray weight is determined as the light field decomposition sequence value of the pixel point;

[0156] The light field decomposition sequence values of each of the pixel points are sorted from large to small, and the corresponding calculation sequence is matched according to the sorting result;

[0157] Determine the transmittance of each pixel point in sequence based on the preset light field decomposition algorithm and the calculation sequence; wherein the calculation sequence of the transmittance of each pixel point in the same sequence is determined randomly.

[0158] In some embodiments, the processor 82 is configured to perform the following steps:

[0159] In response to the sequence value corresponding to the pixel point of the last sequence being zero, terminate the light field decomposition calculation directly.

[0160] In some embodiments, the number of liquid crystal layers is three, and the preset light field decomposition algorithm includes the calculation formula of formula (1):

[0161]

[0162] Wherein, L j is the jth light ray passing through the pixel point a of any liquid crystal layer; b j and c j are the intersection pixel points of the light ray L j and the remaining two liquid crystal layers except the liquid crystal layer where the pixel point a is located; V is the number of light rays passing through the pixel point a; is the jth reconstructed light ray, and a is the pixel transmittance before updating, and a' is the updated pixel transmittance;

[0163] The processor 82 is configured to perform the following steps:

[0164] Randomly initialize the transmittance of each pixel point of each liquid crystal layer, and iteratively calculate a' according to formula (1) until the predetermined ending rule is met, then the corresponding a' is the transmittance of the pixel point a and is locked.

[0165] In some embodiments, the display device includes an edge sensor 83;

[0166] The processor 82 is configured to perform the following steps:

[0167] Obtain edge sensing data using the edge sensor, filter the light field data based on the edge sensing data, and determine the target view image.

[0168] In some embodiments, the edge sensing data includes a camera image; the camera image includes a plurality of regions and each region matches a view image;

[0169] The processor 82 is configured to perform the following steps:

[0170] Use a face recognition model to recognize the face in the camera image and determine the region of the face in the camera image;

[0171] According to the region, a view point image corresponding to the region and a region around the region is determined as a target view point image.

[0172] The apparatuses of the above embodiments are used to implement the corresponding display methods in any of the preceding embodiments, and have the beneficial effects of the corresponding method embodiments, which are not described here again.

[0173] It should be understood by those of ordinary skill in the art that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including claims) of the present disclosure is limited to these examples; the above embodiments or technical features among different embodiments can also be combined under the idea of the present disclosure, steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.

[0174] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the present disclosure difficult to understand, the well-known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, the apparatuses can be shown in the form of block diagrams in order to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform to be implemented (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an illustrative embodiment of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure can be practiced without these specific details or with an implementation varying from these specific details. Therefore, these descriptions should be considered as illustrative rather than limiting.

[0175] Although the present disclosure has been described in conjunction with the specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0176] The embodiments of the present disclosure are intended to cover all such alternatives, modifications and variations as falling within the broad scope of the appended claims. Accordingly, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A light field display method, characterized by, The display method comprises: acquiring light field data; wherein the light field data comprises a plurality of viewpoint images; the viewpoint images comprise pixel weight parameters; processing the viewpoint images by using an image processing model, determining target pixels and non-target pixels according to the processing result, and updating the values of the pixel weight parameters of the target pixels; determining the calculation order and the values of the transmittances of the pixel points of each liquid crystal layer of a spatial light modulation array according to each viewpoint image and the values of the pixel weight parameters of each pixel after the image processing model processing; wherein the calculation order is configured to be determined by sorting each pixel point from large to small according to the maximum value of the ray weight of each light ray passing through the pixel point, and the calculation order of the pixel point in the front is in the front according to the sorting result, and the calculation order of the pixel point in the back is in the back; wherein the ray weight of each light ray is determined based on the value of the pixel weight parameter.

2. The light field display method of claim 1, wherein, The image processing model comprises at least one of a background removal model, a frame recognition model and a character recognition model; and the method comprises at least one of the following steps: determining the background pixels in the viewpoint image as first target pixels based on the background removal model; the value of the pixel weight parameter corresponding to the first target pixel is reduced; determining the frame pixels in the viewpoint image as second target pixels based on the frame recognition model; the value of the pixel weight parameter corresponding to the second target pixel is increased; and determining the character pixels in the viewpoint image as third target pixels based on the character recognition model; the value of the pixel weight parameter corresponding to the third target pixel is increased.

3. The light field display method of claim 1, wherein, The step of determining the calculation order and the values of the transmittances of the pixel points of each liquid crystal layer of a spatial light modulation array according to each viewpoint image and the values of the pixel weight parameters of each pixel after the image processing model processing, specifically comprises: determining the ray weight of each light ray corresponding to each pixel in each viewpoint image according to the value of the pixel weight parameter; determining the calculation order and the values of the transmittances of the pixel points in each liquid crystal layer according to a preset light field decomposition algorithm, the light rays and the ray weights thereof, and locking the transmittance of the corresponding pixel point in response to the transmittance of any pixel point being determined.

4. The light field display method of claim 3, wherein, The step of determining the calculation order and the values of the transmittances of the pixel points in each liquid crystal layer according to a preset light field decomposition algorithm, the light rays and the ray weights thereof, specifically comprises: comparing the ray weights of each light ray passing through the pixel point, and determining the maximum ray weight as the light field decomposition order value of the pixel point; sorting the light field decomposition order values of each pixel point from large to small, and matching the corresponding calculation order according to the sorting result; sequentially determining the values of the transmittances of each pixel point based on the preset light field decomposition algorithm, the calculation order and the light rays; wherein the calculation order of each pixel point in the same order is determined randomly.

5. The light field display method of claim 4, wherein, The method further comprises: in response to the order value corresponding to the pixel point of the last order being zero, directly terminating the light field decomposition calculation.

6. The light field display method of claim 4, wherein, The number of the liquid crystal layers is three, and the preset light field decomposition algorithm comprises a calculation formula of formula (1). （1） wherein, L j is the pixel point through which the light ray passes; j the first light ray; b j and c j is the light ray L j and the intersection pixel point of the other two liquid crystal layers except the liquid crystal layer in which the pixel point is located; V is the number of light rays passing through the pixel point is the first j reconstruction light ray and ; is the pixel transmittance before updating, the pixel transmittance after updating;​​ Randomly initialize the transmittance of each pixel point of each liquid crystal layer, and iteratively calculate according to formula (1) Until the predetermined end rule is met, the corresponding is locked. transmittance of the pixel point 7. The light field display method of claim 1, wherein, Before the step of processing the view point images by using the image processing model, the method further comprises: acquiring edge perception data by using an edge perception device, filtering the light field data based on the edge perception data, and determining a target view point image.

8. The light field display method of claim 7, wherein, The edge perception data comprises a camera image; the camera image comprises a plurality of regions, and each region matches a view point image; The method comprises: identifying a face in the camera image by using a face recognition model, and determining a region of the face in the camera image; determining, according to the region, a view point image corresponding to the region and a region surrounding the region as a target view point image.

9. An optical field display device, characterized by The light field display device comprises a spatial light modulation array and a processor; the spatial light modulation array comprises at least two liquid crystal layers; the liquid crystal layers comprise a plurality of pixel points arranged in an array; The processor is configured to perform the following steps: acquiring light field data; wherein the light field data comprises a plurality of view point images; the view point images comprise pixel weight parameters; processing the view point images by using an image processing model, determining target pixels and non-target pixels according to a processing result, and updating values of the pixel weight parameters of the target pixels; determining a calculation sequence of transmittance and values of the transmittance of each pixel point of each liquid crystal layer of the spatial light modulation array according to the view point images and the values of the pixel weight parameters of each pixel after being processed by the image processing model; wherein the calculation sequence is configured to be determined by sorting each of the pixel points from large to small according to a maximum value of ray weights of each light ray passing through the pixel point, and the calculation sequence of the pixel point in the front of a sorting result is in the front, and the calculation sequence of the pixel point in the back of the sorting result is in the back; wherein the ray weights of the light rays are determined based on the values of the pixel weight parameters.

10. The light field display apparatus of claim 9, wherein, The image processing model comprises at least one of a background removal model, a frame recognition model and a character recognition model; the processor is configured to perform at least one of the following steps: determining, based on the background removal model, background pixels in the view point images as first target pixels; the values of the pixel weight parameters corresponding to the first target pixels are reduced; determining, based on the frame recognition model, frame pixels in the view point images as second target pixels; the values of the pixel weight parameters corresponding to the second target pixels are increased; and determining, based on the character recognition model, character pixels in the view point images as third target pixels; the values of the pixel weight parameters corresponding to the third target pixels are increased.

11. The light field display apparatus of claim 9, wherein, The processor is configured to perform the following steps: determining, according to the values of the pixel weight parameters, ray weights of light rays corresponding to each pixel in each of the view point images; determining, according to a preset light field decomposition algorithm, the light rays and the ray weights thereof, a calculation sequence of transmittance and values of the transmittance of each pixel point in each liquid crystal layer; and in response to the transmittance of any of the pixel points being determined, locking the transmittance of the corresponding pixel point.

12. The light field display apparatus of claim 11, wherein, The number of the liquid crystal layers is three, and the preset light field decomposition algorithm comprises a calculation formula of formula (1): （1） wherein, L j is the pixel point through which the light ray passes; the first j ray; b j and c j is the light ray L j and the intersection pixel point of the other two liquid crystal layers except the liquid crystal layer in which the pixel point is located; V is the number of light rays passing through the pixel point ; is the first j reconstruction light ray and ; is the pixel transmittance before updating, the pixel transmittance after updating; The processor is configured to perform the following steps: Randomly initialize the transmittance of each pixel point of each liquid crystal layer, and iteratively calculate according to formula (1) Until the predetermined end rule is met, the corresponding is locked. transmittance of the pixel point 13. The light field display apparatus of claim 9, wherein, The display device comprises an edge sensor; The processor is configured to perform the following steps: Edge sensing data is acquired by using the edge sensor, the light field data is filtered based on the edge sensing data, and a target view image is determined.

14. The light field display apparatus of claim 13, wherein, The edge sensing data comprises a camera image; the camera image comprises a plurality of regions and each region matches a view image; The processor is configured to perform the following steps: A face recognition model is used to recognize a face in the camera image and determine a region of the face in the camera image; According to the region, a view image corresponding to the region and a region around the region is determined as a target view image.

Images