A multi-projection light field image synthesis method and system

Abstract

The application discloses a kind of multi-projection light field image synthesis method and system, it is related to image synthesis technical field, the method includes: obtaining the coordinates of projector array, the bottom of holographic screen and actual camera array;Determine multiple display restoration light;The display restoration light of each projector is carried out image synthesis, obtains multiple target images, and the holographic screen receives the projection of projector on target image and reconstructs three-dimensional light field;The display restoration light of a projector is carried out image synthesis process is: according to the column pixel of all display restoration light of projector corresponding target image determination column pixel synthesis process is: determining the ideal camera of display restoration light coordinates;According to the ideal camera coordinates and the coordinates of actual camera array determine the actual coordinates of two actual cameras;From according to display restoration light, active range and actual coordinates determine multiple actual imaging pixels.The application improves the accuracy of projection image synthesis and projection reconstruction light field display effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image synthesis, in particular to a multi-projection light field image synthesis method and system. BACKGROUND

[0002] The projection image array is the basis of the multi-projection three-dimensional light field display system, and its quality determines the three-dimensional display effect. When sparse sampling, the light field reconstruction effect is poor due to insufficient sampling information. In the traditional method, the actual camera with the smallest deviation from the display light is directly used to replace the ideal camera for synthesizing the pixel projected by the display light. This method has deviation, and the accuracy of the synthesized pixel is low, which further affects the display effect of the final three-dimensional light field. Therefore, it is a technical problem urgently needed to be solved in the research field to invent a method that can improve the accuracy of the projection image under sparse conditions. SUMMARY

[0003] The purpose of the present application is to provide a multi-projection light field image synthesis method and system, which improves the projection image synthesis quality and rate, and improves the accuracy of the three-dimensional light field display effect of the projection reconstruction.

[0004] To achieve the above purpose, the present application provides the following scheme:

[0005] A multi-projection light field image synthesis method, the method comprising:

[0006] Obtaining the coordinates of the projector array, the coordinates of the bottom edge of the holographic screen and the coordinates of the actual camera array; the projector array and the actual camera array are both arranged on a first circular track with a first preset length as the radius and the center of the bottom edge of the holographic screen as the center; the projector array is arranged on one side of a first straight line, and the actual camera array is arranged on the other side of the first straight line; the first straight line is a straight line on which the bottom edge of the holographic screen is located; the number of the coordinates of the bottom edge of the holographic screen is the same as the value of the horizontal resolution of each projector in the projector array;

[0007] For any one projector in the projector array, connecting the coordinates of the projector with each coordinate of the bottom edge of the holographic screen to obtain a plurality of display and restoration light rays;

[0008] Performing image synthesis operation on the display and restoration light rays of each projector in the projector array to obtain the target image of each projector; the holographic screen is used to receive the projection of the target image by the projector, and reconstructs the three-dimensional light field based on the projection;

[0009] Wherein, the process of performing image synthesis operation on the display and restoration light rays of a projector is:

[0010] determining a column pixel corresponding to each display-retrieving light ray of the projector; the column pixel includes a plurality of actual imaging pixels;

[0011] composing the target image according to all column pixels corresponding to the projector;

[0012] wherein the process of determining the column pixel corresponding to any display-retrieving light ray of the projector is:

[0013] determining a coordinate of an ideal camera corresponding to the display-retrieving light ray, to obtain an ideal coordinate;

[0014] determining a first actual coordinate of a first actual camera and a second actual coordinate of a second actual camera according to the ideal coordinate and a coordinate of the actual camera array; the first actual camera and the second actual camera are two actual cameras adjacent to the ideal camera;

[0015] determining a moving range of the object to be projected; the moving range is a circular range with a center of a bottom edge of the holographic screen as a center and a second preset length as a radius, the second preset length being less than or equal to the first preset length;

[0016] determining an initial imaging column pixel pair according to the display-retrieving light ray, the moving range, the first actual coordinate and the second actual coordinate; the initial imaging column pixel pair includes a first initial column pixel of the first actual camera and a second initial column pixel of the second actual camera;

[0017] determining a plurality of initial pixel groups from the initial imaging column pixel pair, the initial pixel group including the first initial pixel and the second initial pixel; a pixel difference between the first initial pixel and the second initial pixel is less than a set difference value; the first initial pixel is a pixel in the first initial column pixel; the second initial pixel is a pixel in the second initial column pixel;

[0018] determining an actual imaging pixel corresponding to one of the initial pixel groups according to the ideal coordinate, the first actual coordinate, the second actual coordinate and the initial pixel group;

[0019] determining the column pixel according to actual imaging pixels corresponding to all of the initial pixel groups.

[0020] Optionally, the determining of the initial imaging column pixel pair according to the display-retrieving light ray, the moving range, the first actual coordinate and the second actual coordinate specifically includes:

[0021] determining a far field point coordinate and a near field point coordinate according to the display-retrieving light ray and the moving range;

[0022] determining a first far field point imaging pixel coordinate of the first actual camera according to the far field point coordinate and the first actual coordinate, and determining a first near field point imaging pixel coordinate of the first actual camera according to the near field point coordinate and the first actual coordinate;

[0023] determining a second far field point imaging pixel coordinate of the second actual camera according to the far field point coordinate and the second actual coordinate, and determining a second near field point imaging pixel coordinate of the second actual camera according to the near field point coordinate and the second actual coordinate;

[0024] determining an initial imaging column pixel pair according to the first far field point imaging pixel coordinate, the first near field point imaging pixel coordinate, the second far field point imaging pixel coordinate, the second near field point imaging pixel coordinate and the restored display light ray;

[0025] Optionally, determining the actual imaging pixel according to the ideal coordinate, the first actual coordinate, the second actual coordinate and the initial pixel group, specifically comprising:

[0026] determining a first azimuth angle deviation according to the ideal coordinate and the first actual coordinate, and determining a second azimuth angle deviation according to the ideal coordinate and the second actual coordinate;

[0027] determining the actual imaging pixel according to the initial pixel group, the first azimuth angle deviation and the second azimuth angle deviation.

[0028] Optionally, determining the initial imaging column pixel pair according to the first far field point imaging pixel coordinate, the first near field point imaging pixel coordinate, the second far field point imaging pixel coordinate, the second near field point imaging pixel coordinate and the restored display light ray, specifically comprising:

[0029] sequentially obtaining an i-th first to-be-paired imaging pixel coordinate from a first interval; a right end point of the first interval is the first far field point imaging pixel coordinate, and a left end point of the first interval is the first near field point imaging pixel coordinate;

[0030] determining a second to-be-paired imaging pixel coordinate from a second interval according to the first to-be-paired imaging pixel coordinate and the restored display light ray; a right end point of the second interval is the second far field point imaging pixel coordinate, and a left end point of the second interval is the second near field point imaging pixel coordinate;

[0031] determining all the first to-be-paired imaging pixel coordinates in the first interval and the corresponding second to-be-paired imaging pixel coordinates as the initial imaging column pixel pair.

[0032] A multi-projection light field image synthesis system, the system comprising:

[0033] a coordinate acquisition module, configured to acquire coordinates of a projector array, coordinates of a bottom edge of a holographic screen, and coordinates of an actual camera array; the projector array and the actual camera array are both arranged on a first circular track with a first preset length as a radius and a center of the bottom edge of the holographic screen as a circle center; the projector array is arranged on one side of a first straight line, and the actual camera array is arranged on the other side of the first straight line; the first straight line is a straight line on which the bottom edge of the holographic screen is located; a number of the coordinates of the bottom edge of the holographic screen is the same as a value of a horizontal resolution of each projector in the projector array;

[0034] a display light ray determination module, configured to, for any one projector in the projector array, connect the coordinates of the projector and each coordinate of the bottom edge of the holographic screen to obtain a plurality of display light rays;

[0035] a target image synthesis module, configured to perform image synthesis operation on the display light rays of each projector in the projector array to obtain a target image of each projector; the holographic screen is configured to receive projection of the target image by the projector and reconstruct a three-dimensional light field based on the projection;

[0036] In the aspect of performing image synthesis operation on the display light rays of one projector, the target image synthesis module is configured to:

[0037] determine column pixels corresponding to each display light ray of the projector; the column pixels include a plurality of actual imaging pixels;

[0038] synthesize the target image according to all column pixels corresponding to the projector;

[0039] In the aspect of determining column pixels corresponding to any display light ray of the projector, the target image synthesis module is configured to:

[0040] determine coordinates of an ideal camera corresponding to the display light ray to obtain ideal coordinates;

[0041] determine first actual coordinates of a first actual camera and second actual coordinates of a second actual camera according to the ideal coordinates and the coordinates of the actual camera array; the first actual camera and the second actual camera are two actual cameras adjacent to the ideal camera;

[0042] determine a movement range of a to-be-projected object; the movement range is a circular range with a center of the bottom edge of the holographic screen as a circle center and a second preset length as a radius, and the second preset length is less than or equal to the first preset length;

[0043] determine an initial imaging column pixel pair according to the display restored light ray, the activity range, the first actual coordinate and the second actual coordinate; the initial imaging column pixel pair includes a first initial column pixel of the first actual camera and a second initial column pixel of the second actual camera;

[0044] determine a plurality of initial pixel groups from the initial imaging column pixel pair, the initial pixel groups include the first initial pixel and the second initial pixel; a pixel difference between the first initial pixel and the second initial pixel is less than a set difference value; the first initial pixel is a pixel in the first initial column pixel; the second initial pixel is a pixel in the second initial column pixel;

[0045] determine one actual imaging pixel according to the ideal coordinate, the first actual coordinate, the second actual coordinate and one initial pixel group;

[0046] determine the column pixel according to actual imaging pixels corresponding to all the initial pixel groups.

[0047] Optionally, in the aspect of determining the initial imaging column pixel pair according to the display restored light ray, the activity range, the first actual coordinate and the second actual coordinate, the target image synthesis module is configured to:

[0048] determine a far field point coordinate and a near field point coordinate according to the display restored light ray and the activity range;

[0049] determine a first far field point imaging pixel coordinate of the first actual camera according to the far field point coordinate and the first actual coordinate, and determine a first near field point imaging pixel coordinate of the first actual camera according to the near field point coordinate and the first actual coordinate;

[0050] determine a second far field point imaging pixel coordinate of the second actual camera according to the far field point coordinate and the second actual coordinate, and determine a second near field point imaging pixel coordinate of the second actual camera according to the near field point coordinate and the second actual coordinate;

[0051] determine the initial imaging column pixel pair according to the first far field point imaging pixel coordinate, the first near field point imaging pixel coordinate, the second far field point imaging pixel coordinate, the second near field point imaging pixel coordinate and the display restored light ray.

[0052] Optionally, in the aspect of determining one actual imaging pixel according to the ideal coordinate, the first actual coordinate, the second actual coordinate and one initial pixel group, the target image synthesis module is configured to:

[0053] determine a first azimuth deviation according to the ideal coordinate and the first actual coordinate, and determine a second azimuth deviation according to the ideal coordinate and the second actual coordinate;

[0054] determine one of the actual imaging pixels according to one of the initial pixel groups, the first azimuth deviation and the second azimuth deviation.

[0055] Optionally, in the aspect of determining the initial imaging column pixel pair according to the first far-field point imaging pixel coordinate, the first near-field point imaging pixel coordinate, the second far-field point imaging pixel coordinate, the second near-field point imaging pixel coordinate and the display light ray, the target image synthesis module is configured to:

[0056] sequentially obtain an i-th first to-be-paired imaging pixel coordinate from a first interval; a right end point of the first interval is the first far-field point imaging pixel coordinate, and a left end point of the first interval is the first near-field point imaging pixel coordinate;

[0057] determine a second to-be-paired imaging pixel coordinate according to the first to-be-paired imaging pixel coordinate and the display light ray from a second interval; a right end point of the second interval is the second far-field point imaging pixel coordinate, and a left end point of the second interval is the second near-field point imaging pixel coordinate;

[0058] determine all the first to-be-paired imaging pixel coordinates in the first interval and the corresponding second to-be-paired imaging pixel coordinates as the initial imaging column pixel pair.

[0059] According to the specific embodiments provided by the application, the following technical effects are achieved:

[0060] The application discloses a multi-projection light field image synthesis method and system, which utilizes each projector to display a display light ray to synthesize multiple target images; one projector corresponds to one target image; the projector projects the corresponding target image onto a holographic screen, and the holographic screen receives the projection of the target image by the projector and reconstructs a three-dimensional light field; the synthesis of images is performed in a parallel manner, thereby improving the efficiency of projection image synthesis and projection; when an actual imaging pixel is determined: first, the coordinate of an ideal camera corresponding to the display light ray is determined; then, a first actual camera and a second actual camera are determined according to the ideal camera and an actual camera array; then, the synthesis of pixel points is performed according to the first actual camera and the second actual camera; compared with a traditional method of directly utilizing one actual camera with the minimum deviation from the display light ray to replace the ideal camera, the projection image synthesis quality and rate are improved, and the accuracy of the display effect of the reconstructed three-dimensional light field is improved. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0062] Figure 1 A multi-projection light field image synthesis method flowchart is provided for the embodiments of the present application.

[0063] Figure 2 A top view coordinate system diagram is provided for a three-dimensional light field acquisition and reconstruction system.

[0064] Figure 3 A position diagram is provided for a first actual camera and a second actual camera.

[0065] Figure 4 A range of motion, far field point, near field point, and imaging pixel index diagram is provided for a first actual camera.

[0066] Figure 5 A range of motion, far field point, near field point, and imaging pixel index diagram is provided for a second actual camera.

[0067] Figure 6 A projection image synthesis system structure diagram is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0069] The purpose of the present application is to provide a multi-projection light field image synthesis method and system, aiming to improve the quality and rate of projection image synthesis, and improve the accuracy of three-dimensional light field display effect of projection reconstruction, which can be applied to the field of image synthesis technology.

[0070] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0071] Figure 1 A multi-projection light field image synthesis method flowchart is provided for the embodiments of the present application. As shown in Figure 1 the multi-projection light field image synthesis method in the present embodiment includes:

[0072] Step 101: Obtain the coordinates of the projector array, the coordinates of the bottom edge of the holographic screen, and the coordinates of the actual camera array; the projector array and the actual camera array are both arranged on a first circular track with a first preset length as a radius and with the center of the bottom edge of the holographic screen as a center; the projector array is arranged on one side of a first straight line, and the actual camera array is arranged on the other side of the first straight line; the first straight line is a straight line on which the bottom edge of the holographic screen is located; the number of the coordinates of the bottom edge of the holographic screen is the same as the number of the horizontal resolution values of the projectors in the projector array.

[0073] Step 102: For any one of the projectors in the projector array, connect the coordinates of the projector with each of the coordinates of the bottom edge of the holographic screen to obtain a plurality of display-reconstruction light rays.

[0074] Step 103: Perform image synthesis operation on the display-reconstruction light rays of each of the projectors in the projector array to obtain a target image of each of the projectors; the holographic screen is configured to receive the projection of the target image by the projectors and reconstruct a three-dimensional light field based on the projection;

[0075] In step 103, the process of performing image synthesis operation on the display-reconstruction light rays of one of the projectors is as follows:

[0076] Step 1031: Determine the column pixels corresponding to each of the display-reconstruction light rays of the projector; the column pixels include a plurality of actual imaging pixels.

[0077] Step 1032: Synthesize the target image according to all the column pixels corresponding to the projector.

[0078] In step 1031, the process of determining the column pixels corresponding to any one of the display-reconstruction light rays of the projector is as follows:

[0079] Step 10311: Determine the coordinates of an ideal camera corresponding to the display-reconstruction light ray to obtain ideal coordinates;

[0080] Step 10312: Determine the first actual coordinates of a first actual camera and the second actual coordinates of a second actual camera according to the ideal coordinates and the coordinates of the actual camera array; the first actual camera and the second actual camera are two actual cameras adjacent to the ideal camera.

[0081] Step 10313: Determine the activity range of the object to be projected; the activity range is a circular range with the center of the bottom edge of the holographic screen as a center and with a second preset length as a radius, and the second preset length is less than or equal to the first preset length.

[0082] Step 10314: Determine the initial imaging column pixel pair according to the display-reconstruction light ray, the activity range, the first actual coordinates, and the second actual coordinates; the initial imaging column pixel pair includes a first initial column pixel of the first actual camera and a second initial column pixel of the second actual camera.

[0083] Step 10315: determining a plurality of initial pixel groups from the initial imaging column pixel pairs, the initial pixel groups comprising a first initial pixel and a second initial pixel; the pixel difference between the first initial pixel and the second initial pixel is less than a set difference value; the first initial pixel is a pixel in the first initial column pixel; the second initial pixel is a pixel in the second initial column pixel.

[0084] Step 10316: determining an actual imaging pixel according to the ideal coordinate, the first actual coordinate, the second actual coordinate and one initial pixel group.

[0085] Step 10317: determining a column pixel according to the actual imaging pixels corresponding to all the initial pixel groups.

[0086] As an optional implementation, step 10314 specifically comprises:

[0087] determining the far field point coordinate and the near field point coordinate according to the display restoration light and the activity range.

[0088] determining the first far field point imaging pixel coordinate of the first actual camera according to the far field point coordinate and the first actual coordinate, and determining the first near field point imaging pixel coordinate of the first actual camera according to the near field point coordinate and the first actual coordinate.

[0089] determining the second far field point imaging pixel coordinate of the second actual camera according to the far field point coordinate and the second actual coordinate, and determining the second near field point imaging pixel coordinate of the second actual camera according to the near field point coordinate and the second actual coordinate.

[0090] determining the initial imaging column pixel pair according to the first far field point imaging pixel coordinate, the first near field point imaging pixel coordinate, the second far field point imaging pixel coordinate, the second near field point imaging pixel coordinate and the restoration display light.

[0091] As an optional implementation, step 10316 specifically comprises:

[0092] determining the first azimuth angle deviation according to the ideal coordinate and the first actual coordinate, and determining the second azimuth angle deviation according to the ideal coordinate and the second actual coordinate.

[0093] determining an actual imaging pixel according to one initial pixel group, the first azimuth angle deviation and the second azimuth angle deviation.

[0094] As an optional implementation, determining the initial imaging column pixel pair according to the first far field point imaging pixel coordinate, the first near field point imaging pixel coordinate, the second far field point imaging pixel coordinate, the second near field point imaging pixel coordinate and the restoration display light specifically comprises.

[0095] The coordinates of the i-th first image pixel to be paired are obtained sequentially from the first interval; the right endpoint of the first interval is the coordinates of the first far-field image pixel, and the left endpoint of the first interval is the coordinates of the first near-field image pixel.

[0096] The coordinates of the second image pixel to be paired are determined from the second interval based on the coordinates of the first image pixel to be paired and the restored display light; the right endpoint of the second interval is the coordinates of the second far-field image pixel, and the left endpoint of the second interval is the coordinates of the second near-field image pixel.

[0097] All the first image pixel coordinates to be paired and the corresponding second image pixel coordinates in the first interval are determined as the initial image column pixel pairs.

[0098] Specifically, the process of mapping the entire target image synthesis method to a coordinate system is as follows:

[0099] Step 1: As Figure 2 As shown, the coordinate system for the 3D light field reconstruction system is determined using the plane containing the holographic screen as the reference coordinate system. The center of the holographic screen is the origin, the vertical direction passing through the center of the holographic screen is the z-axis, the horizontal direction is the y-axis, and the axis perpendicular to the holographic screen surface is the x-axis. The unit length of the coordinate system is: e = R p / (N×k), where R P denoted by , k represents the radius of the projection array's circular track (i.e., the radius of the first circular track), k is the projector's throw ratio, and N is the projector's resolution in the horizontal direction.

[0100] Step 2: Determine the distribution range of the light field sampling cameras for this 3D light field reconstruction system (i.e., Figure 2 The first circular track in the diagram: This 3D light field reconstruction system only restores the horizontal motion parallax of the original light field information. To simplify the mapping algorithm, the distribution of the sampling cameras must ensure that all display rays requiring mapping can be directly sampled. At a certain height, the top view of this 3D light field acquisition and reconstruction system is as follows: Figure 2 As shown, P e Let α be the azimuth angle corresponding to the projector at the outermost edge. Figure 2 ΔP e AB contains all the light field information projected by the projector onto the holographic screen AB. (Display light beam) PeB Intersecting with the camera track at C e The corresponding azimuth angle of the camera is So the sampling camera array is in The radius of the uniformly distributed area within the azimuth angle is R C On the arc, among which The interval angle is denoted as

[0101] Step 3: Determine the ideal camera position for the display restoration light to be mapped to a projector:

[0102] (1) Determine the mathematical expression of the line connecting a certain projector and a certain pixel S on the holographic screen, i.e., the display restoration light.

[0103] wherein the projector coordinates are:

[0104]

[0105] Pindex∈[0,P num -1] represents the index of the projector, P num represents the number of projectors, P gap represents the interval angle of the projectors, p x is the horizontal coordinate of the projector, and p y is the vertical coordinate of the projector.

[0106] The pixel coordinates are (0, y i ), wherein y i =n·e, n∈[-N / 2, N / 2-1].

[0107] (2) Determine the position of the intersection point of the display restoration light and the camera track, i.e., the ideal camera position, and the azimuth angle corresponding to the ideal camera position is denoted as

[0108] Step 4: In actual situations, considering the constraints of the sampling camera volume and economic cost, the sampling camera arrangement will not be very dense, so there may not be a real camera at the ideal camera position.

[0109] (1) According to the ideal camera position, determine the positions of the two nearest actual cameras (i.e., the first actual camera and the second actual camera), as shown in Figure 3 , denoted as C A (x A ,y A ), C B (x B ,y B ).

[0110] (2) Calculate the deviation of the azimuth angle of the first actual camera and the second actual camera from the azimuth angle of the ideal camera , denoted as θ A , θ B .

[0111] Step 5: Establish a virtual activity range space for the sampled object:

[0112] In fact, the object depth information is difficult to obtain, the activity range space is a cylindrical space with a known radius, and the radius is smaller than the orbit radius of the light field sampling camera, and the orbit of the ring camera and the orbit of the ring projection are concentric. The activity space is large enough to contain all the light field information of the sampled object. As shown in Figure 4 , the orbit of the activity range at a certain height is denoted as x 2 +y 2 =R a 2 .

[0113] Step 6: As shown in Figure 4 , the intersection coordinates of the display restoration light and the virtual activity range orbit in step 3 are calculated, the intersection point far from the camera orbit is denoted as the far field point, and the coordinates are O far (x far ,y far ), and the intersection point close to the camera orbit is denoted as the near field point, and the coordinates are O near (x near ,y near ).

[0114] Step 7: As shown in Figure 4 , the lower right shadow area represents the sampling range of the actual sampling camera, and the far field point and the near field point are determined by the coordinates of the two adjacent cameras and the coordinates of the far field point and the near field point. The imaging pixel index of the two adjacent cameras is denoted as I farA ,I nearA ,I farB ,I nearB . Since the real object point position of the display light needs to be restored is between the far field point and the near field point, the adjacent camera imaging index I OA ,I OB of the real object point information satisfies the following conditions: I nearA ≤I OA ≤I farA ,I farB ≤I OB ≤I nearB .

[0115] Step 8: As shown in Figure 5 , take a pixel index P A ,P A ∈[I nearA ,I farA ], and assume that it is the camera C A . The pixel index of the real object point (that is, the assumed object point position) is the intersection coordinates of the light collection line (that is, the pixel index P A and the camera position C A ) and the display restoration light, and further I nearA ≤I OA ≤I farA ,IfarB ≤I OB ≤I nearB under the constraint of the imaging pixel index P B of the camera C B .

[0116] Step 9: traverse all possible P nearA in the interval [I farA , I A ] and determine the corresponding P B by the above geometric position relationship, forming a plurality of pairs of (P A , P B ) with mathematical relationship. In practice, the mapping relationship at different vertical heights does not change, and the calculated P A , P B correspond to the imaging pixel column index of the real light field information of the sampled object at a certain column, and a group of column pixel pairs with mathematical relationship are obtained.

[0117] Step 10: calculate the variance s of the column pixel pair

[0118]

[0119] where P Bi , P Ai represent the pixel values of the i-th pixel in the pixel column P A , P B , and M represents the vertical resolution of the collected image. When the value of the variance s 2 is the smallest, the similarity of the two pixel columns is the highest, and at this time, I OBi , I OAi are the actual imaging pixel indexes corresponding to the real object point.

[0120] Step 11:

[0121] (1) Determine the synthesized pixel value of the projection pixel column of the projector: S i = (1-Δ)·I OBi -Δ·I OAi , where S i represents the pixel value of the i-th pixel of the synthesized column pixel, and i∈[0,M-1], I OBi , I OAi represent the i-th pixel value of the imaging pixel column of the nearest camera (i.e. the actual imaging pixel index corresponding to the real object point), and Δ = |tanθ B | / (|tanθ B |+|tanθ A |) is the weight, which represents the contribution amount of different collected pixel pairs to the display pixel value.

[0122] (2) calculating the column-pixel value corresponding to each display-recovering light line of a projector, so as to obtain the target composite image of the projector

[0123] For pixel filling of all the projection images, first, the whole image information is obtained according to the nearest neighboring camera index, the useful information in the whole image information is determined according to the index, the actual pixel value of the composite column pixel is calculated by weighted fusion, the pixel filling of the projection image column pixel is completed, and the target image is obtained after filling.

[0124] Figure 6 The projection image synthesis system structure schematic diagram provided by the embodiment of the application is shown in the figure. Figure 6 As shown in the figure, the projection image synthesis system in the embodiment comprises:

[0125] The coordinate acquisition module 201 is configured to acquire the coordinates of the projector array, the coordinates of the bottom edge of the holographic screen and the coordinates of the actual camera array; the projector array and the actual camera array are arranged on a first circular track with a first preset length as the radius and the center of the bottom edge of the holographic screen as the center; the projector array is arranged on one side of a first straight line, and the actual camera array is arranged on the other side of the first straight line; the first straight line is a straight line on which the bottom edge of the holographic screen is located; the number of the coordinates of the bottom edge of the holographic screen is the same as the value of the horizontal resolution of each projector in the projector array.

[0126] The display-recovering light line determination module 202 is configured to, for any one of the projectors in the projector array, connect the coordinates of the projector and the coordinates of the bottom edge of the holographic screen to obtain a plurality of display-recovering light lines.

[0127] The target image synthesis module 203 is configured to perform image synthesis operation on the display-recovering light lines of each projector in the projector array to obtain the target image of each projector; the holographic screen is configured to receive the projection of the target image by the projector and reconstruct a three-dimensional light field based on the projection.

[0128] In the aspect of performing image synthesis operation on the display-recovering light lines of a projector, the target image synthesis module 203 is configured to:

[0129] determine the column pixel corresponding to each display-recovering light line of the projector; the column pixel comprises a plurality of actual imaging pixels.

[0130] synthesize the target image according to all the column pixels corresponding to the projector.

[0131] In the aspect of determining the column pixel corresponding to any one of the display-recovering light lines of the projector, the target image synthesis module 203 is configured to:

[0132] determine the coordinates of the ideal camera corresponding to the display-recovering light line to obtain an ideal coordinate.

[0133] determining a first actual coordinate of the first actual camera and a second actual coordinate of the second actual camera according to the ideal coordinate and the coordinate of the actual camera array; the first actual camera and the second actual camera are two actual cameras adjacent to the ideal camera.

[0134] determining a moving range of the object to be projected; the moving range is a circular range with a center of a bottom edge of the holographic screen as a center and a second preset length as a radius, the second preset length is less than or equal to the first preset length.

[0135] determining an initial imaging column pixel pair according to the display restoration light, the moving range, the first actual coordinate and the second actual coordinate; the initial imaging column pixel pair includes a first initial column pixel of the first actual camera and a second initial column pixel of the second actual camera.

[0136] determining a plurality of initial pixel groups from the initial imaging column pixel pair, the initial pixel group including the first initial column pixel and the second initial column pixel; a pixel difference between the first initial column pixel and the second initial column pixel is less than a set difference value; the first initial column pixel is a pixel in the first initial column pixel; the second initial column pixel is a pixel in the second initial column pixel.

[0137] determining an actual imaging pixel according to the ideal coordinate, the first actual coordinate, the second actual coordinate and one initial pixel group.

[0138] determining a column pixel according to actual imaging pixels corresponding to all initial pixel groups.

[0139] As an optional implementation, in the aspect of determining the initial imaging column pixel pair according to the display restoration light, the moving range, the first actual coordinate and the second actual coordinate, the target image synthesis module 203 is configured to:

[0140] determining a far field point coordinate and a near field point coordinate according to the display restoration light and the moving range.

[0141] determining a first far field point imaging pixel coordinate of the first actual camera according to the far field point coordinate and the first actual coordinate, and determining a first near field point imaging pixel coordinate of the first actual camera according to the near field point coordinate and the first actual coordinate.

[0142] determining a second far field point imaging pixel coordinate of the second actual camera according to the far field point coordinate and the second actual coordinate, and determining a second near field point imaging pixel coordinate of the second actual camera according to the near field point coordinate and the second actual coordinate.

[0143] determining the initial imaging column pixel pair according to the first far field point imaging pixel coordinate, the first near field point imaging pixel coordinate, the second far field point imaging pixel coordinate, the second near field point imaging pixel coordinate and the display restoration light.

[0144] As an optional implementation, in the aspect of determining an actual imaging pixel according to the ideal coordinate, the first actual coordinate, the second actual coordinate and an initial pixel group, the target image synthesis module 203 is configured to:

[0145] Determine a first azimuth deviation according to the ideal coordinate and the first actual coordinate, and determine a second azimuth deviation according to the ideal coordinate and the second actual coordinate.

[0146] Determine an actual imaging pixel according to the initial pixel group, the first azimuth deviation and the second azimuth deviation.

[0147] As an optional implementation, in the aspect of determining an initial imaging column pixel pair according to the first far-field point imaging pixel coordinate, the first near-field point imaging pixel coordinate, the second far-field point imaging pixel coordinate, the second near-field point imaging pixel coordinate and the restored display light, the target image synthesis module 203 is configured to:

[0148] Obtain an i-th first to-be-paired imaging pixel coordinate from the first interval in sequence; the right end point of the first interval is the first far-field point imaging pixel coordinate, and the left end point of the first interval is the first near-field point imaging pixel coordinate.

[0149] Determine a second to-be-paired imaging pixel coordinate from the second interval according to the first to-be-paired imaging pixel coordinate and the restored display light; the right end point of the second interval is the second far-field point imaging pixel coordinate, and the left end point of the second interval is the second near-field point imaging pixel coordinate.

[0150] Determine all the first to-be-paired imaging pixel coordinates in the first interval and the corresponding second to-be-paired imaging pixel coordinates as the initial imaging column pixel pair.

[0151] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiments, the description is relatively simple because it corresponds to the method disclosed in the embodiments. The relevant parts can be referred to the description of the method.

[0152] The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for the general technical personnel in the field, the specific implementation manners and application ranges can be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A method for synthesizing multi-projection light field images, characterized in that, The method includes: The coordinates of the projector array, the bottom edge coordinates of the holographic screen, and the coordinates of the actual camera array are obtained. Both the projector array and the actual camera array are positioned on a first circular track with a radius of a first preset length and the center of the bottom edge of the holographic screen as its center. The projector array is positioned on one side of a first straight line, and the actual camera array is positioned on the other side of the first straight line. The first straight line is the line containing the bottom edge of the holographic screen. The number of bottom edge coordinates of the holographic screen is the same as the horizontal resolution value of each projector in the projector array. For any projector in the projector array, connect the coordinates of the projector with the coordinates of the bottom edge of the holographic screen to obtain multiple display restoration rays; Image synthesis calculations are performed on the display restoration light rays of each projector in the projector array to obtain the target image of each projector; the holographic screen is used to receive the projection of the target image by the projector and reconstruct the three-dimensional light field based on the projection; The process of performing image synthesis calculations on the display restoration light of a projector is as follows: Determine the column pixel corresponding to each display ray of the projector; the column pixel includes multiple actual imaging pixels; The target image is synthesized based on all column pixels corresponding to the projector; The process of determining the column pixel corresponding to any display ray in the projector is as follows: Determine the coordinates of the ideal camera corresponding to the displayed and restored light, and obtain the ideal coordinates; The first actual coordinates of the first actual camera and the second actual coordinates of the second actual camera are determined based on the ideal coordinates and the coordinates of the actual camera array; the first actual camera and the second actual camera are two actual cameras adjacent to the ideal camera; Determine the range of motion of the object to be projected; the range of motion is a circular area with the center of the bottom edge of the holographic screen as the center and a second preset length as the radius, wherein the second preset length is less than or equal to the first preset length; Based on the displayed restored light, the range of motion, the first actual coordinates, and the second actual coordinates, an initial imaging column pixel pair is determined; the initial imaging column pixel pair includes the first initial column pixel of the first actual camera and the second initial column pixel of the second actual camera. Multiple initial pixel groups are determined from the initial imaging column pixel pairs, the initial pixel groups including a first initial pixel and a second initial pixel; the pixel difference between the first initial pixel and the second initial pixel is less than a set difference value; the first initial pixel is a pixel in the first initial column pixel; the second initial pixel is a pixel in the second initial column pixel; An actual imaging pixel is determined based on the ideal coordinates, the first actual coordinates, the second actual coordinates, and an initial pixel group; The column pixels are determined based on the actual imaging pixels corresponding to all the initial pixel groups; The step of determining the initial imaging column pixel pairs based on the displayed restored light, the range of motion, the first actual coordinates, and the second actual coordinates specifically includes: The far-field and near-field coordinates are determined based on the displayed ray and the range of motion. The first far-field point imaging pixel coordinates of the first actual camera are determined based on the far-field point coordinates and the first actual coordinates, and the first near-field point imaging pixel coordinates of the first actual camera are determined based on the near-field point coordinates and the first actual coordinates. The second far-field point imaging pixel coordinates of the second actual camera are determined based on the far-field point coordinates and the second actual coordinates; the second near-field point imaging pixel coordinates of the second actual camera are determined based on the near-field point coordinates and the second actual coordinates. The initial imaging column pixel pairs are determined based on the first far-field point imaging pixel coordinates, the first near-field point imaging pixel coordinates, the second far-field point imaging pixel coordinates, the second near-field point imaging pixel coordinates, and the display restoration ray.

2. The multi-projection light field image synthesis method according to claim 1, characterized in that, Determining an actual imaging pixel based on the ideal coordinates, the first actual coordinates, the second actual coordinates, and an initial pixel group specifically includes: The first azimuth deviation is determined based on the ideal coordinates and the first actual coordinates, and the second azimuth deviation is determined based on the ideal coordinates and the second actual coordinates; An actual imaging pixel is determined based on an initial pixel group, the first azimuth deviation, and the second azimuth deviation.

3. The multi-projection light field image synthesis method according to claim 1, characterized in that, The step of determining the initial imaging column pixel pairs based on the first far-field imaging pixel coordinates, the first near-field imaging pixel coordinates, the second far-field imaging pixel coordinates, the second near-field imaging pixel coordinates, and the display restoration ray specifically includes: The coordinates of the i-th first image pixel to be paired are obtained sequentially from the first interval; the right endpoint of the first interval is the coordinates of the first far-field image pixel, and the left endpoint of the first interval is the coordinates of the first near-field image pixel. The coordinates of the second imaging pixel to be paired are determined from the second interval based on the coordinates of the first imaging pixel to be paired and the display restoration ray; the right endpoint of the second interval is the coordinates of the second far-field imaging pixel, and the left endpoint of the second interval is the coordinates of the second near-field imaging pixel. All the coordinates of the first image pixels to be paired and the corresponding coordinates of the second image pixels to be paired in the first interval are determined as the initial image column pixel pairs.

4. A multi-projection light field image synthesis system, characterized in that, The system includes: A coordinate acquisition module is used to acquire the coordinates of the projector array, the bottom edge coordinates of the holographic screen, and the coordinates of the actual camera array. Both the projector array and the actual camera array are arranged on a first circular track with a radius of a first preset length and the center of the bottom edge of the holographic screen as the center. The projector array is positioned on one side of a first straight line, and the actual camera array is positioned on the other side of the first straight line. The first straight line is the line containing the bottom edge of the holographic screen. The number of bottom edge coordinates of the holographic screen is the same as the horizontal resolution value of each projector in the projector array. The display restoration ray determination module is used to connect the coordinates of any projector in the projector array with the coordinates of the bottom edge of the holographic screen to obtain multiple display restoration rays; The target image synthesis module is used to perform image synthesis operations on the display restoration light of each projector in the projector array to obtain the target image of each projector; the holographic screen is used to receive the projection of the target image by the projector and reconstruct the three-dimensional light field based on the projection; In the aspect of performing image compositing operations on the display reproduction light of a projector, the target image compositing module is used for: Determine the column pixel corresponding to each display ray of the projector; the column pixel includes multiple actual imaging pixels; The target image is synthesized based on all column pixels corresponding to the projector; In determining the column pixel corresponding to any display restoration ray in the projector, the target image synthesis module is configured to: Determine the coordinates of the ideal camera corresponding to the displayed and restored light, and obtain the ideal coordinates; The first actual coordinates of the first actual camera and the second actual coordinates of the second actual camera are determined based on the ideal coordinates and the coordinates of the actual camera array; the first actual camera and the second actual camera are two actual cameras adjacent to the ideal camera; Determine the range of motion of the object to be projected; the range of motion is a circular area with the center of the bottom edge of the holographic screen as the center and a second preset length as the radius, wherein the second preset length is less than or equal to the first preset length; Based on the displayed restored light, the range of motion, the first actual coordinates, and the second actual coordinates, an initial imaging column pixel pair is determined; the initial imaging column pixel pair includes the first initial column pixel of the first actual camera and the second initial column pixel of the second actual camera. Multiple initial pixel groups are determined from the initial imaging column pixel pairs, the initial pixel groups including a first initial pixel and a second initial pixel; the pixel difference between the first initial pixel and the second initial pixel is less than a set difference value; the first initial pixel is a pixel in the first initial column pixel; the second initial pixel is a pixel in the second initial column pixel; An actual imaging pixel is determined based on the ideal coordinates, the first actual coordinates, the second actual coordinates, and an initial pixel group; The column pixels are determined based on the actual imaging pixels corresponding to all the initial pixel groups; In determining the initial imaging column pixel pairs based on the displayed restored light, the range of motion, the first actual coordinates, and the second actual coordinates, the target image synthesis module is configured to: The far-field and near-field coordinates are determined based on the displayed ray and the range of motion. The first far-field point imaging pixel coordinates of the first actual camera are determined based on the far-field point coordinates and the first actual coordinates, and the first near-field point imaging pixel coordinates of the first actual camera are determined based on the near-field point coordinates and the first actual coordinates. The second far-field point imaging pixel coordinates of the second actual camera are determined based on the far-field point coordinates and the second actual coordinates; the second near-field point imaging pixel coordinates of the second actual camera are determined based on the near-field point coordinates and the second actual coordinates. The initial imaging column pixel pairs are determined based on the first far-field point imaging pixel coordinates, the first near-field point imaging pixel coordinates, the second far-field point imaging pixel coordinates, the second near-field point imaging pixel coordinates, and the display restoration ray.

5. The multi-projection light field image synthesis system according to claim 4, characterized in that, In determining an actual imaging pixel based on the ideal coordinates, the first actual coordinates, the second actual coordinates, and the initial pixel group, the target image synthesis module is configured to: The first azimuth deviation is determined based on the ideal coordinates and the first actual coordinates, and the second azimuth deviation is determined based on the ideal coordinates and the second actual coordinates; An actual imaging pixel is determined based on an initial pixel group, the first azimuth deviation, and the second azimuth deviation.

6. The multi-projection light field image synthesis system according to claim 5, characterized in that, In determining the initial imaging column pixel pairs based on the first far-field imaging pixel coordinates, the first near-field imaging pixel coordinates, the second far-field imaging pixel coordinates, the second near-field imaging pixel coordinates, and the display restoration ray, the target image synthesis module is configured to: The coordinates of the i-th first image pixel to be paired are obtained sequentially from the first interval; the right endpoint of the first interval is the coordinates of the first far-field image pixel, and the left endpoint of the first interval is the coordinates of the first near-field image pixel. The coordinates of the second imaging pixel to be paired are determined from the second interval based on the coordinates of the first imaging pixel to be paired and the display restoration ray; the right endpoint of the second interval is the coordinates of the second far-field imaging pixel, and the left endpoint of the second interval is the coordinates of the second near-field imaging pixel. All the coordinates of the first image pixels to be paired and the corresponding coordinates of the second image pixels to be paired in the first interval are determined as the initial image column pixel pairs.

Images