Projection device and projection method
Through the sampling range and interpolation matrix calculation of the mobile flight distance measuring sensor, the problems of unclear images and obstacle impact of the existing projector's flight time ranging and fixed focus method are solved, and the rapid, accurate and automatic projection device is achieved Improved focus and image clarity.
Patent Information
- Application Number
- CN202310723555.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-19
- Publication Date
- 2024-12-20
AI Technical Summary
The existing projector's time-of-flight range-based focusing method ignores the overall focus state of the image when focusing, and it is easy to have focus prediction errors when non-planar projection surfaces or obstacles, resulting in unclear images.
The controller uses the controller to move the sampling range of the flight distance sensor to multiple positions, obtain multiple sets of distance information on the projection surface, and adjust the focal length of the projection lens through interpolation and matrix calculation, and adjust the projection range of the image source in combination with the user interface. , ensure image clarity.
The fast and accurate automatic focus of the projection device is realized, and the good focus image image can be projected in different environments, avoiding the defects of the traditional method, and improving the focus accuracy and flexibility.
Smart Images
Figure CN119172508A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of optoelectronic devices and optoelectronic operation methods, and in particular to a projection device and a projection method. Background Art
[0002] As projectors advance, they are becoming more and more automated, allowing users to use them faster or more conveniently. Among them, auto focus is a very important part of the automation system. Auto focus allows projectors to be installed in a variety of environments, thereby effectively improving the convenience and versatility of projectors.
[0003] Different types of autofocus have different characteristics. For example, camera-based autofocus is relatively cheap, but it is slow because it needs to analyze images of different focal lengths. On the other hand, time-of-flight based (ToF based) autofocus uses simple triangulation to obtain the angle of the projection plane, and then predicts the focus, so that it can focus quickly. However, although the cost of time-of-flight itself is not high, it is expensive after adding precise mechanisms and focal length position recording sensors. In addition, there are some defects in the time-of-flight focusing of general projectors. For example, the overall focus state of the image is ignored in focusing, and only the axis is used as the main focusing area, or the projection surface is non-planar, or there are small obstacles, which will cause the prediction of the focus point of the time-of-flight ranging to be wrong.
[0004] Therefore, it is necessary to design a new projection device and projection method to overcome the above defects. Summary of the invention
[0005] The object of the present invention is to provide a projection device and a projection method, which can quickly and accurately automatically focus to project a well-focused image.
[0006] To achieve the above object, the present invention provides a projection device for projecting an image onto a projection surface, comprising: an image source for emitting an image beam; a projection lens disposed on the optical path of the image beam and for projecting the image beam onto the projection surface to form the image; a time-of-flight distance-finding sensor for measuring the distance of the projection surface; and a controller electrically connected to the time-of-flight distance-finding sensor and the projection lens, the controller for setting a sampling range of the time-of-flight distance-finding sensor, and for obtaining distance information of corresponding measuring points on the projection surface by moving the sampling range of the time-of-flight distance-finding sensor to a plurality of positions, and the controller for adjusting the focal length of the projection lens according to the plurality of sets of the distance information.
[0007] Preferably, the controller sets the sampling range of the time-of-flight ranging sensor by respectively setting a plurality of sampling windows of the time-of-flight ranging sensor, and the controller is used to respectively move the plurality of sampling windows to different positions to obtain the plurality of sets of distance information, wherein the plurality of sampling windows have different sizes or the coverage ranges of the plurality of sampling pixels of the plurality of sampling windows are different.
[0008] Preferably, the controller is used to interpolate the multiple sets of distance information, and the controller is used to adjust the focal length of the projection lens according to the multiple sets of distance information after interpolation.
[0009] Preferably, the controller is used to calculate a difference matrix between the distance values measured by each sampling pixel in the sampling range and the distance values measured by the clearly focused sampling pixels in the sampling range.
[0010] Preferably, the controller is used to obtain abnormal information in the difference matrix and remove the abnormal information in the difference matrix to calculate the average distance of the projection surface.
[0011] Preferably, the controller is used to obtain a focal length offset matrix of the known focal length of the projection lens at each offset state, and add the difference matrix to the focal length offset matrix.
[0012] Preferably, the controller is used to substitute the result of adding the difference matrix and the focal length offset matrix into a preset focus clarity formula to obtain a clarity matrix, and the controller is used to multiply a standard quality matrix of the known quality performance of the projection lens at each position by the clarity matrix to obtain an imaging quality matrix of the projection surface.
[0013] Preferably, the projection device further comprises a user interface electrically connected to the controller, wherein the controller is used to adjust the projection range of the image source according to a user selection from the user interface.
[0014] A projection method is applied to a projection device, wherein the projection device includes an image source, a projection lens, and a time-of-flight ranging sensor, and includes the following steps:
[0015] The image source emits an image beam, and the projection lens projects the image beam onto a projection surface to form an image screen;
[0016] The time-of-flight ranging sensor measures the distance to the projection surface;
[0017] Setting a sampling range of the time-of-flight ranging sensor;
[0018] Moving the sampling range of the time-of-flight ranging sensor to a plurality of positions to respectively obtain a plurality of sets of distance information of corresponding measuring points on the projection surface; and
[0019] The focal length of the projection lens is adjusted according to the multiple sets of distance information.
[0020] Preferably, the “setting the sampling range of the time-of-flight ranging sensor” comprises respectively setting a plurality of sampling windows of the time-of-flight ranging sensor, and the “moving the sampling range of the time-of-flight ranging sensor to different positions” comprises respectively moving the plurality of sampling windows to different positions to obtain a plurality of sets of the distance information;
[0021] The various sampling windows have different sizes or the coverage ranges of the sampling pixels of the various sampling windows are different.
[0022] Preferably, it also includes:
[0023] interpolating the distance information; and
[0024] The focal length of the projection lens is adjusted according to the multiple sets of interpolated distance information.
[0025] Preferably, it also includes:
[0026] A difference matrix between the distance values measured by each sampling pixel in the sampling range and the distance values measured by the sampling pixels with clear focus in the sampling range is calculated.
[0027] Preferably, the method further includes: obtaining abnormal information in the difference matrix, and removing the abnormal information in the difference matrix; and
[0028] Calculate the average distance of the projection surface.
[0029] Preferably, the method further includes: obtaining a focal length offset matrix of the known focal length of the projection lens at each offset state; and adding the difference matrix to the focal length offset matrix.
[0030] Preferably, the method further includes: substituting the result of adding the difference matrix and the focal length offset matrix into a preset focus clarity formula to obtain a clarity matrix; and
[0031] The known standard quality matrix of the quality performance of the projection lens at each position is multiplied by the definition matrix to obtain the imaging quality matrix of the projection surface.
[0032] Preferably, it also includes:
[0033] The projection range of the image source is reduced to a range where the quality value in the imaging quality matrix is above a threshold.
[0034] Preferably, the method further comprises: shifting the focus distance of the projection lens to expand the range of the quality value in the imaging quality matrix above a threshold; and
[0035] The projection range of the image source is reduced to a range where the quality value in the imaging quality matrix is above the threshold.
[0036] Preferably, the projection device further comprises a user interface, and the projection method further comprises adjusting the projection range of the image source according to a user selection from the user interface.
[0037] Compared with the prior art, in the projection device and projection method provided by the present invention, a controller is used to move the sampling range of the time-of-flight ranging sensor to different positions to obtain multiple sets of distance information of the measurement points on the projection surface, and the controller is used to adjust the focal length of the projection lens according to the multiple sets of distance information. Therefore, the projection device can automatically focus quickly and accurately to project a well-focused image. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 FIG. 4 is a schematic structural diagram of a projection device according to an embodiment of the present invention.
[0039] FIG. 2A to FIG. 2C Draw separately Figure 1 The sampling range of the TOF sensor in the image sensor is moved to three different positions.
[0040] FIG. 3A to FIG. 3C Draws Figure 1 Other sampling windows of the TOF sensor in FIG.
[0041] Figure 4 for Figure 1 A portion of the distance information graph of the time-of-flight sensor in Figure 1.
[0042] Figure 5 Draw Figure 1 The measurement points and average projection distance of the projection device.
[0043] Figure 6 Draw on Figure 1 Take a range near the average projection distance of the projection device.
[0044] Fig. 7A Show Figure 1 The difference matrix of the projection device.
[0045] Figure 7B Show Figure 1 The focal length offset matrix of the projection device.
[0046] Figure 7C Show that Fig. 7A The difference matrix and Figure 7B The matrix obtained by adding the focal length offset matrix.
[0047] Figure 8 Show Figure 1The projection device has a difference matrix of external interference factors.
[0048] Fig. 9 Draw Figure 1 The average distance obtained by the projection device after removing the abnormal information.
[0049] Fig.10 for Figure 1 A graph of the focus clarity formula for a projection device.
[0050] Fig.11 Show Figure 1 The clarity matrix of the projection device.
[0051] Fig.12 Draw Figure 1 The standard quality matrix of the projection device is multiplied by the clarity matrix to obtain the imaging quality matrix.
[0052] Fig.13 for Figure 1 The imaging quality matrix of the projection device under trapezoidal distortion and the projection range of the conventional technology.
[0053] Fig.14A 1 is an imaging quality matrix of a projection device under trapezoidal distortion and a projection range of an embodiment of the present invention.
[0054] Fig. 14B 1 is an imaging quality matrix of a projection device under trapezoidal distortion and a projection range of another embodiment of the present invention.
[0055] Fig.15 FIG. 4 is a flow chart of a projection method according to an embodiment of the present invention.
[0056] Fig.16 for Figure 1 Flow chart of auto-focusing of a time-of-flight sensor of a projection device.
[0057] Fig.17 for Fig.15 Schematic diagram of the framework structure of the projection method. DETAILED DESCRIPTION
[0058] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.
[0059] Certain words are used in the specification and claims to refer to specific components. Those with ordinary knowledge in the field should understand that manufacturers may use different terms to refer to the same component. This specification and claims do not use differences in names as a way to distinguish components, but rather use differences in components' functions as the criteria for distinction. The term "including" mentioned throughout the specification and claims is an open-ended term and should be interpreted as "including but not limited to".
[0060] Figure 1 is a schematic diagram of the structure of a projection device according to an embodiment of the present invention, and FIG. 2A to FIG. 2C Draw separately Figure 1 The sampling range of the TOF sensor in the is moved to three different positions. Figure 1 and FIG. 2A to FIG. 2C The projection device 100 of the present embodiment is used to project an image onto a projection surface 50. In the present embodiment, the projection surface 50 is, for example, a screen, a curtain, a wall, a diffusion plate, or any surface that can be projected. The projection device 100 includes an image source 110, a projection lens 120, a time-of-flight ranging sensor 130, and a controller 140. The image source 110 is used to emit an image beam 112. The projection lens 120 is disposed on the optical path of the image beam 112 and is used to project the image beam 112 onto the projection surface 50 to form an image. In the present embodiment, the image source 110 may be a digital micro-mirror device (DMD), a liquid-crystal-on-silicon panel (LCOS panel), a transmissive liquid crystal panel, or other appropriate spatial light modulators (SLM), and a light source may be used to provide an illumination beam to illuminate the image source 110, so that the image source 110 converts the illumination beam into the image beam 112.
[0061] The TOF sensor 130 is used to measure the distance D1 of the projection surface 50. The controller 140 is electrically connected to the TOF sensor 130 and the projection lens 120. The controller 140 is used to set the sampling range of the TOF sensor 130, and to obtain multiple sets of distance information of the measurement points on the projection surface 50 by moving the sampling range of the TOF sensor 130 to different positions, and to adjust the focal length of the projection lens 120 according to the multiple sets of distance information through the controller 140.
[0062] Specifically, in the present embodiment, the TOF sensor 130 includes a plurality of sensing units 132 arranged in an array, and the above setting of the sampling range of the TOF sensor 130 is to set the sampling window W1 of the TOF sensor 130. For example, the controller 140 is used to set the sampling window W1 of the sensing units 132, and the sampling window W1 has a plurality of sampling pixels P (e.g., sampling pixels P1, P2, P3, P4, P5, P6, P7, P8, and P9), and each sampling pixel P includes several sensing units 132 among the sensing units 132. Each sensing unit 132 is used to measure a distance value (i.e., the distance between the sensing unit 132 and the projection surface 50), and the distance value of a sampling pixel P is obtained by taking the average of the distance values measured by the plurality of sensing units 132 included in the sampling pixel P, so that the sensing error generated by a single sensing unit 132 can be reduced.
[0063] Figure 2A , Figure 2B and Figure 2C Schematic diagrams showing the sampling window W1 moving to different positions are shown respectively. Figure 2B Relative to Figure 2A The sampling window W1 moves to the right by a distance of one sensing unit 132, and Figure 2C Relative to Figure 2A The sampling window W1 moves downward by the distance of two sensing units 132. In other words, Figure 2A , Figure 2B and Figure 2C The sampling window W1 includes different groups of sensing units 132, and the sensing units 132 of these groups may be partially the same or completely different ( Figure 2A , Figure 2B and Figure 2C are partially identical examples).
[0064] For example, Figure 2A The center point coordinates of the first four sensing units 132 in the first row (i.e., a horizontal row) from the upper left corner are (1,1), (2,1), (3,1), and (4,1) in sequence, and the center point coordinates of the first four sensing units 132 in the first row (i.e., a vertical row) from the upper left corner are (1,1), (1,2), (1,3), and (1,4) in sequence, and the center point coordinates of the other sensing units 132 are similar. Therefore, Figure 2AThe center coordinates of the sampling pixels P1 to P9 are (3, 2.5), (7.5, 2.5), (12, 2.5), (3, 4.5), (7.5, 4.5), (12, 4.5), (3, 6.5), (7.5, 6.5), and (12, 6.5), respectively. The center coordinates of the sampling pixels P1, P2, and P3 of the sampling window W1 (3, 2.5), (7.5, 2.5), and (12, 2.5) are used as the set of base generating points. The five groups of sampling pixels P1, P2, and P3 that move horizontally (i.e., in the x direction) are The center point coordinates of P3 can be written as (3+1*i,2.5), (7.5+1*i,2.5), (12+1*i,2.5), where i=0 to 4, that is, the sampling window W1 can be moved to 5 positions in the horizontal direction, and the center point coordinates of the three groups of sampling pixels P1, P2, and P3 moving in the vertical direction (i.e., y direction) can be written as (3,2.5+2*j), (7.5,2.5+2*j), (12,2.5+2*j), where j=0 to 2, that is, the sampling window W1 can be moved to 3 positions in the vertical direction, where Figure 2A The position of the sampling window W1 is the position where i=0 and j=0, Figure 2B The position of the sampling window W1 is the position where i=1 and j=0, and Figure 2C The position of the sampling window W1 is the position where i=0 and j=2, and in the present embodiment, the position of the sampling window W1 can be 15 positions generated by the combination of i=0 to 4 and j=0 to 2.
[0065] In this embodiment, by moving the sampling range (i.e., sampling window W1) of the TOF sensor 130 to the 15 different positions, 15 sets of distance information of the measurement points on the projection surface 50 can be obtained, and the controller 140 can adjust the focal length of the projection lens 120 according to the 15 sets of distance information.
[0066] In the projection device 100 of the present embodiment, the controller 140 is used to move the sampling range of the time-of-flight ranging sensor 130 to different positions to obtain multiple sets of distance information of the measurement points on the projection surface 50, and the controller 140 is used to adjust the focal length of the projection lens 120 according to the multiple sets of distance information. In this way, more measurement points can be generated on the projection surface 50, thereby obtaining more accurate distance information. On the other hand, the use of the time-of-flight ranging sensor 130 can achieve fast measurement, so the projection device 100 of the present embodiment can quickly and accurately automatically focus to project a well-focused image.
[0067] In one embodiment, setting the sampling range of the TOF sensor 130 includes respectively setting a plurality of sampling windows W1, W2, W3 and W4 of the TOF sensor 130, wherein the sampling window W1 is as follows: FIG. 2A to FIG. 2C As shown, the sampling window W2 is Figure 3AAs shown, the sampling window W3 is Figure 3B As shown, the sampling window W4 is as follows Figure 3C The controller 140 is used to obtain multiple sets of distance information by moving the various sampling windows W1, W2, W3 and W4 to different positions, wherein the various sampling windows W1, W2, W3 and W4 have different sizes or the coverage of the multiple sampling pixels P of the various sampling windows W1, W2, W3 and W4 is different.
[0068] For example, the center coordinates of the sampling pixels P1, P2, P3, and P4 of the sampling window W2 are (2.5, 1.5), (6, 1.5), (9, 1.5), and (12.5, 1.5) as the set of base generating points. The center coordinates of the five groups of sampling pixels P1, P2, P3, and P4 that move laterally can be written as the general formula (2.5+1*i, 1.5), (6+1*i, 1.5), (9+1*i, 1.5), (12.5, 1.5). 5+1*i,1.5), where i=0 to 4, that is, the sampling window W2 can be moved to 5 positions in the horizontal direction, and the coordinates of the center points of the three groups of sampling pixels P1, P2, P3, and P4 that move vertically can be written as the general formula (2.5,1.5+2*j), (6.5,1.5+2*j), (9,1.5+2*j), (12.5,1.5+2*j), where j=0 to 2, that is, the sampling window W2 can be moved to 3 positions in the vertical direction. In other words, the position of the sampling window W2 can be 15 positions generated by the combination of i=0 to 4 and j=0 to 2.
[0069] In addition, with the center point coordinates (3, 2), (7.5, 2), (12, 2) of the sampling pixels P1, P2, P3 of the sampling window W3 as the set of base generating points, the center point coordinates of the five groups of sampling pixels P1, P2, P3 that move horizontally can be written as the general formula (3+1*i, 2), (7.5+1*i, 2), (12+1*i, 2), where i=0 to 4, that is, the sampling window W3 can move to five positions in the horizontal direction, and the center point coordinates of the two groups of sampling pixels P1, P2, P3 that move vertically can be written as the general formula (3, 2+2*j), (7.5, 2+2*j), (12, 2+2*j), where j=0 to 1, that is, the sampling window W3 can move to two positions in the vertical direction. In other words, the position of the sampling window W3 can be 10 positions generated by the combination of i=0 to 4 and j=0 to 1.
[0070] In addition, taking the center point coordinates (3,3), (7.5,3), (12,3) of the sampling pixels P1, P2, P3 of the sampling window W4 as the set of base generating points, the center point coordinates of the five groups of sampling pixels P1, P2, P3 that move horizontally can be written as the general formula (3+1*i,3), (7.5+1*i,3), (12+1*i,3), where i=0 to 4, that is, the sampling window W4 can move to five positions in the horizontal direction, and the center point coordinates of the two groups of sampling pixels P1, P2, P3 that move vertically can be written as the general formula (3,3+2*j), (7.5,3+2*j), (12,3+2*j), where j=0 to 1, that is, the sampling window W4 can move to two positions in the vertical direction. In other words, the position of the sampling window W4 can be 10 positions generated by the combination of i=0 to 4 and j=0 to 1.
[0071] When measuring the distance of the projection surface 50, the TOF sensor 130 may first move the sampling window W1 to the above 15 positions to obtain multiple sets of distance information, then move the sampling window W2 to the above 15 positions to obtain another multiple sets of distance information, then move the sampling window W3 to the above 10 positions to obtain another multiple sets of distance information, then move the sampling window W4 to the above 10 positions to obtain another multiple sets of distance information, and then adjust the focal length of the projection lens 120 according to all the multiple sets of distance information obtained by the sampling windows W1, W2, W3 and W4 through the controller 140. In this way, the distance information obtained can cover more measurement points on the projection surface, so the distance information obtained is more sufficient and correct, and the projection device 100 of this embodiment can quickly and more accurately auto-focus, thereby projecting a well-focused image.
[0072] The present invention does not limit the number of types of sampling windows used to 4 (i.e., 4 types of sampling windows W1, W2, W3 and W4). In other embodiments, the number of types of sampling windows can be other numbers, for example, the sampling windows can be 1 type or multiple types different from 4, and these one or more sampling windows can be moved to different positions to obtain distance information of multiple sets of measurement points on the projection surface 50.
[0073] In one embodiment, the controller 140 is used to interpolate the above-mentioned multiple sets of distance information to improve the resolution, and the controller 140 is used to adjust the focal length of the projection lens 120 according to the multiple sets of distance information after interpolation. For example, after the resolution is improved, the range of the distance information map is from 2.5 to 16.5 in the x coordinate and from 1.5 to 11.5 in the y coordinate. A portion of the 28x20 distance information map (where some points are interpolated from adjacent points) with a unit of 0.5 is as follows: Figure 4As shown, the square points are generated by sampling window W1, the triangle points are generated by sampling window W2, the circle points are generated by sampling window W3, the parallelogram points are generated by sampling window W4, and the cross points in the rectangle are repeated adjacent points or interpolated points between the left and right points. Figure 4 Each of these points in corresponds to a distance value, forming the above-mentioned distance information.
[0074] In this embodiment, the controller 140 is used to calculate the difference matrix between the distance value measured by each sampling pixel P in the sampling range and the distance value measured by the sampling pixel P with good focus (i.e., clear focus) in the sampling range. For example, after obtaining the distance information map with improved resolution, the controller 140 can obtain the average projection distance OrtD, such as Figure 5 As shown, the general autofocus based on time-of-flight measurement ends after obtaining the average projection distance of the 9 points in the 9-grid.
[0075] Then, the controller 140 takes a range (OrtD–d to OrtD+d) from OrtD. Figure 6 The distance within the range shown in the figure is taken to calculate DeD(dist,x,y), where the d value is a region value within the range where the focus quality may vary, and dist is a variable, which means that this distance is currently applied as the focus distance, and x, y are the pixel coordinates of the 28×20 distance information map, and DeD(dist,x,y) is the distance difference matrix from the point to dist, that is, the difference matrix mentioned above, and an example of the difference matrix DeD(dist,x,y) is Fig. 7A For a projection surface 50 with a maximum tilt of about 0.2 meters, a DeD(OrtD,x,y) distance of 0 represents that this is the place where the focus quality is the best.
[0076] Since the projection surface 50 is not an ideal plane in design, but is roughly a concave or convex shape with a little skew, the average is the best when focusing, but the center and the periphery have different positive or negative displacements. This phenomenon Figure 7B The DeD_Shift(dist,x,y) matrix shown in FIG. 1 is a focal length offset matrix representing the offset state of the focal length at each position known to the projection lens 120. Next, in this embodiment, the controller 140 adds the difference matrix DeD(dist,x,y) to the focal length offset matrix DeD_Shift(dist,x,y) to obtain Figure 7C A matrix that combines the effects of the difference matrix and the focal length offset matrix.
[0077] In one embodiment, by Figure 8The difference matrix DeD(dist,x,y) shows that there are external interference factors (such as obstacles between the projection device 100 and the projection surface 50). For example, the value of area Q1 in the matrix is obviously different from the change trend of the surrounding values. Therefore, the controller 140 can mark area Q1 and ignore it. This is the benefit of improving the resolution of this embodiment, which is very helpful for improving autofocus. In other words, the controller 140 removes abnormal information in the difference matrix DeD(dist,x,y) (such as -0.5 in area Q1) and calculates the average distance to the projection surface 50. In this embodiment, abnormal information includes information with an absolute value that is too large, or information with an absolute value that is too different from the surrounding values. In addition, if Figure 6 As shown, before removing the information of the region Q1 (eg, removing the information of the measuring point QP1), the average distance is, for example, OrtD, and after removing the information of the region Q1 (eg, removing the information of the measuring point QP1), the average distance is, for example, OrtD. Fig. 9 As shown, the controller 140 calculates the average distance of other measurement points other than the measurement point QP1 as, for example, OrtD+a. This average distance eliminates the influence of abnormal information caused by obstacles.
[0078] In this embodiment, the controller 140 is used to substitute the result of adding the difference matrix DeD(dist,x,y) and the focal length offset matrix DeD_Shift(dist,x,y) into the focus definition formula DeF to obtain the definition matrix DF(dist,x,y). The definition formula DeF can be obtained by discrete cosine transform (DCT), fast Fourier transform (FFT) and other methods, and then quantized to obtain a formula used online, and the result is as follows: Fig.10 The formula curve shown is as follows:
[0079] DF(dist,x,y)=DeF(DeD(dist,x,y)+DeD_Shift(dist,x,y))-Equation 1.
[0080] exist Fig.10 For example, when the displacement is 0 (i.e. the lens position is 0), the best focus quality is achieved. Substituting the displacement into DeF, we can get the reduction value of the blur level, that is, Fig.10 The vertical axis is the focus quality value. When the value is 1, it means the focus quality is the best, and when the value is 0, it means the focus quality is very poor. Then the controller 140 calculates the DeF formula to obtain the definition matrix DF (dist, x, y). An example of the definition matrix DF (dist, x, y) is as follows: Fig.11 ,Depend on Fig.11It can be seen that the skewness of the projection plane 50 affects the focus.
[0081] In this embodiment, the controller 140 is used to multiply a standard quality matrix BFQT(dist,x,y) representing the quality performance at each position of a known projection lens by the definition matrix DF(dist,x,y) to obtain the imaging quality matrix DFQT(dist,x,y) of the projection surface 50, that is:
[0082] Formula 2: DFQT(dist, x, y)= BFQT(dist, x,y) * DF(dist, x, y)
[0083] A practical example of formula 2 is Fig.12 shown.
[0084] Since the projection provided by the projection device 100 is not a perfect projection, but is clear in the middle and a little blurry outside, a formula (i.e., Formula 2) can be applied to obtain the final result, wherein BFQT(dist) is the quality performance of different areas at the optical distance dist of the projection device 100, and DFQT(dist) is the quality perception finally seen by the user after the projection device 100 is projected onto the projection surface 50 (whose distance is dist).
[0085] In this embodiment, the projection device further includes a user interface 150 electrically connected to the controller 140 , wherein the controller 140 is used to adjust the projection range of the image source 110 according to a user's selection from the user interface 150 .
[0086] In this embodiment, when the projected image has keystone distortion or the user allows automatic adjustment of the projection area to avoid obstacles, the projection range can be adjusted, or when the surrounding quality is poor from the imaging quality matrix DFQT, the projection range can be moved or reduced to meet the focus quality. The conventional method to improve keystone distortion is to select the largest rectangle in the middle that meets this ratio, such as Fig.13 The projection range A0 of the image source 110 is shown in Table 1. Of course, this does not take into account the focus quality. However, those with general knowledge in the field know that when the projection angle of the projector is skewed, the image will be more blurred at the edges farther away from the center. The above-mentioned quality value judgment method shows that when the center quality is the best, the DeD value deviation around is large (due to the depth of field). In this embodiment, the controller 140 can be used to reduce the projection range of the image source 110 to a range A1 where the quality value in the imaging quality matrix is above a threshold value (e.g., 6 or other values), such as Fig.14A . Fig.13This is an example of DFQT when the center is adjusted to the best when projecting at an inclined angle (for example, the quality value of the center is 10). Quality values below 5 in the figure mean that the projection focus in this area is not achieved.
[0087] When you encounter Fig.13 In the case of a situation where the projection device 100 supports a displacement or digital zoom function, the user can use the user interface 150 to manually adjust to avoid the situation where the peripheral depth of field is poor. In this embodiment, the controller 140 can use the best DFQT result to automatically help the user narrow the projection range to the range A1 to avoid the effect of poor peripheral depth of field. The result is as follows: Fig.14A , which can be called quality fit. If the user does not care that the center of the image is the best quality, the controller 140 can also use the DFQT result of the surrounding area of OrtD (the range of OrtD-d to OrtD+d) to select a larger projection range A2 with poorer quality, such as Fig. 14B , which can be called the largest fit. The result of the largest fit is not the best quality when the projection plane 50 is orthogonal, but the focus position of the larger acceptable projection range A2 is generated after the optimal position of the intermediate depth of field is shifted to the adjacent area. At this time, the average quality is not the best nor the best area is the largest, but all areas in the projection range A2 have reached a certain level (for example, the quality value is greater than 6). In other words, the controller 140 is used to offset the focus distance of the projection lens 120 to expand the range of the quality value in the imaging quality matrix DFQT above a threshold value (for example, above 6), and the controller 140 is used to reduce the projection range of the image source 110 to the range A2 where the quality value in the imaging quality matrix is above a threshold value (for example, above 6).
[0088] In addition, the above-mentioned well-focused sampling pixel P is the sampling pixel P with the highest DFQT value when dist in DFQT (dist, x, y) is within the range of OrtD-d to OrtD+d. The area where this pixel P is located is the best focus of this segment.
[0089] In one embodiment, the controller 140 is, for example, a central processing unit (CPU), a microprocessor (microprocessor), a digital signal processor (DSP), a programmable controller, a programmable logic device (PLD), or other similar devices or a combination of these devices, and the present invention is not limited thereto. In addition, in one embodiment, each function of the controller 140 can be implemented as a plurality of source codes. These source codes are stored in a memory and executed by the controller 140. Alternatively, in one embodiment, each function of the controller 140 can be implemented as one or more circuits. The present invention is not limited to implementing each function of the controller 140 in software or hardware.
[0090] In summary, the projection device 100 of the present embodiment utilizes a re-sampling method of the TOF sensor 130 to increase the depth resolution of the TOF sensor 130, and also proposes a set of judgment methods utilizing focus analysis. The judgment method utilizing focus analysis can achieve automatic obstacle avoidance and improve poor projection focus, so that the TOF sensor 130 does not need to provide additional equipment during automatic focusing (the above operations only require a small matrix lookup table and simple calculations, and this operation can be processed by its own single-chip system (system-on-a-chip, SOC) without the need for an additional powerful processor) and the price greatly increases the function and additional value of the projection device 100.
[0091] When projection devices are used in living rooms and entertainment applications, more flexible and accurate autofocus is a challenge. The autofocus method of the TOF sensor 130 of this embodiment provides the following advantages:
[0092] 1. Low-cost and relatively accurate autofocus based on time-of-flight ranging;
[0093] 2. Additional analysis of the focusing method to obtain a better focusing effect than the general time-of-flight range-finding table lookup;
[0094] 3. Provide users with functions for intelligent focusing and selecting projection range.
[0095] Fig.15 This is a flowchart of a projection method according to an embodiment of the present invention. Figure 1 and Fig.15The projection method of this embodiment can be performed by the projection device (e.g., projection device 100) of the above embodiment, but the present invention is not limited thereto. The projection device of this embodiment includes the following steps. First, step S110 is executed to provide the projection device 100. Next, step S120 is executed to set the sampling range of the time-of-flight ranging sensor 130 using the controller 140. Then, step S130 is executed to obtain multiple sets of distance information of the measurement points on the projection surface 50 by moving the sampling range of the time-of-flight ranging sensor 130 to different positions using the controller 140. Thereafter, step S140 is executed to adjust the focal length of the projection lens 120 according to the multiple sets of distance information by the controller 140. The other detailed steps of the projection method of this embodiment have been fully described in the above embodiment of the projection device 100, so please refer to the description of the above embodiment.
[0096] Fig.16 for Figure 1 Please refer to the auto focus flowchart of the time-of-flight sensor of the projection device. Figure 1 and Fig.16 In this embodiment, the automatic focusing process of the TOF sensor 130 may include the following steps. First, execute step S210, which is the start of automatic focusing. Then, execute step S220 to obtain an advanced TOF sensor map, which is, for example, Figure 4 Then, step S230 is executed to obtain an average focus distance equal to OrtD, as described above. Figure 5 Then, execute S240 to calculate DeD(dist), as described in the above embodiment. Then, execute step S250 to filter out abnormal information for each DeD(dist), as described in the above embodiment. Fig. 9 After that, step S260 is executed to obtain DFQT(dist) for each DeD(dist), as described in the above embodiment. Next, step S270 is executed to determine whether the projection range needs to be adjusted, for example, Fig.13 If adjustment is required, step S280 is executed to select a partial area of the best DFQT (dist) and apply the focus distance, as shown in detail. Fig.14A or Fig. 14B If no adjustment is required, step S290 is executed to select the best DFQT(dist) and apply the focus distance, that is, no need to adjust the focus distance. Fig.14A and Fig. 14B Finally, step S295 is executed, and the automatic focusing is completed.
[0097] Fig.17 for Fig.15The framework diagram of the projection method includes a time-of-flight ranging sensor 130, an automatic focus control unit 141, a time-of-flight ranging sensor resampling unit 142, a depth and focus analysis 143, a user focus input 144, a focus motor position sensor 145, a focus motor control 146, and a projection display control 147. After the automatic focus control unit 141 reads the value of the time-of-flight ranging sensor 130, it controls the function of the focus motor control 146 to achieve the estimated position of the focus motor position sensor. In addition, in this embodiment, the functions of the time-of-flight ranging sensor resampling unit 142, the depth and focus analysis 143, the user focus input 144, and the projection display control 147 are added. The TOF sensor resampling unit 142 is responsible for obtaining the distance information after the resolution is improved on the TOF sensor 130, the depth and focus analysis 143 is responsible for obtaining the projection quality result after implementing a certain focal length, the user focus input 144 is responsible for accepting the input of the user's quality requirements, and assisting the user to obtain the best projection range (i.e., projection area) when there is keystone distortion or poor projection focus quality, and the projection display control 147 is used to change the projection range (i.e., projection area). Each of the above-mentioned autofocus control unit 141, TOF sensor resampling unit 142, depth and focus analysis 143, user focus input 144, focus motor position sensor 145, focus motor control 146 and projection display control 147 can be implemented by a module (e.g., a software, hardware or firmware module).
[0098] In summary, in the projection device and the projection method of the embodiments of the present invention, since the controller is used to move the sampling range of the time-of-flight ranging sensor to different positions to obtain multiple sets of distance information of the measurement points on the projection surface, and the controller is used to adjust the focal length of the projection lens according to the multiple sets of distance information, the projection device can automatically focus quickly and accurately to project a well-focused image.
[0099] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.
Claims
1. A projection device for projecting an image onto a projection surface, characterized in that: include: An image source, for emitting an image beam; A projection lens is disposed on the optical path of the image beam and is used to project the image beam onto the projection surface to form the image frame; A time-of-flight ranging sensor for measuring the distance of the projection surface; as well as A controller is electrically connected to the time-of-flight ranging sensor and the projection lens. The controller is used to set a sampling range of the time-of-flight ranging sensor, and to obtain distance information of corresponding measuring points on the projection surface by moving the sampling range of the time-of-flight ranging sensor to a plurality of positions. The controller adjusts the focal length of the projection lens according to the plurality of sets of the distance information.
2. The projection device according to claim 1, characterized in that: The controller sets the sampling range of the time-of-flight ranging sensor by respectively setting a plurality of sampling windows of the time-of-flight ranging sensor, and the controller is used to respectively move the plurality of sampling windows to different positions to obtain the plurality of sets of distance information, wherein the plurality of sampling windows have different sizes or the coverage ranges of the plurality of sampling pixels of the plurality of sampling windows are different.
3. The projection device according to claim 1, characterized in that: The controller is used for interpolating the multiple sets of distance information, and the controller is used for adjusting the focal length of the projection lens according to the multiple sets of distance information after interpolation.
4. The projection device according to claim 1, wherein: The controller is used for calculating a difference matrix between the distance values measured by each sampling pixel in the sampling range and the distance values measured by the sampling pixels with clear focus in the sampling range.
5. The projection device according to claim 4, characterized in that: The controller is used for acquiring abnormal information in the difference matrix and removing the abnormal information in the difference matrix, and calculating the average distance of the projection surface.
6. The projection device according to claim 5, characterized in that: The controller is used for obtaining a focal length offset matrix of the known focal length of the projection lens at each position and adding the difference matrix to the focal length offset matrix.
7. The projection device according to claim 6, characterized in that: The controller is used to substitute the result of adding the difference matrix and the focal length offset matrix into a preset focus clarity formula to obtain a clarity matrix, and the controller is used to multiply a standard quality matrix of the known quality performance of the projection lens at each position by the clarity matrix to obtain an imaging quality matrix of the projection surface.
8. The projection device according to claim 7, characterized in that: Also included is a user interface electrically connected to the controller, wherein the controller is used to adjust the projection range of the image source according to a user selection from the user interface.
9. A projection method, applied to a projection device, the projection device comprising an image source, a projection lens and a time-of-flight ranging sensor, characterized in that: The following steps are involved: The image source emits an image beam, and the projection lens projects the image beam onto a projection surface to form an image screen; The time-of-flight ranging sensor measures the distance to the projection surface; Setting a sampling range of the time-of-flight ranging sensor; Moving the sampling range of the time-of-flight ranging sensor to a plurality of positions to respectively obtain a plurality of sets of distance information of corresponding measuring points on the projection surface; as well as The focal length of the projection lens is adjusted according to the multiple sets of distance information.
10. The projection method according to claim 9, characterized in that: The “setting the sampling range of the time-of-flight ranging sensor” includes respectively setting a plurality of sampling windows of the time-of-flight ranging sensor, and the “moving the sampling range of the time-of-flight ranging sensor to different positions” includes respectively moving the plurality of sampling windows to different positions to obtain a plurality of sets of the distance information; The various sampling windows have different sizes or the coverage ranges of the sampling pixels of the various sampling windows are different.
11. The projection method according to claim 9, characterized in that: Also includes: interpolating the distance information; as well as The focal length of the projection lens is adjusted according to the multiple sets of interpolated distance information.
12. The projection method according to claim 9, characterized in that: Also includes: A difference matrix between the distance values measured by each sampling pixel in the sampling range and the distance values measured by the sampling pixels with clear focus in the sampling range is calculated.
13. The projection method according to claim 12, characterized in that: Also includes: Obtaining abnormal information in the difference matrix, and removing the abnormal information in the difference matrix; as well as Calculate the average distance of the projection surface.
14. The projection method according to claim 13, characterized in that: Also includes: Obtaining a focal length offset matrix of the known focal length offset state of the projection lens at each position; The difference matrix is added to the focal length offset matrix.
15. The projection method according to claim 14, characterized in that: Also includes: Substituting the result of adding the difference matrix to the focal length offset matrix into a preset focus clarity formula to obtain a clarity matrix; as well as The known standard quality matrix of the quality performance of the projection lens at each position is multiplied by the definition matrix to obtain the imaging quality matrix of the projection surface.
16. The projection method according to claim 15, characterized in that: Also includes: The projection range of the image source is reduced to a range where the quality value in the imaging quality matrix is above a threshold.
17. The projection method according to claim 15, characterized in that: Also includes: offsetting the focus distance of the projection lens to expand the range of quality values in the imaging quality matrix that are above a threshold; as well as The projection range of the image source is reduced to a range where the quality value in the imaging quality matrix is above the threshold.
18. The projection method according to claim 15, characterized in that: The projection device further comprises a user interface, and the projection method further comprises adjusting the projection range of the image source according to a user selection from the user interface.