Focal length calculation method, display method of projector, and photographing system

Abstract

Provided are a focal length calculation method, a display method of a projector, and a photographing system, which can calculate a focal length of a photographing device by simple operation. The method includes: an image acquisition unit (201) that acquires image data obtained by photographing a rectangular object; a four-side determination unit (203) that determines, in an image represented by the image data, a quadrangle (L1) corresponding to the object, a side (L2) opposite to the side (L1), a side (L3), and a side (L4) opposite to the side (L3); a coordinate determination unit (204) that determines a coordinate value of a vertical vanishing point which is an intersection of an extension line of the side (L1) and an extension line of the side (L2), and a coordinate value of a horizontal vanishing point which is an intersection of an extension line of the side (L3) and an extension line of the side (L4); and a calculation unit (205) that calculates a focal length of a photographing device that photographs the object, according to the coordinate value of the vertical vanishing point and the coordinate value of the horizontal vanishing point.

Description

Technical Field

[0001] This invention relates to a focal length calculation method, a projector display method, and a shooting system. Background Technology

[0002] In recent years, projectors have become smaller, lighter, and easier to carry, leading to their use in various settings. Therefore, it is necessary to adjust the projected image according to the location and screen used.

[0003] Therefore, techniques such as those described in Patent Document 1 are known, which involve installing a camera in a projector, capturing the projected image, and correcting the projected image based on the captured image. In this technique, internal parameters such as the camera's focal length are treated as predetermined parameters.

[0004] In recent years, almost all information terminal devices, such as smartphones, have camera functions. Therefore, it is desirable to use images captured by these devices, but the average user often lacks knowledge of the internal parameters of the camera built into the device. Consequently, techniques, such as those described in Patent Document 2, are known for determining camera parameters, such as focal length, by using images obtained from a grid of images captured by the camera.

[0005] Patent Document 1: International Publication No. 2013 / 038656

[0006] Patent Document 2: Japanese Patent Application Publication No. 2021-1113020

[0007] However, the technology described in the aforementioned patent document 2 has the following problem: it requires a camera to photograph the grid panels arranged in various postures, which requires a lot of labor and time. Summary of the Invention

[0008] One aspect of the focal length calculation method disclosed herein includes: acquiring image data obtained by photographing a rectangular subject; determining, in the image represented by the image data, a first side, a second side opposite to the first side, a third side, and a fourth side opposite to the third side of a quadrilateral corresponding to the subject; determining, in the image, the coordinates of a first point that is the intersection of the extension of the first side and the extension of the second side, and the coordinates of a second point that is the intersection of the extension of the third side and the extension of the fourth side; and calculating, based on the coordinates of the first point and the second point, the focal length of the device that photographed the subject.

[0009] A display method of a projector according to one aspect of this disclosure includes: acquiring image data obtained by photographing a rectangular projection area; determining, in the image represented by the image data, a first side, a second side opposite to the first side, a third side, and a fourth side opposite to the third side of a quadrilateral corresponding to the projection area; determining the coordinates of a first point, which is the intersection of the extension of the first side and the extension of the second side, and the coordinates of a second point, which is the intersection of the extension of the third side and the extension of the fourth side; calculating the focal length of the device that photographed the projection area based on the coordinates of the first point and the second point; determining the coordinates of a third point, which is the intersection of the line connecting the first point and the second point and the extension of the diagonal of the quadrilateral; calculating the aspect ratio of the projection area based on the coordinates of the first point, the second point, and the third point; and projecting an image based on the calculated aspect ratio of the projection area.

[0010] One aspect of the imaging system disclosed herein is characterized by having a processor that performs the following processing: acquiring image data obtained by photographing a rectangular subject; determining, in the image represented by the image data, a first side, a second side opposite to the first side, a third side, and a fourth side opposite to the third side of a quadrilateral corresponding to the subject; determining, in the image, the coordinates of a first point that is the intersection of the extension of the first side and the extension of the second side, and the coordinates of a second point that is the intersection of the extension of the third side and the extension of the fourth side; and calculating, based on the coordinates of the first point and the second point, the focal length of the device that photographed the subject. Attached Figure Description

[0011] Figure 1 This is a diagram showing the structure of the imaging system according to the implementation method.

[0012] Figure 2 This is a diagram showing the hardware structure of the information terminal device in the shooting system.

[0013] Figure 3 It is a block diagram representing the functional structure of the software of an information terminal device.

[0014] Figure 4 This is a diagram showing the hardware structure of the projector in the shooting system.

[0015] Figure 5 This is a block diagram representing the functional structure of the projector's software.

[0016] Figure 6 It is a graph representing the vanishing point.

[0017] Figure 7 It is a diagram representing the camera coordinate system and the world coordinate system.

[0018] Figure 8 It is a graph representing the orthogonality of the vanishing points in the normalized camera coordinate system.

[0019] Figure 9 It is a diagram used to illustrate the vanishing point of the diagonal.

[0020] Figure 10 This is a flowchart illustrating the actions of the information terminal device and projector in the shooting system.

[0021] Label Explanation

[0022] 1: Shooting system; 10: Information terminal device; 20: Projector; 30: Screen; 32: Screen frame; 202: Image acquisition unit; 203: Quadrilateral determination unit; 204: Coordinate determination unit; 205: Calculation unit. Detailed Implementation

[0023] The imaging system according to an embodiment of the present invention will now be described with reference to the accompanying drawings. Furthermore, in each drawing, the dimensions and scale of each part are appropriately different from the actual dimensions and scale. Additionally, the embodiments described below are preferred examples, and therefore various technically preferred limitations have been added. However, unless the scope of the present invention is specifically limited in the following description, the scope of the present invention is not limited to these embodiments.

[0024] Figure 1 This is a diagram showing the structure of the imaging system 1 according to the embodiment.

[0025] The shooting system 1 includes an information terminal device 10 and a projector 20. The information terminal device 10 is, for example, a smartphone or a portable tablet device, and has shooting and communication functions. In this embodiment, the information terminal device 10 captures the shape of the screen 30, specifically, a predetermined area of ​​the screen 30 from which the image is projected by the projector 20.

[0026] For ease of explanation, the predetermined area in screen 30 where the image is projected by projector 20 will be referred to as the projection area. In the illustrated screen 30, it is a black mask screen surrounded by black borders on all four sides. Therefore, strictly speaking, the projection area refers to the white area surrounded by the rectangular screen frame 32 that serves as the black mask. However, determining the shape of the screen frame 32 is essentially equivalent to determining the projection area surrounded by the screen frame 32.

[0027] Alternatively, the screen 30 can also be a type where the entire surface is white without a black mask. If it is a type without a black mask, the projection area can be determined, for example, based on the difference in contrast between the white background and the back (wall).

[0028] Projector 20 magnifies and projects an image based on an image signal provided from a main device (not shown) onto screen 30. Furthermore, the figure shows an example where projector 20 is placed on the upper surface of lectern Tp, and screen 30 is suspended from the wall.

[0029] Figure 2 This is a diagram showing the hardware structure of the information terminal device 10. The information terminal device 10 includes a central processing unit 100, a storage device 120, a connection device 130, a camera device 140, a display device 150, and a communication device 160.

[0030] The central processing unit 100 is a processor, such as a CPU (Central Processing Unit) or other processing circuits, which uniformly controls the various elements of the information terminal device. In addition to a CPU, the central processing unit 100 may also be composed of circuits such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit).

[0031] Storage device 120 may be a single or multiple memory devices constructed from known recording media such as magnetic recording media or semiconductor recording media, storing programs executed by central processing unit 100, various data used by central processing unit 100, and captured image data. Alternatively, storage device 120 may be constructed using a combination of various recording media. Furthermore, removable recording media that can be attached to or detached from information terminal device 10, or external recording media (e.g., in-line storage) that allows information terminal device 10 to communicate via a communication network, may also be used as storage device 120.

[0032] The connection device 130 is an interface for connecting to the imaging device 140, the display device 150, and the communication device 160. The connection device 130 includes, for example, interface circuitry. The imaging device 140 has a lens and an image sensor, and is capable of capturing images through user operation. In this embodiment, it particularly captures the screen frame 32. The imaging device 140 is, for example, a camera.

[0033] Additionally, image data captured by the imaging device 140 is provided to the display device 150, which then displays an image based on that data for user confirmation. The display device 150 includes, for example, a display panel.

[0034] The communication device 160 is a device for communicating information with devices other than the information terminal device 10. Specifically, in this embodiment, the communication device 160 transmits image data, including the image of the screen frame 32, captured by the imaging device 140 to the projector 20. The communication device 160 includes, for example, an antenna for wireless communication and a connection terminal for wired communication.

[0035] In this description, "device" may also be replaced by other terms such as circuit, equipment, or unit. The elements of the information terminal device 10 consist of one or more devices. Some elements of the information terminal device 10 may also be omitted.

[0036] Figure 3 This is a block diagram illustrating the functions constructed by the central processing unit 100 in the information terminal device 10. As shown in the figure, the central processing unit 100 constructs the shooting control unit 102 and the communication control unit 104 by executing various programs stored in the storage device 120.

[0037] Figure 4 This diagram illustrates the hardware structure of the projector 20. The projector 20 includes a central processing unit 200, a storage device 220, a connection device 230, a projection device 250, and a communication device 260. The central processing unit 200, storage device 220, connection device 230, and communication device 260 in the projector 20 are composed of the same devices as the central processing unit 100, storage device 120, connection device 130, and communication device 160 in the information terminal device 10 described above.

[0038] Furthermore, the projection device 250 in the projector 20 is an optical engine that amplifies and projects an image based on an image signal provided from a main device (not shown) onto the screen 30. The projection device may include, for example, a light source (not shown), a light modulation device, and a projection lens. The light modulation device may be, for example, a liquid crystal panel or a digital micromirror device. In this embodiment, the projection device 250 is not critical, and therefore a detailed description is omitted.

[0039] Figure 5 This is a block diagram representing the functions constructed by the central processing unit 200 in the projector 20.

[0040] As shown in the figure, in the central processing unit 200, a processing control unit 201, an image acquisition unit 202, a quadrilateral determination unit 203, a coordinate determination unit 204, a calculation unit 205, and a projection control unit 206 are constructed by executing various programs stored in the storage device 220.

[0041] In this embodiment, the principle of calculating the focal length of the imaging device 140 will be explained.

[0042] Figure 6This is a diagram showing an image captured by the imaging device 140, for example, when viewing a cuboid Cub from a horizontal perspective. In the actual coordinate system (world coordinate system), the edges La and Lb of the cuboid Cub are parallel to each other and do not intersect. However, in the coordinate system of the captured image, the extensions of edge La and edge Lb sometimes intersect. This point of intersection is called the vanishing point Vnp.

[0043] Figure 7 This diagram illustrates the relationship between the captured image and the actual object. As shown, the captured image plane Imp is the imaging plane of the image sensor in the imaging device 140, located at a focal length f from the center point (focal point) Org of the lens 142 in the imaging device 140. Furthermore, the actual lens 142 comprises multiple lenses, but... Figure 7 The middle part is the simplified label.

[0044] When an object P located at coordinates (X, Y, Z) in the world coordinate system with the center point Org as the origin is photographed by the imaging device 140, the object P is mapped to coordinates (X·f / Z, Y·f / Z, f) in the camera coordinate system.

[0045] Furthermore, the coordinate information in the camera coordinate system is represented in pixels on the image sensor, not in meters or other units like in the world coordinate system. Additionally, the information output from the image sensor typically loses the Z-component and uses the top-left corner of the image as the origin.

[0046] In the imaging device 140 of the information terminal device 10, the position of the center point Org of the lens 142 cannot be accurately determined in most cases, but the center point Cen of the captured image is basically consistent with the center point Org of the lens 142 (except for the Z component). Therefore, by shifting the coordinates of the image data output from the image sensor using half the value of the vertical pixel size and half the value of the horizontal pixel size in the image sensor, it is possible to transform it into the camera coordinate system.

[0047] For example, if the resolution of the captured image is 1920 pixels horizontally and 1080 pixels vertically, and a coordinate in the image output from the image sensor is (Xa, Ya) in pixels, then by moving the image by half the resolution in both the horizontal and vertical directions, it can be transformed into the camera coordinate system. Specifically, if the coordinate in the image is (Xa, Ya) in pixels, then it becomes (Xa-960, Ya-560) in the camera coordinate system.

[0048] Because it is difficult to uniformly analyze images with different focal lengths, a coordinate system called the normalized camera coordinate system is used in projection geometry. In this normalized camera coordinate system, the focal length f is normalized to "1". Coordinates such as (X·f / Z, Y·f / Z, f) in the camera coordinate system can be represented by coordinates such as (X / Z, Y / Z, 1) in the normalized camera coordinate system.

[0049] In fact, since the Z component information is lost in the camera coordinate system, the values ​​obtained by dividing the X and Y components by the focal length f are called the normalized camera coordinate system.

[0050] The normalized camera coordinate system has several advantages. One of these is that the vertical and horizontal vanishing points are orthogonal. This advantage will be explained below.

[0051] When the information terminal device 10 uses the imaging device 140 to capture a rectangular screen frame 32, the image in the image plane Imp is captured as follows: Figure 8 The quadrilateral Sf shown is represented by the following sides L1 to L4. Specifically, quadrilateral Sf in the image plane Imp is composed of four sides: side L1, which projects the screen frame 32's side La1; side L2, which projects the screen frame 32's side La2; side L3, which projects the screen frame 32's side La3; and side L4, which projects the screen frame 32's side La4.

[0052] In the actual screen frame 32, edges La1 and La2 are parallel, and edges La3 and La4 are also parallel. Edges La1 and La2 are orthogonal to edges La3 and La4.

[0053] exist Figure 7 or Figure 8 In the image sensor, the imaging plane is represented by a dashed line, but the image plane Imp, specified by the camera coordinate system, is an imaginary plane magnified in a way that includes the imaging plane. In the quadrilateral Sf obtained by projecting the screen frame 32 onto the image plane Imp, the intersection of the extensions of side L1 and L2 (the vertical vanishing point Vvnp) and the intersection of the extensions of side L3 and L4 (the horizontal vanishing point Hvnp) have the following relationship: Specifically, in the normalized camera coordinate system, the line Lv connecting the vertical vanishing point Vvnp to the origin (Org, the center point of the lens) and the line Lh connecting the horizontal vanishing point Hvnp to the origin (Org, the center point of the lens) are orthogonal at the center point Org.

[0054] The transformation from the camera coordinate system to the normalized camera coordinate system can be obtained by dividing each coordinate value in the camera coordinate system by the focal length f. Therefore, in other words, if the vertical vanishing point Vvnp is orthogonal to the horizontal vanishing point Hvnp by dividing each coordinate value by a certain value F, then it can be said that F is equal to the focal length f.

[0055] Here, let the coordinates of the vertical vanishing point Vvnp after movement, represented by the normalized camera coordinate system, be (Vx / f, Vy / f, 1), and the coordinates of the horizontal vanishing point Hvnp after movement, represented by the normalized camera coordinate system, be (Hx / f, Hy / f, 1). Furthermore, in the normalized camera coordinate system, the z-coordinate is "1" as described above.

[0056] The line Lv connecting the vertical vanishing point Vvnp (Vx / f, Vy / f, 1) to the center point Org and the line Lh connecting the horizontal vanishing point Hvnp (Hx / f, Hy / f, 1) to the center point Org are orthogonal at the origin. Therefore, the inner product of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp is zero. Thus, equation (1) holds.

[0057]

[0058] Here, when representing lines Lv and Lh uniformly, they are sometimes written as line Lvh.

[0059] If we multiply both sides of equation (1) by f2 to solve for f, we get equation (2), and thus we can find the focal length f.

[0060]

[0061] In addition, when the coordinates of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp are determined in the normalized camera coordinate system, the aspect ratio m of the screen frame 32 can be calculated as follows.

[0062] In detail, such as Figure 9 As shown, firstly, let the line connecting the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp be line Lvh. Next, the intersection of line Lvh and the extension of the diagonal Ld in quadrilateral Sf is determined as the diagonal vanishing point Dvnp. Then, if the coordinates (Dx, Dy, l) of the diagonal vanishing point Dvnp are obtained, the aspect ratio m of the screen frame 32 can be calculated by the following formula (3).

[0063]

[0064] Furthermore, the aspect ratio m refers to the ratio of the vertical dimension to the horizontal dimension in a rectangular shape (m:1), specifically 4 / 3 (=1.33), 16 / 9 (=1.78), etc. Additionally, the aspect ratio m does not necessarily refer to the ratio of the vertical dimension to the horizontal dimension (m:1); depending on the context, it sometimes refers to the ratio of the horizontal dimension to the vertical dimension (1:m).

[0065] exist Figure 9 In the example of quadrilateral Sf, the extension of the diagonal connecting the intersection of sides L1 and L4 with the intersection of sides L2 and L3 intersects the line Lvh. However, depending on the shape of quadrilateral Sf, sometimes the extension of the diagonal connecting the intersection of sides L1 and L3 with the intersection of sides L2 and L4 intersects the line Lvh.

[0066] Figure 10 This is a flowchart illustrating the specific actions within the shooting system 1.

[0067] First, in the information terminal device 10, when a user launches an application by clicking an icon or the like displayed on the display device 150, the camera control unit 102 causes the display device 150 to display a message urging the user to take a picture of the screen frame 32. Following this message, the user takes a picture of the screen frame 32 by operating a software button or the like displayed on the display device 150 (step S11).

[0068] The shooting control unit 102 transmits the image data Dt captured by the screen frame 32 to the communication control unit 106, and the communication control unit 106 controls the communication device 260 to send the image data Dt to the projector 20 (step S12).

[0069] In the information terminal device 10, after step S12, the execution of the application launched by the shooting control unit 102 ends.

[0070] On the other hand, in the projector 20, when the user or others turn on the power, the focal length f and aspect ratio m are calculated and processed as described below.

[0071] First, the processing control unit 201 instructs the projection control unit 206 to obtain the current aspect ratio (step S21). Specifically, the current aspect ratio refers to the aspect ratio of the image size specified by the image signal provided from the host device. For example, if the image size specified by image data Dt is 1920 pixels wide and 1080 pixels high, then the projection control unit 206 obtains information indicating an aspect ratio of 1.78 (=1920 / 1080).

[0072] When the projection control unit 206 obtains the aspect ratio at the current moment, it provides the aspect ratio information to the processing control unit 201.

[0073] When the processing control unit 201 obtains the aspect ratio at the current time point from the projection control unit 206, it instructs the image acquisition unit 202 to receive image data Dt from the information terminal device 10 (step S22). When the communication device 260 receives the image data Dt, the image acquisition unit 202 notifies the processing control unit 201 of the received information.

[0074] The processing control unit 201 causes the image acquisition unit 202 to transfer the received image data Dt to the quadrilateral determination unit 203. The processing control unit 201 instructs the quadrilateral determination unit 203 to perform the following analysis on the quadrilateral Sf contained in the image represented by the image data Dt.

[0075] In detail, the processing control unit 201 instructs the quadrilateral determination unit 203 to obtain information for determining the four sides L1 to L4 of the quadrilateral Sf (step S23). While the actual sides of the screen frame 32 are straight lines, the sides L1 to L4 of the quadrilateral Sf represented by the image data Dt may sometimes be curves due to aberrations, distortions, etc., in the optical system of the imaging device 140. Therefore, the information used to determine the four sides L1 to L4 may be either information for determining straight lines corresponding to the sides L1 to L4, or information for determining regression curves or approximate straight lines corresponding to the sides L1 to L4.

[0076] The processing control unit 201 instructs the quadrilateral determination unit 203 to output the information of determining sides L1 to L4 to the coordinate determination unit 204.

[0077] When the information of edges L1 to L4 is output to the coordinate determination unit 204, the processing control unit 201 instructs the coordinate determination unit 204 to determine the coordinates of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp (step S24). According to this instruction, the coordinate determination unit 204 takes the intersection of the extension of edge L1 and the extension of edge L2 as the vertical vanishing point Vvnp and determines the coordinates of the vertical vanishing point Vvnp. In addition, it takes the intersection of the extension of edge L3 and the extension of edge L4 as the horizontal vanishing point Hvnp and determines the coordinates of the horizontal vanishing point Hvnp.

[0078] Furthermore, after determining the coordinates, the coordinate determination unit 204 moves the coordinates of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp in parallel, so that the center point Cen of the captured image becomes the origin. The processing control unit 201 causes the coordinate determination unit 204 to output the moved coordinates of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp.

[0079] Furthermore, depending on the shape of quadrilateral Sf, it is sometimes impossible to determine the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp. In this case, the coordinate determination unit 204 notifies the processing control unit 201 that it is impossible to determine the two vanishing points.

[0080] The processing control unit 201 determines whether the number of vanishing points determined is "2" based on the output or notification from the coordinate determination unit 204 (step S25). If the number of vanishing points is "2", that is, the determination result of step S24 is "yes", then the processing control unit 201 instructs the calculation unit 205 to perform the following calculation (step S26). In detail, the processing control unit 201 instructs the calculation unit 205 to substitute the coordinates of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp output from the coordinate determination unit 204 into equation (2) to calculate the focal length f of the imaging device 140.

[0081] Next, the processing control unit 201 instructs the coordinate determination unit 204 to determine the coordinates of the intersection point of the straight line Lvh connecting the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp with the extension line Ld of the diagonal in quadrilateral Sf, i.e., the diagonal vanishing point Dvnp. Then, the processing control unit 201 transfers the determined coordinates of the diagonal vanishing point Dvnp to the calculation unit 205, and substitutes the coordinates of the diagonal vanishing point Dvnp, the vertical vanishing point Vvnp, and the horizontal vanishing point Hvnp into equation (3) to calculate the aspect ratio m of the screen frame 32 (step S27).

[0082] The calculation unit 205 provides the calculated aspect ratio m to the processing control unit 201.

[0083] The processing control unit 201 determines whether the difference between the aspect ratio obtained in step S21 and the aspect ratio m provided in step S27 is greater than or equal to a threshold, that is, it determines whether the difference between the aspect ratio of the projected image and the aspect ratio m of the screen frame 32 obtained by calculation is greater than or equal to a threshold (step S28).

[0084] If the difference is above the threshold, that is, the judgment result of step S29 is "yes", then the processing control unit 201 decides to maintain the aspect ratio of the projected image as obtained in step S21 (step S29).

[0085] If the difference is less than the threshold, that is, the judgment result of step S29 is "no", then the processing control unit 201 changes the aspect ratio of the projected image to the aspect ratio of the screen frame 32 (step S30).

[0086] The processing control unit 201 instructs the projection control unit 206 to project the image signal provided from the main device at the maintained aspect ratio or the changed aspect ratio (step S31). According to this instruction, the projection control unit 206 causes the projection device 250 to project an image based on the image signal at the aspect ratio.

[0087] If the difference between the aspect ratio of the image represented by the image signal and the calculated aspect ratio m is greater than or equal to a threshold, the aspect ratio of the image represented by the image signal is prioritized for projection. On the other hand, if the difference is less than the threshold, the aspect ratio of the image represented by the image signal is changed to match the aspect ratio m of the screen frame 32, thus enabling projection that effectively utilizes the screen frame 32. Furthermore, in this case, although the aspect ratio is changed, the change is small enough not to cause any sense of disharmony for the user.

[0088] Furthermore, in step S25 above, if the number of vanishing points determined by the processing control unit 201 based on the notification from the coordinate determination unit 204 is not "2", that is, if the determination result of step S25 is "No", the coordinate value of the vanishing point is too large or the quadrilateral is trapezoidal, etc. In this case, the processing control unit 201 uses the predetermined focal length as the focal length of the imaging device 140 in the information terminal device 10 (step S32).

[0089] In projector 20, after step S31 or S32, the calculation and processing of focal length f and aspect ratio m ends, but the projection based on the image signal continues.

[0090] As the information terminal device 10 uses various devices such as smartphones and tablets, it is generally difficult for ordinary users to know the focal length f of the camera device 140 built into the information terminal device 10. However, according to the camera system 1 of this embodiment, the camera device 140 of the information terminal device 10 captures the screen frame 32, and the captured image is analyzed to calculate the focal length f of the camera device 140. Therefore, it is not necessary to capture the grid, and it can be done without a lot of labor and time.

[0091] <Application Examples / Variations>

[0092] The embodiments illustrated above can be modified in various ways. The following examples illustrate specific modifications that can be applied to the embodiments.

[0093] In this implementation, the focal length f of the shooting device 140 in the information terminal device 10 and the aspect ratio of the projection area surrounded by the screen frame 32 can also be used when performing keystone correction and adjusting the magnification / reduction rate in the projector 20.

[0094] In one embodiment, the projector 20 has the functions of analyzing a quadrilateral Sf and calculating the focal length f based on coordinate values. However, the information terminal device 10 may also have the functions of analyzing a quadrilateral Sf and calculating the focal length f based on coordinate values, in addition to the shooting function. Specifically, the information terminal device 10 may also have the image acquisition unit 202, quadrilateral determination unit 203, coordinate determination unit 204, and calculation unit 205 of the projector 20.

[0095] Furthermore, the image acquisition unit 202, the quadrilateral determination unit 203, the coordinate determination unit 204, and the calculation unit 205 can also be configured to enable the cloud server to perform these functions. For example, it can also be configured to send the image data Dt captured by the information terminal device 10 to the server via a communication network, and send the aspect ratio obtained by the server in the processing steps S22 to S27 to the projector 20.

[0096] <Postscript>

[0097] Based on the above description, the preferred embodiments of this disclosure are as follows. It should be noted that, for ease of understanding, the reference numerals in the accompanying drawings are listed in parentheses below, but this does not mean that the invention is limited to the illustrated embodiments.

[0098] <Postscript 1>

[0099] One method for calculating focal length includes: obtaining image data (Dt) obtained by photographing a rectangular subject (32); determining, in the image represented by the image data (Dt), the first side (L1), the second side (L2) opposite to the first side (L1), the third side (L3), and the fourth side (L4) opposite to the third side (L3) of the quadrilateral (Sf) corresponding to the subject (32); determining the coordinates of the first point (Vvnp) which is the intersection of the extension of the first side (L1) and the extension of the second side (L2), and the coordinates of the second point (Hvnp) which is the intersection of the extension of the third side (L3) and the extension of the fourth side (L4); and calculating the focal length (f) of the device (10) that photographs the subject (32) based on the coordinates of the first point (Vvnp) and the second point (Hvnp). According to method 1, it is not necessary to photograph the grid board in order to determine the focal length (f) of the device (10) for photographing the subject, which requires less effort.

[0100] <Appendix 2>

[0101] In the specific method 2 of Method 1, the focal length calculation involves finding the focal length (f) when the inner product of the first coordinate value (Vx / f, Vy / f, 1) obtained by transforming the coordinates of the first point (Vvnp) to the normalized camera coordinate system and the second coordinate value (Hx / f, Hy / f, 1) obtained by transforming the coordinates of the second point (Hvnp) to the normalized camera coordinate system is zero. The specific calculation of the focal length (f) is implemented according to Method 2.

[0102] <Appendix 3>

[0103] In another specific method 3 of method 1, when the coordinates of the first point (Vvnp) are (Vx, Vy), the coordinates of the second point (Hvnp) are (Hx, Hy), and the focal length is f, then...

[0104] f = {-(HxVx + HyVy)} 1 / 2

[0105] Calculate the focal length (f). Implement the specific calculation of the focal length (f) according to method 3.

[0106] <Appendix 4>

[0107] The display method of the projector in Method 4 includes: acquiring image data (Dt) obtained by photographing a rectangular projection area (32); determining, in the image represented by the image data (Dt), the first side (L1), the second side (L2) opposite to the first side (L1), the third side (L3), and the fourth side (L4) opposite to the third side (L3) of the quadrilateral (Sf) corresponding to the projection area (32); acquiring the coordinates (Vx, Vy) of the first point (Vvnp) at the intersection of the extension of the first side (L1) and the extension of the second side (L2), and the coordinates (Hx, Hy) of the second point (Hvnp) at the intersection of the extension of the third side (L3) and the extension of the fourth side (L4); based on the first point (Vvnp) The focal length (f) of the device (10) for photographing the projection area (32) is calculated based on the coordinates (Vx, Vy) of the first point (Vvnp) and the coordinates (Hx, Hy) of the second point (Hvnp). The coordinates (Dx, Dy) of the third point (Dvnp) are determined based on the intersection of the straight line (Lvh) connecting the first point (Vvnp) and the second point (Hvnp) with the extension of the diagonal (Ld) of the quadrilateral (Sf). Based on the coordinates (Vx, Vy) of the first point (Vvnp), the coordinates (Hx, Hy) of the second point (Hvnp), and the coordinates (Dx, Dy) of the third point (Dvnp), the aspect ratio (m) of the projection area (32) is calculated. Based on the calculated aspect ratio (m) of the projection area (32), the image is projected.

[0108] According to method 4, it is not necessary to photograph the grid plate in order to determine the focal length (f) of the device (10) for photographing the projection area (32), thus requiring less effort. In addition, it is possible to perform appropriate projection based on the aspect ratio (m) of the projection area (32) calculated by calculation.

[0109] <Appendix 5>

[0110] In the projector display method of Method 5, which is a specific embodiment of Method 4, if the difference between the aspect ratio of the image represented by the image signal and the aspect ratio (m) of the projection area (32) is above a threshold, the image is projected according to the aspect ratio of the image represented by the image signal. According to Method 5, in this case, the aspect ratio of the image represented by the image signal is prioritized for projection.

[0111] <Appendix 6>

[0112] In the display method of the projector in another specific embodiment of Method 4, Method 6, if the difference between the aspect ratio of the image represented by the image signal and the aspect ratio (m) of the projection area (32) is less than a threshold, the projector will project an image whose aspect ratio is changed to that of the projection area (32). According to Method 6, in this case, the image is displayed over the entire area of ​​the projection area (32), and even if the aspect ratio of the image represented by the image signal is changed, the change is small, thus avoiding any sense of disharmony for the user.

[0113] <Appendix 7>

[0114] The shooting system (1) of method 7 includes a processor (200), which performs the following processing: acquiring image data (Dt) obtained by shooting a rectangular subject (32); determining, in the image represented by the image data (Dt), the first side (L1) of the quadrilateral (Sf) corresponding to the subject (32), the second side (L2) opposite to the first side (L1), the third side (L3), and the fourth side (L4) opposite to the third side (L3); and acquiring the first side as the fourth side (Sf). The coordinates (Vx, Vy) of the first point (Vvnp), the intersection of the extension of the second side (L2), and the coordinates (Hx, Hy) of the second point (Hvnp), the intersection of the extension of the third side (L3) and the extension of the fourth side (L4), are used to calculate the focal length (f) of the device (10) that photographs the subject (32). According to method 7, it is not necessary to photograph the grid board in order to determine the focal length (f) of the device (10) that photographs the subject (32), thus requiring less effort.

Claims

1. A focal length calculation method, comprising: Image data is obtained by taking a picture of a rectangular screen frame that surrounds the projection area of ​​the image projected by the projector; In the image represented by the image data, determine the first side, the second side, the third side opposite to the first side, and the fourth side opposite to the third side of the quadrilateral corresponding to the screen frame; Determine the coordinates of the first point, which is the intersection of the extension of the first side and the extension of the second side, and the coordinates of the second point, which is the intersection of the extension of the third side and the extension of the fourth side. Based on the coordinates of the first point and the second point, the focal length of the device that captured the screen frame was calculated. Determine the coordinates of the third point, which is the intersection of the line connecting the first point and the second point with the extension of the diagonal of the quadrilateral; as well as Based on the coordinates of the first point, the second point, and the third point, the aspect ratio of the screen frame is calculated. The calculated focal length and the aspect ratio of the screen frame are used to control the projection of the image by the projector.

2. The focal length calculation method according to claim 1, wherein, The calculation of the focal length includes finding the focal length in which the inner product of the first coordinate value and the second coordinate value is zero. The first coordinate value is obtained by transforming the coordinate value of the first point to the normalized camera coordinate system, and the second coordinate value is obtained by transforming the coordinate value of the second point to the normalized camera coordinate system.

3. The focal length calculation method according to claim 1, wherein, Given that the coordinates of point 1 are (Vx, Vy), the coordinates of point 2 are (Hx, Hy), and the focal length is f, the focal length f is calculated using the following formula: f={-(HxVx+HyVy)} 1 / 2 。 4. A display method for a projector, comprising: Image data is obtained by capturing images of a rectangular screen frame that surrounds the projection area of ​​the image projected by the projector; In the image represented by the image data, determine the first side, the second side, the third side opposite to the first side, and the fourth side opposite to the third side of the quadrilateral corresponding to the screen frame; Determine the coordinates of the first point, which is the intersection of the extension of the first side and the extension of the second side, and the coordinates of the second point, which is the intersection of the extension of the third side and the extension of the fourth side. Based on the coordinates of the first point and the second point, the focal length of the device that captured the screen frame is calculated. Determine the coordinates of the third point, which is the intersection of the line connecting the first point and the second point with the extension of the diagonal of the quadrilateral; The aspect ratio of the screen frame is calculated based on the coordinates of the first point, the second point, and the third point. Determine whether the difference between the aspect ratio of the image represented by the image signal and the aspect ratio of the screen frame is greater than or equal to a threshold. If the difference is determined to be above a threshold, the image is projected according to the aspect ratio of the image represented by the image signal; as well as If the difference is determined to be less than a threshold, the image is projected with the aspect ratio of the screen frame changed.

5. A shooting system comprising a processor, The processor performs the following processing: Image data is obtained by capturing images of a rectangular screen frame that surrounds the projection area of ​​the image projected by the projector; In the image represented by the image data, determine the first side, the second side, the third side opposite to the first side, and the fourth side opposite to the third side of the quadrilateral corresponding to the screen frame; Determine the coordinates of the first point, which is the intersection of the extension of the first side and the extension of the second side, and the coordinates of the second point, which is the intersection of the extension of the third side and the extension of the fourth side. Based on the coordinates of the first point and the second point, the focal length of the device that captured the screen frame was calculated. Determine the coordinates of the third point, which is the intersection of the line connecting the first point and the second point with the extension of the diagonal of the quadrilateral; The aspect ratio of the screen frame is calculated based on the coordinates of the first point, the second point, and the third point. Determine whether the difference between the aspect ratio of the image represented by the image signal and the aspect ratio of the screen frame is greater than or equal to a threshold. If it is determined that the difference is above a threshold, the projector projects the image with the aspect ratio of the image represented by the image signal; as well as If the difference is determined to be less than a threshold, the projector projects an image with the aspect ratio of the screen frame changed.

Images