Focal length calculation method, projector display method, and imaging system

Abstract

To obtain, for example, the focal length of an imaging device by a simple operation.SOLUTION: An imaging system includes an image acquisition unit 201 that acquires image data obtained by photographing a rectangular object, a quadrilateral identification unit 203 that identifies a side L1 of a quadrilateral corresponding to a subject in an image represented by image data, a side L2 facing the side L1, a side L3, and a side L4 facing the side L3, a coordinate identification unit 204 that identifies the coordinate values of the vertical vanishing point which is the intersection of the extension line of the side L1 and the extension line of the side L2, and the coordinate value of the horizontal vanishing point which is the intersection of the extension line of the side L3 and the extension line of the side L4, and a calculation unit 205 that calculates the focal length of an imaging device that images the subject on the basis of the coordinate values of the vertical vanishing point and the coordinate values of the horizontal vanishing point.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a focal length calculation method, a display method for a projector, and an imaging system. [Background technology]

[0002] Recent projectors are small, lightweight, and easily portable, and therefore are used in a variety of locations. For this reason, it is necessary to correct the projected image depending on the location and screen where the projector is used. Therefore, a technique is known in which a camera is attached to a projector to capture a projected image and correct the projected image based on the captured image, as described in Patent Document 1. In this technique, internal parameters such as the focal length of the camera are treated as if they were predetermined.

[0003] In recent years, most information terminal devices such as smartphones have a photography function. Therefore, it is conceivable to use images captured by the information terminal device, but it is difficult for general users to know the internal parameters of the camera built into the information terminal device. For this reason, a technology is known, such as that described in Patent Document 2, in which an image of a checkerboard captured by a camera is used to determine the internal parameters of the camera, such as the focal length. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2013 / 038656 [Patent Document 2] Patent Publication No. 2021-1113020 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technique described in Patent Document 2 requires taking pictures of checkerboards placed in various positions with a camera, which is a problem in that it requires a lot of work. [Means for solving the problem]

[0006] A focal length calculation method according to one embodiment of the present disclosure includes acquiring image data of a rectangular subject; identifying a first side of a quadrilateral corresponding to the subject in an image represented by the image data, a second side opposite the first side, a third side opposite the third side, and a fourth side opposite the third side; identifying coordinate values ​​of a first point that is an intersection of an extension of the first side and an extension of the second side, and coordinate values ​​of a second point that is an intersection of an extension of the third side and an extension of the fourth side; and calculating a focal length of a device that photographed the subject based on the coordinate values ​​of the first point and the coordinate values ​​of the second point.

[0007] A display method of a projector according to one aspect of the present disclosure includes acquiring image data of an image of a rectangular projection area, identifying a first side of a quadrangle corresponding to the projection area in an image represented by the image data, a second side opposite the first side, a third side opposite the third side, and a fourth side opposite the third side, identifying coordinate values ​​of a first point that is an intersection of an extension of the first side and an extension of the second side, and a second point that is an intersection of an extension of the third side and an extension of the fourth side, calculating a focal length of a device that captured the projection area based on the coordinate values ​​of the first point and the coordinate values ​​of the second point, identifying coordinate values ​​of a third point that is an intersection of a line connecting the first point and the second point and an extension of a diagonal of the quadrangle, calculating an aspect ratio of the projection area based on the coordinate values ​​of the first point, the coordinate values ​​of the second point, and the coordinate values ​​of the third point,

[0008] A photography system according to one embodiment of the present disclosure includes a processor that performs the following operations: acquires image data of a rectangular subject; identifies a first side of a quadrilateral corresponding to the subject in an image represented by the image data, a second side opposite the first side, a third side opposite the first side, and a fourth side opposite the third side; identifies coordinate values ​​of a first point that is an intersection of an extension of the first side and an extension of the second side, and coordinate values ​​of a second point that is an intersection of an extension of the third side and an extension of the fourth side; and calculates a focal length of a device that photographed the subject based on the coordinate values ​​of the first point and the coordinate values ​​of the second point. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a configuration of an imaging system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of an information terminal device in the imaging system. [Figure 3] FIG. 2 is a block diagram showing the software functional configuration of the information terminal device. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of a projector in the imaging system. [Figure 5] FIG. 2 is a block diagram showing the functional configuration of software of the projector. [Figure 6] FIG. [Figure 7] FIG. 2 is a diagram illustrating a camera coordinate system and a world coordinate system. [Figure 8] FIG. 1 illustrates the orthogonality of vanishing points in a normalized camera coordinate system. [Figure 9] FIG. 10 is a diagram for explaining diagonal vanishing points. [Figure 10] The operation of the information terminal device and the projector in the imaging system DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments of the present invention will be described. Shooting SystemThe following description will be given with reference to the drawings. It should be noted that the dimensions and scale of each part in each drawing are appropriately different from the actual dimensions. Furthermore, the embodiments described below are preferred examples, and therefore various technically preferable limitations are applied. However, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited thereto.

[0011] FIG. 1 is a diagram showing the configuration of an imaging system 1 according to an embodiment. The imaging system 1 includes an information terminal device 10 and a projector 20. The information terminal device 10 is, for example, a smartphone or a portable tablet device, and has imaging and communication functions. In this embodiment, the information terminal device 10 captures an image of the shape of the screen 30, more specifically, an area of ​​the screen 30 onto which an image is to be projected by the projector 20.

[0012] For ease of explanation, the area of ​​the screen 30 onto which an image is to be projected by the projector 20 will be referred to simply as the projection area. The screen 30 shown in the figure is a black mask screen surrounded by black borders on all four sides. Therefore, strictly speaking, the projection area is a white area surrounded by a rectangular screen frame 32, which is a black mask. However, specifying the shape of the screen frame 32 is essentially equivalent to specifying the projection area surrounded by the screen frame 32. The screen 30 may be a type with a completely white background without a black mask. If it is a type without a black mask, the projection area can be identified, for example, from the difference in contrast between the white background and the back surface (wall surface).

[0013] The projector 20 enlarges and projects an image based on an image signal supplied from a host device (not shown) onto the screen 30. In the figure, the projector 20 is placed on the top surface of a podium Tp, and the screen 30 is hung from the wall.

[0014] 2 is a diagram showing the hardware configuration of the information terminal device 10. The information terminal device 10 includes a central processing unit 100, a storage device 120, a connection device 130, an image capturing device 140, a display device 150, and a communication device 160.

[0015] The central processing unit 100 is a processor, and is configured with one or more processing circuits such as a CPU (Central Processing Unit), and controls all elements of the information terminal device. Note that the central processing unit 100 may be configured with circuits such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) in addition to a CPU.

[0016] The storage device 120 is one or more memories configured with known recording media such as magnetic recording media or semiconductor recording media, and stores programs executed by the central processing unit 100, various data used by the central processing unit 100, and captured image data. The storage device 120 may be configured by combining multiple types of recording media. Alternatively, the storage device 120 may be a portable recording medium that is detachable from the information terminal device 10, or an external recording medium (e.g., online storage) that the information terminal device 10 can communicate with via a communication network.

[0017] The connection device 130 is an interface to which the image capture device 140, the display device 150, and the communication device 160 are connected. The connection device 130 includes, for example, an interface circuit. The image capture device 140 has a lens, an image sensor, and the like, and is capable of capturing an image, for example, by user operation. In this embodiment, the image capture device 140 particularly captures an image of the screen frame 32. The image capture device 140 is, for example, a camera. The data of the image captured by the image capturing device 140 is supplied to the display device 150, and an image based on the data is displayed on the display device 150 for the user's confirmation. Re The display device 150 includes, for example, a display panel.

[0018] The communication device 160 is a device for communicating information with other devices other than the information terminal device 10. Specifically, in this embodiment, the communication device 160 transmits image data of an image including the screen frame 32 captured by the imaging device 140 to the projector 20. The communication device 160 includes, for example, an antenna for wireless communication and a connection terminal for wired communication. In this description, the term "apparatus" may be replaced with other terms such as circuit, device, or unit. Each element of the information terminal device 10 may be composed of one or more devices. Some elements of the information terminal device 10 may be omitted.

[0019] 3 is a block diagram showing functions implemented by the central processing unit 100 in the information terminal device 10. As shown in this figure, the central processing unit 100 executes various programs stored in the storage device 120 to implement functions such as an imaging control unit 102 and a communication control unit 103. 106 is constructed.

[0020] 4 is a diagram showing the hardware configuration of the projector 20. The projector 20 includes a central processing unit 200, a storage device 220, a connection device 230, a projection device 250, and a communication device 260. The central processing unit 200, storage device 220, connection device 230, and communication device 260 in the projector 20 are configured from devices similar to the central processing unit 100, storage device 120, connection device 130, and communication device 160 in the information terminal device 10 described above.

[0021] The projection device 250 in the projector 20 is an optical engine that enlarges and projects an image based on an image signal supplied from a host device (not shown) onto the screen 30. The projection device includes, for example, a light source, a light modulation device, and a projection lens (not shown). The light modulation device can be, for example, a liquid crystal panel or a digital mirror device. In this embodiment, the projection device 250 is not important, so a detailed description thereof will be omitted.

[0022] FIG. 5 is a block diagram showing functions implemented in the central processing unit 200 in the projector 20. As shown in FIG. As shown in this figure, the central processing unit 200 executes various programs stored in the storage device 220 to construct a processing control unit 201, an image acquisition unit 202, a four-side identification unit 203, a coordinate identification unit 204, a calculation unit 205, and a projection control unit 206.

[0023] In this embodiment, the principle of calculating the focal length of the image capturing device 140 will be described. 6 is a diagram showing an image of a rectangular parallelepiped Cub, for example, captured by the image capture device 140 when viewed from the side. In the actual coordinate system (world coordinate system), sides La and Lb of the rectangular parallelepiped Cub are parallel to each other and do not intersect, but in the coordinate system of the captured image, the extension of side La and the extension of side Lb may intersect. This intersection point is called the vanishing point Vnp.

[0024] 7 is a diagram illustrating the relationship between a captured image and an actual object. As shown in this diagram, the captured image plane Imp is the imaging plane of the image sensor in the image capture device 140, and is located at a point away from the center point (focal point) Org of the lens 142 in the image capture device 140 by the focal length f of the lens 142. Note that the actual lens 142 is a combination of multiple lenses, but FIG. 7 shows a simplified representation.

[0025] When an object P located at coordinates (X, Y, Z) in a world coordinate system with the center point Org as the origin is photographed by the photographing device 140, the object P is projected to coordinates (X·f / Z, Y·f / Z, f) in the camera coordinate system.

[0026] It should be noted that information indicating coordinates in the camera coordinate system is in units of pixels on the image sensor, and is not in units such as meters as in the world coordinate system. In addition, the information output from an image sensor generally loses the Z component and is output with the top left corner of the image as the origin. In the image capturing device 140 of the information terminal device 10, the position of the center point Org of the lens 142 is often not accurately known, but basically the center point Cen of the captured image coincides with the center point Org of the lens 142 (excluding the Z component). Therefore, the coordinates of the image data output from the image sensor can be converted into the camera coordinate system by shifting them by half the vertical pixel size and half the horizontal pixel size of the image sensor.

[0027] For example, if the resolution of a captured image is 1920 pixels horizontally and 1080 pixels vertically, and a certain coordinate in the image output from the image sensor is (Xa, Ya) in pixel units, it can be converted to the camera coordinate system by shifting it horizontally and vertically by half the resolution. Specifically, if the coordinate in the image is (Xa, Ya) in pixel units, it becomes coordinate (Xa-960, Ya-560) in the camera coordinate system.

[0028] Since it is difficult to uniformly analyze images with different focal lengths, a coordinate system called the normalized camera coordinate system is used in projective geometry. In this normalized camera coordinate system, the focal length f is normalized to 1. The coordinate (X·f / Z, Y·f / Z, f) in the camera coordinate system can be expressed as the coordinate (X / Z, Y / Z, 1) in the normalized camera coordinate system. In reality, the information of the Z component is lost in the camera coordinate system, so the normalized camera coordinate system is obtained by dividing the X and Y components by the focal length f.

[0029] The normalized camera coordinate system has several useful features. One of them is that the vertical vanishing point and the horizontal vanishing point are orthogonal. This will be explained below. When rectangular screen frame 32 is photographed by imaging device 140 of information terminal device 10, it is represented on image plane Imp by a quadrilateral Sf as shown in Fig. 8. This quadrilateral Sf is made up of the following sides L1 to L4. In detail, quadrilateral Sf has four sides on image plane Imp: side L1, which is a projection of side La1 of screen frame 32; side L2, which is a projection of side La2; side L3, which is a projection of side La3; and side L4, which is a projection of side La4. In the actual screen frame 32, sides La1 and La2 are parallel to each other, sides La3 and La4 are also parallel to each other, and sides La1 and La2 are perpendicular to sides La3 and La4.

[0030] In FIG. 7 or 8, the imaging plane of the image sensor is indicated by a dashed line, but the image plane Imp specified in the camera coordinate system is a virtual plane expanded to include the imaging plane. In the quadrilateral Sf obtained by projecting the screen frame 32 onto the image plane Imp, the vertical vanishing point Vvnp, which is the intersection of the extension line of side L1 and the extension line of side L2, and the horizontal vanishing point Hvnp, which is the intersection of the extension line of side L3 and the extension line of side L4, have the following relationship: In detail, in the normalized camera coordinate system, the straight line Lv connecting the vertical vanishing point Vvnp to the origin, which is the center point Org of the lens, and the straight line Lh connecting the horizontal vanishing point Hvnp to the origin, which is the center point Org of the lens, intersect at right angles at the center point Org. The transformation from the camera coordinate system to the normalized camera coordinate system can be found by dividing each coordinate value in the camera coordinate system by the focal length f. In other words, by dividing each coordinate value by a certain value F, if the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp are perpendicular, then F can be said to be equal to the focal length f.

[0031] Here, the coordinates of the vertical vanishing point Vvnp after movement expressed in the normalized camera coordinate system are (Vx / f, Vy / f, 1), and the coordinates of the horizontal vanishing point Hvnp after movement expressed in the normalized camera coordinate system are (Hx / f, Hy / f, 1). Note that in the normalized camera coordinate system, the z coordinate is "1" as described above.

[0032] The line Lv connecting the vertical vanishing point Vvnp (Vx / f, Vy / f, 1) to the center point Org and the line Lh connecting the horizontal vanishing point Hvnp (Hx / f, Hy / f, 1) to the center point Org are perpendicular at the origin, so The coordinate values ​​of the two vanishing points The inner product becomes zero. Therefore, the following equation (1) holds.

number

[0033] Here, when expressing the straight lines Lv and Lh collectively, they may be written as a straight line Lvh. Multiplying both sides of equation (1) by f2 and solving for f gives the following equation (2), which allows us to find the focal length f.

number

[0034] Furthermore, once the coordinates of the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp are determined in the normalized camera coordinate system, the aspect ratio m of the screen frame 32 can be calculated as follows. 9, first, a straight line Lvh is defined as connecting the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp. Next, the intersection of the straight line Lvh and an extension Ld of the diagonal line of the quadrangle Sf is defined as a diagonal vanishing point Dvnp. Next, once the coordinates (Dx, Dy, 1) of the diagonal vanishing point Dvnp are determined, the aspect ratio m of the screen frame 32 can be calculated using the following equation (3).

[0035]

number

[0036] The aspect ratio m refers to the ratio of the vertical size to the horizontal size in a rectangular shape (m:1), and is specifically 4 / 3 (=1.33), 16 / 9 (=1.78), etc. Also, the aspect ratio m does not necessarily refer to the ratio of the vertical size to the horizontal size (m:1), and in some cases may refer to the ratio of the horizontal size to the vertical size (1:m).

[0037] In FIG. 9, an example is shown in which the extension line Ld of the diagonal line connecting the intersection point of the sides L1 and L4 and the intersection point of the sides L2 and L3 intersects with the straight line Lvh. However, depending on the shape of the quadrilateral Sf, the extension line of the diagonal line connecting the intersection point of the sides L1 and L3 and the intersection point of the sides L2 and L4 may intersect with the straight line Lvh. Ruko It also says:

[0038] FIG. 10 is a flowchart showing a specific operation of the photography system 1. First, in the information terminal device 10, when the user starts an application program by tapping an icon displayed on the display device 150, the imaging control unit 102 causes the display device 150 to display a message prompting the user to capture an image of the screen frame 32. Following this message, the user operates a software button or the like displayed on the display device 150 to capture an image of the screen frame 32 (step S11). The photographing control unit 102 transfers the image data Dt of the photographed screen frame 32 to the communication control unit 106, and the communication control unit 106 controls the communication device 260 to transmit the image data Dt to the projector 20 (step S12). In the information terminal device 10, after step S12, the execution of the application program started by the imaging control unit 102 ends.

[0039] On the other hand, when the projector 20 is turned on by a user or the like, the following calculation process is performed to determine the focal length f and the aspect ratio m. First, the process control unit 201 instructs the projection control unit 206 to acquire the current aspect ratio (step S21). The current aspect ratio is specifically the aspect ratio of the image size specified by the image signal supplied from the host device. For example, if the image size specified by the image data Dt is 1920 pixels wide and 1080 pixels high, the projection control unit 206 acquires information indicating that the aspect ratio is 1.78 (=1920 / 1080). When the projection control unit 206 acquires the current aspect ratio, it supplies information about the aspect ratio to the processing control unit 201 .

[0040] When the processing control unit 201 obtains the current aspect ratio from the projection control unit 206, it instructs the image acquisition unit 202 to receive image data Dt from the information terminal device 10 (step S22). When the communication device 260 receives the image data Dt, the image acquisition unit 202 notifies the processing control unit 201 that the image data Dt has been received. The process control unit 201 causes the image acquisition unit 202 to transfer the received image data Dt to the four-side identification unit 203. The process control unit 201 instructs the four-side identification unit 203 to perform the following analysis on the quadrangle Sf included in the image represented by the image data Dt.

[0041] In detail, the process control unit 201 instructs the four-side identification unit 203 to obtain information for identifying the four sides L1 to L4 of the quadrilateral Sf (step S23). While each side of the actual screen frame 32 is a straight line, the sides L1 to L4 of the quadrilateral Sf represented by the image data Dt may not necessarily be straight lines but may be curved lines due to the influence of aberrations, distortions, and the like of the optical system in the imaging device 140. For this reason, the information for identifying the four sides L1 to L4 may be information for identifying straight lines corresponding to the sides L1 to L4, or may be information for identifying regression curves or approximate straight lines corresponding to the sides L1 to L4. The process control unit 201 instructs the four side identification unit 203 to output to the coordinate identification unit 204 information identifying the sides L1 to L4.

[0042] When the information identifying sides L1 to L4 is output to the coordinate identification unit 204, the process control unit 201 instructs the coordinate identification unit 204 to identify the coordinates of the vertical vanishing point Vvnp and the coordinates of the horizontal vanishing point Hvnp (step S24). In response to this instruction, the coordinate identification unit 204 determines the intersection of the extension line of side L1 and the extension line of L2 as the vertical vanishing point Vvnp and identifies the coordinates of this vertical vanishing point Vvnp, and also determines the intersection of the extension line of side L3 and the extension line of L4 as the horizontal vanishing point Hvnp and identifies the coordinates of this horizontal vanishing point Hvnp.

[0043] After specifying the coordinates, the coordinate specification unit 204 translates the coordinates of the vertical vanishing point Vvnp and the coordinates of the horizontal vanishing point Hvnp so that the center point Cen of the captured image becomes the origin position. The process control unit 201 causes the coordinate specification unit 204 to output the coordinates of the vertical vanishing point Vvnp and the coordinates of the horizontal vanishing point Hvnp after the movement. Furthermore, depending on the shape of the quadrilateral Sf, it may be impossible to identify the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp. In such cases, the coordinate identification unit 204 notifies the process control unit 201 that the two vanishing points could not be identified.

[0044] The process control unit 201 determines whether the number of identified vanishing points is "2" from the output or notification from the coordinate identification unit 204 (step S25). If the number of vanishing points is "2", that is, if the determination result in step S24 is "Yes", the process control unit 201 instructs the calculation unit 205 to perform the following calculation (step S26). In detail, the process control unit 201 causes the calculation unit 205 to substitute the coordinates of the vertical vanishing point Vvnp and the coordinates of the horizontal vanishing point Hvnp output from the coordinate identification unit 204 into equation (2) to calculate the focal length f of the image capturing device 140.

[0045] Next, the process control unit 201 causes the coordinate determination unit 204 to determine the coordinates of the diagonal vanishing point Dvnp, which is the intersection of the straight line Lvh connecting the vertical vanishing point Vvnp and the horizontal vanishing point Hvnp and the extension line Ld of the diagonal line in the quadrangle Sf. Then, the process control unit 201 transfers the coordinates of the determined diagonal vanishing point Dvnp to the calculation unit 205, and causes the calculation unit 205 to calculate the aspect ratio m of the screen frame 32 by substituting the coordinates of the diagonal vanishing point Dvnp, the coordinates of the vertical vanishing point Vvnp, and the coordinates of the horizontal vanishing point Hvnp into equation (3) (step S27). The calculation unit 205 supplies the calculated aspect ratio m to the processing control unit 201.

[0046] The processing control unit 201 determines whether the difference between the aspect ratio obtained in step S21 and the aspect ratio m supplied in step S27 is greater than or equal to a threshold value, i.e., whether the difference between the aspect ratio of the image to be projected and the aspect ratio m of the screen frame 32 calculated by calculation is greater than or equal to a threshold value (step S28).

[0047] The difference is greater than or equal to the threshold, i.e., step S28 If the result of the determination in step S21 is "Yes," the process control section 201 determines to maintain the aspect ratio of the image to be projected at the aspect ratio acquired in step S21 (step S29). If the difference is less than the threshold value, that is, if the determination result in step S29 is "No", the process control section 201 changes the aspect ratio of the image to be projected to the aspect ratio of the screen frame 32 (step S30).

[0048] The processing control unit 201 instructs the projection control unit 206 to project the image signal supplied from the host device at the maintained aspect ratio or the changed aspect ratio (step S31). Projector 250 The projector projects an image based on the image signal at the aspect ratio. If the difference between the aspect ratio of the image represented by the image signal and the calculated aspect ratio m is equal to or greater than a threshold, the aspect ratio of the image represented by the image signal is prioritized and the image is projected. On the other hand, if the difference is less than the threshold, the aspect ratio of the image represented by the image signal is changed to match the aspect ratio m of the screen frame 32, thereby enabling projection that makes effective use of the screen frame 32, for example. In this case, the aspect ratio is changed, but the change is small, so the user does not feel uncomfortable.

[0049] In step S25, if the process control unit 201 receives a notification from the coordinate specifying unit 204 that the number of specified vanishing points is other than "2," that is, if the determination result in step S25 is "No," the coordinate values ​​of the vanishing points are too large, or the quadrilateral is a trapezoid, etc. is In this case, the process control unit 201 uses a predetermined focal length as the focal length of the image capturing device 140 in the information terminal device 10 (step S32). In the projector 20, after step S31 or S32, the calculation process of the focal length f and the aspect ratio m ends, but projection based on the image signal continues.

[0050] Because various devices such as smartphones and tablet devices are used as the information terminal device 10, it is difficult for a general user to know the focal length f of the imaging device 140 built into the information terminal device 10. However, according to the imaging system 1 of this embodiment, the screen frame 32 is photographed by the imaging device 140 of the information terminal device 10, and the focal length f of the imaging device 140 can be calculated by analyzing the photographed image, so there is no need to photograph a checkerboard, which saves a lot of effort.

[0051] <Applications / Modifications> The above-described exemplary embodiment may be modified in various ways. Specific modifications that may be applied to the exemplary embodiment are exemplified below.

[0052] In the embodiment, the focal length f of the image capturing device 140 in the information terminal device 10 and the aspect ratio of the projection area surrounded by the screen frame 32 may be used in the projector 20 when adjusting keystone correction and the enlargement / reduction ratio.

[0053] In the embodiment, the projector 20 has the function of analyzing the quadrilateral Sf and the function of calculating the focal length f from the coordinate values, but the information terminal device 10 may have the function of analyzing the quadrilateral Sf and the function of calculating the focal length f from the coordinate values ​​in addition to the photographing function. Specifically, the image acquisition unit 202, the four-side identification unit 203, the coordinate identification unit 204, and the calculation unit 205 of the projector 20 may be provided in the information terminal device 10.

[0054] The image acquisition unit 202, the four-side identification unit 203, the coordinate identification unit 204, and the calculation unit 205 may be configured to be provided on a server on the cloud. For example, the image data Dt captured by the information terminal device 10 may be transmitted to the server via a communication network, and the server may transmit the aspect ratio calculated by executing the processes of steps S22 to S27 to the projector 20.

[0055] <Additional Notes> From the above description, for example, preferred embodiments of the present disclosure can be understood as follows: Note that, in order to facilitate understanding of each embodiment, reference numerals in the drawings are written in parentheses for convenience, but this is not intended to limit the present invention to the embodiments shown in the drawings.

[0056] <Appendix 1> A focal length calculation method according to one aspect 1 includes acquiring image data (Dt) of a rectangular subject (32), identifying a first side (L1) of a quadrangle (Sf) corresponding to the subject (32) in an image represented by the image data (Dt), a second side (L2) opposite the first side (L1), a third side (L3), and a fourth side (L4) opposite the third side (L3), and determining a distance between the first side ( L1and determining the coordinate values ​​of a first point (Vvnp) at the intersection of an extension line of the third side (L3) and an extension line of the second side (L2) and the coordinate values ​​of a second point (Hvnp) at the intersection of an extension line of the third side (L3) and an extension line of the fourth side (L4), and calculating the focal length (f) of the device (10) that captured the subject (32) based on the coordinate values ​​of the first point (Vvnp) and the coordinate values ​​of the second point (Hvnp). According to the first aspect, there is no need to capture an image of the checkerboard in order to determine the focal length (f) of the device (10) that captured the subject (32), and this does not require much effort.

[0057] <Appendix 2> In a focal length calculation method according to a specific aspect 2 of aspect 1, calculating the focal length (f) includes determining the focal length (f) such that the dot product of the first coordinate value (Vx / f, Vy / f, 1) obtained by converting the coordinate value of the first point (Vvnp) into the normalized camera coordinate system and the second coordinate value (Hx / f, Hy / f, 1) obtained by converting the coordinate value of the second point (Hvnp) into the normalized camera coordinate system is zero. According to aspect 2, a specific calculation of the focal length (f) is realized.

[0058] <Appendix 3> In a focal length calculation method according to another specific aspect 3 of aspect 1, the coordinate values ​​of the first point (Vvnp) are (Vx, Vy), the coordinate values ​​of the second point (Hvnp) are (Hx, Hy), and the focal length f If so, the focal length f of f={-(HxVx+HyVy)} 1 / 2 According to the third aspect, the focal length f The specific calculation of is realized.

[0059] <Appendix 4> A display method of the projector according to the fourth aspect includes acquiring image data (Dt) obtained by capturing a rectangular projection area (32), identifying a first side (L1), a second side (L2) opposite the first side (L1), a third side (L3), and a fourth side (L4) opposite the third side (L3) of a quadrangle (Sf) corresponding to the projection area (32) in an image represented by the image data (Dt), acquiring coordinate values ​​(Vx, Vy) of a first point (Vvnp) at an intersection between an extension line of the first side (L1) and an extension line of the second side (L2), and acquiring coordinate values ​​(Hx, Hy) of a second point (Hvnp) at an intersection between an extension line of the third side (L3) and an extension line of the fourth side (L4), and , Vy) and the coordinate values ​​(Hx, Hy) of the second point (Hvnp); determining the coordinate values ​​(Dx, Dy) of a third point (Dvnp), which is the intersection of a line (Lvh) connecting the first point (Vvnp) and the second point (Hvnp) with an extension (Ld) of the diagonal of the quadrangle (Sf); calculating the aspect ratio (m) of the projection area (32) based on the coordinate values ​​(Vx, Vy) of the first point (Vvnp), the coordinate values ​​(Hx, Hy) of the second point (Hvnp), and the coordinate values ​​(Dx, Dy) of the third point (Dvnp); and projecting an image based on the calculated aspect ratio (m) of the projection area (32). According to the fourth aspect, there is no need to photograph a checkerboard in order to determine the focal length (f) of the device (10) that photographed the projection area (32), which does not require much effort. Also, appropriate projection becomes possible based on the aspect ratio (m) of the projection area (32) that is calculated.

[0060] <Appendix 5> A display method of a projector according to a fifth specific example of the fourth example projects an image at the aspect ratio of the image indicated by the image signal when the difference between the aspect ratio of the image indicated by the image signal and the aspect ratio (m) of the projection area (32) is equal to or greater than a threshold. According to the fifth example, in this case, the aspect ratio of the image indicated by the image signal is given priority for projection.

[0061] <Appendix 6> A display method of a projector according to a sixth specific aspect of the fourth aspect is to project an image in which the aspect ratio of the image represented by the image signal is changed to the aspect ratio (m) of the projection area (32) when the difference between the aspect ratio of the image represented by the image signal and the aspect ratio (m) of the projection area (32) is less than a threshold value. According to the sixth aspect, in this case, the image is displayed over the entire projection area (32), and even if the aspect ratio of the image represented by the image signal is changed, the change is small, so that the user does not feel uncomfortable.

[0062] <Appendix 7> The photographing system (1) according to the seventh aspect includes a processor (200), and the processor (200) acquires image data (Dt) of a rectangular subject (32), identifies a first side (L1) of a quadrangle (Sf) corresponding to the subject (32) in an image represented by the image data (Dt), a second side (L2) opposite the first side (L1), a third side (L3), and a fourth side (L4) opposite the third side (L3), and determines the first side (L1) and the fourth side (L4) of the quadrangle (Sf) corresponding to the subject (32). The coordinate values ​​(Vx, Vy) of a first point (Vvnp) which is the intersection of an extension line of the third side (L1) and an extension line of the second side (L2) and the coordinate values ​​(Hx, Hy) of a second point (Hvnp) which is the intersection of an extension line of the third side (L3) and an extension line of the fourth side (L4) are acquired, and the focal length (f) of the device (10) which photographed the subject (32) is calculated based on the coordinate values ​​(Vx, Vy) of the first point (L1) and the coordinate values ​​(Hx, Hy) of the second point (L2). 7 According to this method, there is no need to photograph a checkerboard in order to determine the focal length (f) of the device (10) that photographed the subject, and it does not require much effort. [Explanation of symbols]

[0063] 1...imaging system, 10...information terminal device, 20...projector, 30...screen, 32...screen frame, 202...image acquisition unit, 203...four-side determination unit, 204...coordinate determination unit, 205...calculation unit.

Claims

1. Acquiring image data obtained by capturing an image of a rectangular screen frame that surrounds a projection area onto which an image is projected from a projector; Identifying a first side, a second side opposite to the first side, a third side, and a fourth side opposite to the third side of a quadrangle corresponding to the screen frame in the image represented by the image data; Identifying coordinate values ​​of a first point that is an intersection between an extension line of the first side and an extension line of the second side, and coordinate values ​​of a second point that is an intersection between an extension line of the third side and an extension line of the fourth side; calculating a focal length of a device that photographed the screen frame based on the coordinate values ​​of the first point and the coordinate values ​​of the second point; A focal length calculation method including:

2. Calculating the focal length includes: a first coordinate value obtained by transforming the coordinate value of the first point into a normalized camera coordinate system; and calculating the focal length at which an inner product of the coordinate value of the second point and a second coordinate value obtained by transforming the coordinate value of the second point into the normalized camera coordinate system becomes zero. The focal length calculation method according to claim 1 .

3. When the coordinate values ​​of the first point are (Vx, Vy), the coordinate values ​​of the second point are (Hx, Hy), and the focal length is f, The focal length f is calculated using the following formula: The focal length calculation method according to claim 1 . f={-(HxVx+HyVy)} １／２

4. Identifying the first side, the second side, the third side, and the fourth side includes: specifying at least one regression curve or approximation line corresponding to the first side, the second side, the third side, and the fourth side based on the image data; The focal length calculation method according to claim 1 .

5. Acquiring image data of a rectangular screen frame that surrounds a projection area onto which an image is projected from a projector; Identifying a first side, a second side opposite to the first side, a third side, and a fourth side opposite to the third side of a quadrangle corresponding to the screen frame in the image represented by the image data; Identifying coordinate values ​​of a first point that is an intersection between an extension line of the first side and an extension line of the second side, and coordinate values ​​of a second point that is an intersection between an extension line of the third side and an extension line of the fourth side; calculating a focal length of a device that photographed the screen frame based on the coordinate values ​​of the first point and the coordinate values ​​of the second point; Identifying the coordinate values ​​of a third point, which is an intersection point between a straight line connecting the first point and the second point and an extension of a diagonal line of the quadrangle; calculating an aspect ratio of the screen frame based on the coordinate values ​​of the first point, the coordinate values ​​of the second point, and the coordinate values ​​of the third point; projecting an image based on the calculated aspect ratio of the screen frame; Projector display methods, including:

6. When the difference between the aspect ratio of the image represented by the image signal and the aspect ratio of the screen frame is equal to or greater than a threshold value, Projecting the image at the aspect ratio indicated by the image signal. The display method of the projector according to claim 5 .

7. When the difference between the aspect ratio of the image represented by the image signal and the aspect ratio of the screen frame is less than a threshold value, The aspect ratio of the image represented by the image signal is changed to the aspect ratio of the screen frame, and the image is projected. The display method of the projector according to claim 5 .

8. determining whether there are two vanishing points; If the determined number of vanishing points is not two, projecting the image with a preset aspect ratio; further comprising The first point and the second point are the vanishing points. The display method of the projector according to claim 5 .

9. Equipped with a processor, The processor: Acquiring image data obtained by capturing an image of a rectangular screen frame that surrounds a projection area onto which an image is projected from a projector; Identifying a first side, a second side opposite to the first side, a third side, and a fourth side opposite to the third side of a quadrangle corresponding to the screen frame in the image represented by the image data; Identifying coordinate values ​​of a first point that is an intersection between an extension line of the first side and an extension line of the second side, and coordinate values ​​of a second point that is an intersection between an extension line of the third side and an extension line of the fourth side; calculating a focal length of a device that photographed the screen frame based on the coordinate values ​​of the first point and the coordinate values ​​of the second point; To carry out A photography system characterized by:

Images