Image clipping apparatus and program

Abstract

To directly generate a cut-out image after projective transformation with correct perspective when cutting out a predetermined area from an input image.SOLUTION: A projection transformation matrix calculation means 10 of an image clipping device 1 obtains a projection transformation matrix M based on a horizontal angle of view θ and an elevation angle δ0 of a real camera, and a pan angle α, a tilt angle δ and a zoom magnification z of a clipping virtual camera. The scanning unit 12 scans the image coordinates on the cut-out image J as the target image coordinates (xtgt, ytgt). The transformation means 13 transforms the target image coordinates (xtgt, ytgt) into the reference image coordinates (xref, yref) using the projective transformation matrix M. The storage means reads the pixel value I (xref, yref) of the input image I from the frame memory 11, stores the pixel value I (xref, yref) in the frame memory 14 as the pixel value J (xtgt, ytgt) of the cut image J, reads all the pixel values J (xtgt, ytgt) from the frame memory 14 as the cut image J, and outputs the cut image J.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of image transformation, and in particular to an apparatus and program for extracting an image with correct perspective. [Background technology]

[0002] Conventionally, techniques for generating a low-resolution image by cutting out a rectangular (including square) partial image from a high-resolution image are known, such as video editing devices and digital processing programs for photographs and images.

[0003] When cutting out a partial image from this high-resolution image, a process may be performed in which any quadrangular area, such as a parallelogram, trapezoid, or trapezoid, contained in the high-resolution image is mapped to a rectangular area. For example, a technology has been disclosed in which an image is cut out from an overhead image captured by an actual camera by operating a virtual camera (see, for example, Patent Document 1).

[0004] The technique in Patent Document 1 generates an aerial viewpoint image (an image obtained by facing the ground from directly above) by projectively transforming a frame of an overhead view image, and then cuts out an image from the aerial viewpoint image as a camera image (cut-out image) that corresponds to the gaze point and field of view determined based on the pan angle, tilt angle, and zoom amount of the virtual camera. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2023-70220 Summary of the Invention [Problem to be solved by the invention]

[0006] When generating a partial image by cutting out a rectangular area from an image captured using perspective projection, unless the center of the cutout area coincides with the optical axis of the captured image, the partial image will be an image that has undergone lens shifting, which will result in an unnatural shape and perspective of the subject in the partial image.

[0007] Here, if an arbitrary quadrangular area is mapped onto a rectangular area for cropping, it is possible to generate a partial image similar to that obtained by perspective projection in an optical system in which the optical axis passes through the center of the cropped area. However, no means for appropriately setting the quadrangular area for cropping has been disclosed.

[0008] Furthermore, the method of Patent Document 1 mentioned above converts the overhead view video frame into an aerial viewpoint image, and then cuts out the camera image from the aerial viewpoint image. Patent Document 1 also describes that when cutting out the camera image from the overhead view video frame, the projective transformation and cutting out process are performed in a single calculation, without generating an aerial viewpoint image (see paragraph 41 of Patent Document 1).

[0009] However, the specific processing contents and calculation formulas for the process of directly extracting a camera image from a frame of an overhead video are not disclosed.

[0010] Therefore, the present invention has been made to solve the above-mentioned problems, and its object is to provide an image cutting-out device and program that can directly generate a cut-out image after projective transformation with correct perspective when cutting out a specified area from an input image. [Means for solving the problem]

[0011] In order to solve the above problem, the image cropping device of claim 1 is an image cropping device that crops out a predetermined area from an input image and generates a cropped image after projective transformation, characterized in that it comprises: a projective transformation matrix calculation means that calculates a projective transformation matrix based on information on the attitude of a real camera that captured the input image, information on the attitude of the cropping virtual camera when the cropped image is captured by the cropping virtual camera, and a zoom magnification of the cropping virtual camera; and cropping means that uses the projective transformation matrix calculated by the projective transformation matrix calculation means to determine pixel positions of the input image that correspond to pixel positions of the cropped image, and generates the cropped image from the input image by using pixel values ​​of the pixel positions of the input image as pixel values ​​of the pixel positions of the cropped image.

[0012] According to the image cropping device of claim 1, a cropped image with an accurate sense of perspective can be generated in accordance with the virtual orientation of the cropping virtual camera and zoom operation, while taking into consideration the orientation of the real camera.

[0013] Furthermore, the image cutting-out device of claim 2 is characterized in that, in the image cutting-out device of claim 1, it further comprises a calculation means for calculating information regarding the attitude of the cutting-out virtual camera from the pixel position of a representative point within the specified area cut out from the input image.

[0014] According to the image cropping device of claim 2, a cropped image with an accurate sense of perspective can be generated in accordance with the pixel position of the representative point specified on the input image and the virtual zoom operation of the cropping virtual camera, while taking into account the attitude of the real camera.

[0015] Furthermore, the image cropping device of claim 3 is characterized in that, in the image cropping device of claim 1 or 2, the information regarding the attitude of the real camera is the angle of view of the real camera and the angle of elevation of the real camera, and the information regarding the attitude of the cropping virtual camera is the pan angle and tilt angle of the cropping virtual camera.

[0016] Furthermore, a program according to a fourth aspect of the present invention is characterized in that it causes a computer to function as the image cutting device according to the first or second aspect of the present invention. [Effects of the Invention]

[0017] As described above, according to the present invention, when a predetermined area is cut out from an input image, it is possible to directly generate a cut-out image after projective transformation with correct perspective. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a block diagram showing an example of the configuration of an image cropping device according to a first embodiment; [Figure 2] 4 is a flowchart showing an example of processing by the image cropping device according to the first embodiment. [Figure 3] 1(1) is a diagram for explaining the pixel values, resolution, etc. of an input image I. FIG. 1(2) is a diagram for explaining the pixel values, resolution, etc. of a cut-out image J. [Figure 4] 1A is a diagram illustrating the elevation angle δ0 of the real camera, and FIG. 1B is a diagram illustrating the pan angle α and tilt angle δ of the cutout virtual camera. [Figure 5] 10(1) is a diagram illustrating an example of processing by the conversion means 13. FIG. 10(2) is a diagram illustrating an example of processing by the storage means. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of an image extraction device according to a second embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of image coordinates (gx, gy) of a representative point in the image extraction device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. [Image Extraction Device / Example 1] First, a description will be given of an image clipping device according to the first embodiment. Fig. 1 is a block diagram showing an example of the configuration of the image clipping device according to the first embodiment, and Fig. 2 is a flowchart showing an example of processing by the image clipping device according to the first embodiment.

[0020] This image cutting-out device 1 comprises a projective transformation matrix calculation means 10 and a cutting-out means 20. The cutting-out means 20 comprises frame memories 11 and 14, a scanning means 12, a conversion means 13, and a storage means (not shown).

[0021] The image cropping device 1 inputs an input image I, camera parameters (information regarding the angle of view (e.g., horizontal angle of view θ) and elevation angle δ0), pan angle α, tilt angle δ, and zoom magnification z, calculates a projection transformation matrix M, crops out a predetermined area (part or all of the image) of the input image I as a cropping area based on the projection transformation matrix M, performs a projection transformation of the cropping area to generate a cropped image J, and outputs it.

[0022] The camera parameters are information about the attitude of a camera (hereinafter referred to as a "real camera") when an input image I is captured by the real camera. The pan angle α and tilt angle δ are information about the attitude of a camera (hereinafter referred to as a "cropping virtual camera") when it is assumed that a cropped image J is captured by the cropping virtual camera. The zoom magnification z is the zoom magnification of the cropping virtual camera, and is the value obtained by dividing the width of the entire input image I by the width of the cropped area cropped from the input image I when α=δ=0. The same applies to the image cropping device 2 of the second embodiment described later.

[0023] In other words, the image cropping device 1 generates a rectangular image corresponding to the zoom magnification z of the cropping virtual camera specified in advance by the user as a cropped image J, based on a predetermined area in the input image I indicated by the pan angle α and tilt angle δ of the cropping virtual camera specified in advance by the user.

[0024] FIG. 3(1) is a diagram for explaining the pixel values ​​and resolution of the input image I. The resolution of the input image I is H(in) Pixels, vertically V (in) The image coordinates (pixel position) of the input image I are defined as x-axis in the horizontal right direction and y-axis in the vertical down direction, and the origin of the image coordinates may be, for example, the top left vertex of the input image I or the center point of the input image I. In the following, unless otherwise specified, the top left vertex of the input image I is defined as the origin.

[0025] FIG. 3(2) is a diagram for explaining the pixel values ​​and resolution of the cut-out image J. The resolution of the cut-out image J is H (out) Pixels, vertically V (out) The image coordinates (pixel position) of the cut-out image J are defined as x-axis in the horizontal right direction and y-axis in the vertical down direction, similar to the input image I, and the origin of the image coordinates may be, for example, the top left vertex of the cut-out image J, similar to the input image I, or may be the center point of the cut-out image J. In the following, unless otherwise specified, the top left vertex of the cut-out image J is defined as the origin.

[0026] For example, the image cropping device 1 may calculate a projection transformation matrix M such that the horizontal direction is H (in) = 7680 pixels and vertical direction is V (in) From the input image I with 4320 pixels, a rectangular area (cut-out area) corresponding to the cut-out image J is cut out, and by performing a projective transformation of the image of the cut-out rectangular area, the horizontal direction is H (out) = 1920 pixels and vertical direction is V (out) A rectangular cut-out image J of 1080 pixels is generated.

[0027] In other words, the image in the cutout area cut out from the input image I shown in Figure 3(1) is resampled, and the horizontal direction is H (out) Pixel and vertical direction is V (out) The image is generated as a rectangular cutout image J of pixels.

[0028] Figure 4(1) is a diagram explaining the elevation angle δ0 of the real camera, and at the top is shown a plan view of the installed real camera as seen from directly above, and directly below the plan view is shown an elevation view of the real camera as seen from the side. Also, to the left of the elevation view is shown a view A as seen from the direction of arrow A in the plan view. In view A, assuming a vector of the optical axis of the real camera relative to the ground plane in the direction of the subject, the elevation angle of that vector is δ0. Note that when a depression angle occurs with respect to the horizontal plane, as in view A, the elevation angle δ0 takes a negative value.

[0029] FIG. 4(2) illustrates the pan angle α and tilt angle δ of the clipped virtual camera. The top of the figure shows a plan view of the installed clipped virtual camera as seen from directly above, and directly below the plan view shows an elevation view of the clipped virtual camera as seen from directly to the side. To the left of the elevation view is a perspective view B, seen from the direction of arrow B in the plan view. When the optical axis of the real camera and the optical axis of the clipped virtual camera are projected onto a horizontal plane in the plan view, and a vector of the optical axis of the clipped virtual camera pointing in the direction of the subject and a vector of the optical axis of the real camera pointing in the direction of the subject are assumed, the pan angle α is the clockwise rotation angle of the projected image of the former vector relative to the projected image of the latter vector, as seen from above. In the perspective view B, when the vector of the optical axis of the clipped virtual camera pointing in the direction of the subject relative to the ground plane is assumed, the tilt angle δ is the elevation angle of that vector. Note that when a depression angle occurs with respect to the horizontal plane, as in the perspective view B, the tilt angle δ is a negative value.

[0030] For example, when α=δ=0, and the zoom magnification z=1, the image cropping device 1 crops the entire input image I, and the horizontal direction (width) H (out) Pixel and vertical direction (height) V (out) The image is resampled to a rectangular crop of pixels, J.

[0031] Also, for example, when α=δ=0, and the zoom magnification z=4, the width H (in) / 4 pixels and height V (in) A region of 1 / 4 pixels is cut out, and this has a width of H (out)Pixels and height V (out) The image is resampled to a rectangular crop of pixels, J.

[0032] The information about the angle of view included in the camera parameters used by the image cropping device 1 when calculating the projective transformation matrix M may be the focal length f in pixel units, or any one of the horizontal angle of view θ, the vertical angle of view, or the diagonal angle of view. Hereinafter, the horizontal angle of view θ will be used as the information about the angle of view. The same applies to the image cropping device 2 of the second embodiment described later.

[0033] (Projection transformation matrix calculation means 10) Returning to Figures 1 and 2, the projective transformation matrix calculation means 10 inputs the camera parameters (horizontal angle of view θ and elevation angle δ0) of the real camera specified by the user (step S201), and also inputs the pan angle α, tilt angle δ, and zoom magnification z of the cut-out virtual camera specified by the user (step S202).

[0034] The projective transformation matrix calculation means 10 calculates a projective transformation matrix M for generating a cut-out image J from the input image I based on the camera parameters (horizontal angle of view θ and elevation angle δ0), pan angle α, tilt angle δ, zoom magnification z, etc. (step S203).

[0035] Each component of the projective transformation matrix M is expressed as follows:

number

[0036] In this case, the components of the projective transformation matrix M are as follows:

number

[0037] However, q 11 ,q 12 ,···,q 33 can be calculated using the following formula:

number

[0038] In the formula (2), (c x (in) ,c y (in) ) is the image coordinate of the point where the optical axis of the input image I (the optical axis of the real camera) intersects with the input image I, and is set in advance. x (out) ,c y (out) ) are the image coordinates of the point where the optical axis of the cut-out image J (the optical axis of the cut-out virtual camera) and the cut-out image J intersect, and are set in advance.

[0039] For example, if the intersection of the optical axis of input image I and input image I is at the center of input image I, then the image coordinates (c x (in) ,c y (in) ) is set by the following formula:

number

[0040] As a specific example, if the resolution of the input image I is H (in) = 7680 pixels, vertically V (in) = 4320 pixels, then c x (in) =3839.5,c y (in) = 2159.5. If the origin of the image coordinates of the input image I is at the center of the input image I, then c x (in) =c y (in) =0.

[0041] In addition, if the intersection point between the optical axis of the input image I and the input image I can be measured in advance, the image coordinates of the intersection point obtained by the measurement can be set to the preset image coordinates (c x (in) ,c y (in) ) may be used.

[0042] Also, for example, if the intersection of the optical axis of the cutout image J and the cutout image J is at the center of the cutout image J, the image coordinates (c x (out) ,c y (out) ) is set by the following formula:

number

[0043] As a specific example, if the resolution of the cutout image J is H (out) = 1920 pixels, vertically V (out) If c = 1080 pixels, x (out) =959.5,c y (out) = 539.5. If the origin of the image coordinates of the cut-out image J is at the center of the cut-out image J, then c x (out) =c y (out) =0.

[0044] In addition, any image coordinate in the cutout image J can be set to a preset image coordinate (c x (out) ,c y (out) ) may be used as the preset image coordinates (c x (out) ,c y (out) ) is used, the image coordinates arbitrarily shifted from the result of the above equation (5) which indicates the image coordinates of the center of the cropped image J, the image cropping device 1 can obtain a cropped image J as if the lens of the cropping virtual camera had been shifted.

[0045] In addition, in the above formula (3), r represents the ratio of the resolution of the input image I to the cut-out image J. That is, it is expressed by the following formula.

number

[0046] For example, the image cropping device 1 may be configured to crop H (in) = 7680 pixels and vertically V (in) From an input image I with a resolution of 4320 pixels, (out) = 1920 pixels, vertically V (out) When generating a cut-out image J having a resolution of 1080 pixels, the resolution ratio r is 4.

[0047] Furthermore, in the above equation (3), ψ is the number of pixels H in the horizontal direction of the input image I. (in) and a value defined by the following equation using the horizontal angle of view θ included in the camera parameters.

number

[0048] (Frame Memory 11) The frame memory 11 is a memory for storing an input image I, and is randomly accessed to store reference image coordinates (x ref ,y ref ) pixel value I(x ref ,y ref ) is read out. A storage means (not shown in FIG. 1) provided in the extraction means 20 inputs the input image I, and stores the pixel value I(x, y) of the image coordinates (x, y) in the frame memory 11 (step S204).

[0049] (scanning means 12) The scanning means 12 converts the image coordinates on the cut-out image J into target image coordinates (x tgt ,y tgt ), and scans all pixels of the cut-out image J stored in the frame memory 14 in a predetermined order (step S205).

[0050] This scanning is performed at the target image coordinates (x tgt ,ytgt ), the rectangular image corresponding to the cut-out image J to be cut out is raster scanned, for example, with the main scan moving from left to right (increasing x-coordinate) and the sub-scan moving from top to bottom (increasing y-coordinate).

[0051] (Conversion method 13) The transformation means 13 receives the projection transformation matrix M from the projection transformation matrix calculation means 10 and also receives the target image coordinates (x tgt ,y tgt ) and use the projection transformation matrix M to calculate the target image coordinates (x tgt ,y tgt ) reference image coordinates (x ref ,y ref ) (step S206). As a result, the target image coordinates (x tgt ,y tgt ) is the reference image coordinate (x ref ,y ref ) is calculated.

[0052] Specifically, the transformation means 13 calculates the projection transformation matrix M and the target image coordinates (x tgt ,y tgt ) based on the reference image coordinates (x ref ,y ref ) is found.

number

[0053] FIG. 5(1) is a diagram for explaining an example of processing by the conversion means 13. As shown in FIG. 5(1), the target image coordinates (x tgt ,y tgt ) is the reference image coordinate (x ref ,y ref The coordinate transformation process using this projection transformation matrix M is performed in the order of the arrows, for example, in accordance with raster scanning by the scanning means 12.

[0054] (frame memory 14) The frame memory 14 is a memory for storing the cut-out image J, and is randomly accessed to store the target image coordinates (x tgt ,y tgt ), the pixel value I(x ref ,y ref ) is stored.

[0055] (memory means) The storage means (not shown in FIG. 1) provided in the extraction means 20 receives the reference image coordinates (x ref ,y ref ) of the input image I, the pixel value I(x ref ,y ref ) is read out (step S207).

[0056] The storage means stores the pixel value I(x ref ,y ref ) into the target image coordinates (x tgt ,y tgt ) pixel value J(x tgt ,y tgt ) and stored in the frame memory 14 (step S208), and the process proceeds to step S209.

[0057] FIG. 5(2) is a diagram for explaining an example of processing by the storage means. As shown in FIG. 5(2), the pixel values ​​I(x) of the input image I are stored in the frame memory 11. ref ,y ref ) is read out, and the pixel value J(x tgt ,y tgt ) are stored in the frame memory 14. That is, the reference image coordinates (x ref ,y ref ) of the input image I, the pixel value I(x ref ,y ref ) is the target image coordinate (x tgt ,y tgt ) pixel value J(x tgt ,y tgt), and a cut-out image J is generated from the input image I.

[0058] The storage means proceeds from step S208 and determines whether or not scanning of all pixels in the frame memory 14 has been completed by the scanning means 12 (step S209). If the storage means determines in step S209 that scanning of all pixels has not been completed (step S209: N), it proceeds to step S205. As a result, scanning of the next pixel is performed in step S205, and the processes of steps S206 to S208 are performed.

[0059] On the other hand, if it is determined in step S209 that scanning of all pixels has been completed (step S209: Y), the storage means stores all pixel values ​​J(x tgt ,y tgt ) is read out as a cut-out image J and output (step S210).

[0060] As described above, according to the image cropping device 1 of the first embodiment, the projective transformation matrix calculation means 10 calculates the projective transformation matrix M using the above formulas (1) to (3) based on the camera parameters (horizontal angle of view θ and elevation angle δ0) of the real camera specified by the user, as well as the pan angle α, tilt angle δ, and zoom magnification z of the cropping virtual camera.

[0061] The scanning means 12 converts the image coordinates on the cut-out image J into target image coordinates (x tgt ,y tgt ) and scans the image using the projection transformation matrix M. tgt ,y tgt ) reference image coordinates (x ref ,y ref )

[0062] The storage means stores pixel values ​​I(x ref ,y ref ) is stored in the frame memory 11. Then, the storage means stores the reference image coordinates (x ref ,y ref) of the input image I, the pixel value I(x ref ,y ref ) and convert it to the target image coordinates (x tgt ,y tgt ) pixel value J(x tgt ,y tgt ) and stored in the frame memory 14.

[0063] When the scanning means 12 has completed scanning all the pixels in the frame memory 14, the storage means stores all the pixel values ​​J(x tgt ,y tgt ) is read as cut-out image J and output.

[0064] This allows for the generation of a cut-out image J with an accurate sense of perspective in accordance with the virtual pan, tilt and zoom operations of the cut-out virtual camera, while taking into consideration the attitude of the real camera that captured the input image I.

[0065] Therefore, when cutting out a specified area from the input image I, there is no need to generate an intermediate image such as the aerial viewpoint image described in the aforementioned Patent Document 1, and a cut-out image J can be directly generated from the input image I after projective transformation with correct perspective.

[0066] Furthermore, the pan angle α and tilt angle δ of the cutout virtual camera input to the projective transformation matrix calculation means 10 of the image cutout device 1 are acquired as angles according to the operation of the tripod, for example, when a cameraman, who is a user, uses a tripod to change the attitude of the cutout virtual camera. Therefore, the pan angle α and tilt angle δ of the cutout virtual camera can be easily acquired by operating the tripod by the cameraman.

[0067] [Image Extraction Device / Example 2] Next, an image extraction device according to a second embodiment will be described below with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the configuration of the image extraction device according to the second embodiment.

[0068] This image cutout device 2 includes an angle calculation means 30 in addition to the configuration of the image cutout device 1 of the first embodiment shown in Fig. 1. Specifically, the image cutout device 2 includes a projective transformation matrix calculation means 10, cutout means 20, and angle calculation means 30. The cutout means 20 includes frame memories 11 and 14, a scanning means 12, a conversion means 13, and a storage means (not shown).

[0069] The image cropping device 1 of the first embodiment shown in FIG. 1 inputs the pan angle α and tilt angle δ of the cropping virtual camera. In contrast, the image cropping device 2 uses the coordinates specified on the input image I as the image coordinates (pixel position) of the representative point (g x ,g y ) and convert it into the pan angle α and tilt angle δ of the extracted virtual camera.

[0070] FIG. 7 shows the image coordinates (g x ,g y 7 is a diagram illustrating an example of the image coordinates (g x ,g y ) are the image coordinates corresponding to the center point of the cut-out image J that is generated by cutting out the input image I. The image coordinates of this representative point (g x ,g y ) is specified by the user.

[0071] That is, the image cropping device 2 calculates the image coordinates (g x ,g y ) as a reference, a rectangular image corresponding to the zoom magnification z of the cut-out virtual camera designated in advance by the user is generated as a cut-out image J.

[0072] The angle calculation means 30 calculates the image coordinates (g x ,g y) and the camera parameters (horizontal angle of view θ and elevation angle δ0). Then, the angle calculation means 30 calculates the number of pixels H in the horizontal direction of the input image I. (in) Based on the horizontal angle of view θ included in the camera parameters, the defined value ψ is calculated using the above equation (7).

[0073] The image coordinates (g x ,g y ) indicates the coordinate values ​​at which the cutout is performed to correspond to the image coordinates of the center of the cutout image J when the cutout image J is generated from the input image I.

[0074] The angle calculation means 30 calculates the elevation angle δ0 included in the input camera parameters, the preset image coordinates (c x (in) ,c y (in) ) (see equation (4) above) and the calculated definition value ψ, the image coordinates (g x ,g y ) and convert it into the pan angle α and tilt angle δ of the virtual camera.

number

[0075] In the above equation (9), the function atan2(Y,X) is a function for finding the argument of the point (X,Y) on the XY plane, and is defined by the following equation.

number

[0076] The angle calculation means 30 outputs the pan angle α and tilt angle δ calculated by the above equation (9) to the projective transformation matrix calculation means 10.

[0077] The projective transformation matrix calculation means 10 and the cutout means 20 are similar to the projective transformation matrix calculation means 10 and the cutout means 20 of the image cutout device 1 shown in FIG. 1, and therefore a description thereof will be omitted here.

[0078] As described above, according to the image cropping device 2 of the second embodiment, the angle calculation means 30 calculates the image coordinates (g x ,g y ) and convert it into the pan angle α and tilt angle δ of the virtual camera.

[0079] Then, similarly to the image cutout device 1 of the first embodiment, the projective transformation matrix calculation means 10 calculates the projective transformation matrix M, and the cutout means 20 generates a cutout image J from the input image I using the projective transformation matrix M.

[0080] This allows us to calculate the image coordinates (g x ,g y ) and a virtual zoom operation of the virtual camera for cropping, a cropped image J having an accurate sense of perspective can be generated.

[0081] Therefore, when cutting out a specified area from the input image I, there is no need to generate an intermediate image such as the aerial viewpoint image described in the aforementioned Patent Document 1, and a cut-out image J can be directly generated from the input image I after projective transformation with correct perspective.

[0082] Although the present invention has been described above with reference to the first and second embodiments, the present invention is not limited to the first and second embodiments and can be modified in various ways without departing from the technical concept thereof.

[0083] For example, in the first and second embodiments, the projective transformation matrix calculation means 10 uses the horizontal angle of view θ and the elevation angle δ0 as information about the attitude of the real camera, and the pan angle α and the tilt angle δ as information about the attitude of the cut-out virtual camera, when calculating the projective transformation matrix M. In the present invention, the information about the attitude of the real camera is not limited to the horizontal angle of view θ and the elevation angle δ0, and other information may be used. Furthermore, in the present invention, the information about the attitude of the cut-out virtual camera is not limited to the pan angle α and the tilt angle δ, and other information may be used.

[0084] In the second embodiment, for example, in the example of FIG. 7, the image coordinates (g x ,g y ) are the image coordinates corresponding to the center point of the cut-out image J. In the present invention, the image coordinates of the representative point (g x ,g y ) is not limited to the image coordinates of the center point of the cut-out image J, but may be any image coordinates within the cut-out image J.

[0085] A normal computer can be used as the hardware configuration of the image cutout device 1 according to the first embodiment of the present invention. The image cutout device 1 is configured by a computer equipped with a CPU, a volatile storage medium such as RAM, a non-volatile storage medium such as ROM, an interface, etc. The same applies to the image cutout device 2 according to the second embodiment of the present invention.

[0086] The functions of the projective transformation matrix calculation means 10 and the cutting means 20 (frame memories 11, 14, scanning means 12, conversion means 13, and storage means not shown in FIG. 1) provided in the image cutting device 1 are each realized by causing a CPU to execute a program describing these functions.

[0087] The functions of the projective transformation matrix calculation means 10, the cutout means 20 (frame memories 11, 14, scanning means 12, conversion means 13, and storage means not shown in FIG. 6), and the angle calculation means 30 provided in the image cutout device 2 are also realized by causing the CPU to execute a program describing these functions.

[0088] These programs are stored in the storage medium and are read and executed by the CPU. These programs can also be stored in a storage medium such as a magnetic disk (HDD, etc.), an optical disk (CD-ROM, DVD, etc.), or a semiconductor memory (SSD, etc.) and distributed, or can be transmitted and received via a network. [Explanation of symbols]

[0089] 1,2 Image extraction device 10 Projection transformation matrix calculation means 11,14 frame memory 12 Scanning means 13 Conversion methods 20 Cutting Method 30 Angle calculation means I Input image J Cutout image M Projection transformation matrix α Pan angle δ Tilt angle θ Horizontal angle of view δ0 elevation angle z Zoom ratio (x ref ,y ref ) Reference image coordinates (x tgt ,y tgt ) Target image coordinates (g x ,g y ) Image coordinates of the representative point

Claims

1. An image cropping device that crops a predetermined area from an input image and generates a cropped image after projective transformation, a projection transformation matrix calculation means for calculating a projection transformation matrix based on information about the attitude of a real camera that captured the input image, information about the attitude of the cut-out virtual camera when the cut-out image is captured by the cut-out virtual camera, and a zoom magnification of the cut-out virtual camera; a cutout means for determining pixel positions of the input image corresponding to pixel positions of the cutout image using the projective transformation matrix calculated by the projective transformation matrix calculation means, and generating the cutout image from the input image by setting pixel values ​​of the pixel positions of the input image as pixel values ​​of the pixel positions of the cutout image; An image extraction device comprising:

2. 2. The image extraction device according to claim 1, The image cutting device further comprises a calculation means for calculating information relating to the attitude of the cutting virtual camera from the pixel position of a representative point within the predetermined area cut out from the input image.

3. 3. The image extraction device according to claim 1, The information about the attitude of the real camera is the angle of view of the real camera and the angle of elevation of the real camera, 10. An image cropping device, wherein the information about the attitude of the cropping virtual camera is a pan angle and a tilt angle of the cropping virtual camera.

4. A program for causing a computer to function as the image extraction device according to claim 1 or 2.

Images