Imaging device, method for controlling imaging device, program, and storage medium

Abstract

To provide an imaging device for making it possible for a user to easily adjust a focus when capturing VR images.SOLUTION: Included are: imaging units (101, 102, 103) that acquire a captured image; a conversion processing unit (107) that performs prescribed conversion processing on at least one partial region of the captured image and generates a converted image; a peaking processing unit (105) that performs peaking processing for adjusting a focus of at least one of the captured image and the converted image and generates a peaking image; an image synthesis unit (106) that generates a synthesized image of the peaking image and at least one of the captured image and the converted image; and a display unit (110) that displays the synthesized image. When it is set so that the peaking processing is performed on the converted image and then the synthesized image is displayed, the display unit displays the synthesized image of the peaking image and the converted image. The partial region includes a periphery of the captured image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, a control method for an imaging apparatus, a program, and a storage medium. [Background technology]

[0002] Patent Literature 1 discloses an imaging device capable of capturing a celestial sphere image at once as an imaging device for acquiring VR content (captured image) such as a photo or video for VR (Virtual Reality). The VR content is visually recognized by a user using, for example, a non-transparent HMD (Head Mounted Display).

[0003] In recent years, imaging devices having a peaking (focus peaking) function have been known. The peaking function is a function that displays a composite image by extracting and amplifying a high-frequency component from a luminance signal included in an input image signal and synthesizing the original input image with a peaking image, thereby emphasizing the contour of a focused portion. By displaying the composite image as a live view on an EVF (electric view finder) or a liquid crystal monitor (rear monitor) of the imaging device, the user can visually know the focused portion, making it easier to adjust the focus. Patent Document 2 discloses an imaging device that switches between performing peaking processing on a captured image or performing peaking processing on a reduced image of the captured image according to the amount of noise. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6897268 [Patent Document 2] Patent Publication No. 2021-64837 Summary of the Invention [Problem to be solved by the invention]

[0005] The image acquired by VR shooting becomes a fisheye image (circular fisheye image). After peaking processing is performed on the fisheye image, if a live view is displayed on the EVF or rear monitor, the image display will be different from that on the HMD used for actual viewing by the user, and the focus state may differ from that intended by the user. In particular, the subject is significantly distorted in the peripheral area of ​​the circular fisheye image, and is therefore likely to be extracted as a high-frequency component. For this reason, in the imaging devices disclosed in Patent Documents 1 and 2, it is difficult for the user to determine whether the peripheral area of ​​the circular fisheye image is actually in focus.

[0006] Therefore, an object of the present invention is to provide an imaging device that allows a user to easily adjust the focus during VR shooting. [Means for solving the problem]

[0007] An imaging device as one aspect of the present invention has an imaging unit that acquires an image, a conversion processing unit that performs a predetermined conversion process on at least one partial region of the image to generate a converted image, a peaking processing unit that performs a peaking process for focus adjustment on at least one of the image or the converted image to generate a peaking image, an image synthesis unit that generates a composite image of at least one of the image or the converted image and the peaking image, and a display unit that displays the composite image, wherein when the display unit is set to perform the peaking process on the converted image and display the composite image, it displays the composite image of the converted image and the peaking image, and the partial region includes a peripheral portion of the image.

[0008] Other objects and features of the present invention will be described in the following embodiments. Effect of the Invention

[0009] According to the present invention, it is possible to provide an imaging device that allows a user to easily adjust the focus during VR shooting. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram of an imaging device according to a first embodiment. [Diagram 2] 5 is an explanatory diagram of a peaking process in each embodiment. FIG. [Diagram 3] 4A to 4C are explanatory diagrams of captured images and perspective projected images during VR shooting in each embodiment. [Figure 4] 5A and 5B are explanatory diagrams of the correspondence relationship between a captured image and a hemisphere in a three-dimensional virtual space in each embodiment. [Diagram 5] 1A to 1C are explanatory diagrams illustrating positions of a virtual camera in a three-dimensional virtual space and an area where perspective projection transformation is performed in a hemispherical image in each embodiment. [Figure 6] 5 is a flowchart showing a display process of the imaging device in the first embodiment. [Figure 7] FIG. 2 is a diagram showing the display content of the imaging device in the first embodiment. [Figure 8] 1A to 1C are diagrams illustrating images captured by the VR 180 in the first embodiment. [Figure 9] FIG. 2 is a diagram showing the display contents of a VR 180 of the imaging device in the first embodiment. [Figure 10] FIG. 11 is a block diagram of an imaging device according to second and third embodiments. [Figure 11] 10 is a flowchart showing a display process of an imaging apparatus according to a second embodiment. [Figure 12] FIG. 11 is a diagram showing the display content of an imaging device in a second embodiment. [Figure 13] FIG. 11 is a diagram showing the display contents of a VR 180 of an imaging device in the second embodiment. [Figure 14] 13 is a flowchart showing a display process of an imaging apparatus according to a third embodiment. [Figure 15] FIG. 13 is a diagram showing the display content of an imaging device in a third embodiment. [Figure 16] FIG. 13 is a diagram showing the display contents of a VR 180 of an imaging device in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0012] (First embodiment) First, an imaging device 100 according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram of the imaging device 100. The imaging device 100 has a lens unit 101, an image sensor unit 102, an imaging processing unit 103, a recording unit 104, a peaking processing unit 105, an image synthesis unit 106, a conversion processing unit 107, a user operation unit 108, a display control unit 109, and a display unit 110.

[0013] The lens unit 101 has an optical system (image pickup optical system) that forms an image of a subject (optical image) on an image pickup surface of the image pickup device unit 102, and has a zoom function, a focus adjustment function, and an aperture adjustment function. The image pickup device unit 102 has an image pickup device in which a large number of photoelectric conversion elements are arranged, receives the image of a subject formed by the lens unit 101, and converts it into an image signal in pixel units. The image pickup device is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charged Coupled Device) image sensor. The image pickup processing unit 103 performs image processing for recording and displaying the image signal (captured image data) output from the image pickup device unit 102 after correcting scratches and the like caused by the image pickup device unit 102. The recording unit 104 records the captured image data output from the image pickup processing unit 103 in a recording medium (not shown) such as an SD card. In this embodiment, the lens unit 101 and the image pickup device unit 102 constitute an image pickup unit. Alternatively, the image pickup unit may further include the image pickup processing unit 103.

[0014] The peaking processor 105 has an FIR (Finite Impulse Response) filter. The peaking processor 105 can adjust the intensity and frequency of the peaking signal by a gain adjustment signal and a frequency adjustment signal (not shown). Here, the focus assist function using the peaking process will be described in detail with reference to Figs. 2(a) to (c). Figs. 2(a) to (c) are explanatory diagrams of the peaking process. Note that the description here will be given using an image captured by a normal lens, not an image captured by a fisheye lens (fisheye image).

[0015] A luminance signal or an RGB development signal as shown in FIG. 2(a) is input to the peaking processing unit (edge ​​extraction unit) 105. FIG. 2(a) shows an image before the focus assist function is executed. The user activates the focus assist function by operating the user operation unit 108. As a result, edge information (peaking image) 301 of the original image 300 is extracted, emphasized, and output from the peaking processing unit 105 as shown in FIG. 2(b). An image (synthetic image) in which the edge information 301 is superimposed on the original image 300 is displayed on the display unit 110 as shown in FIG. 2(c). The area in which the edge information 301 is displayed indicates that the image is in focus, and the user can visually know the in-focus state.

[0016] The image synthesis unit 106 has a function of superimposing and outputting two images. In this embodiment, the output (captured image) of the imaging processing unit 103 or the output (converted image) of the conversion processing unit 107 is superimposed with the output (peaking image) of the peaking processing unit 105, and a synthetic image as shown in FIG. 2(c) is output.

[0017] When a user selects to display a perspective projection converted image (perspective projection image) via a user operation unit 108, a conversion processing unit (perspective projection conversion processing unit) 107 performs perspective projection conversion processing on the captured image data processed by the image capture processing unit 103. Note that, since the perspective projection conversion is performed by setting a viewing angle, the perspective projection image is generated by converting at least one partial region of the captured image.

[0018] Here, the method of generating a perspective projection image in this embodiment will be described in detail with reference to Fig. 3 and Fig. 4, taking a hemispherical image capture as an example. Fig. 3(a) to (c) are explanatory diagrams of captured images and perspective projection images during VR capture. Fig. 4(a) and (b) are explanatory diagrams of the correspondence between a captured image (circular fisheye image) and a hemisphere in a three-dimensional virtual space.

[0019] Fig. 3(a) shows a captured image captured when a fisheye lens is used in the imaging device 100. As shown in Fig. 3(a), the captured image data output from the imaging processing unit 103 is a circularly cut and distorted image (circular fisheye image). The conversion processing unit 107 first uses a three-dimensional computer graphics library such as Open GL ES (Open Graphics Library for Embedded Systems) to draw a hemisphere as shown in Fig. 4(a). Then, the circular fisheye image is pasted inside the hemisphere.

[0020] Specifically, as shown in FIG. 4(b), the circular fisheye image is associated with a coordinate system that is composed of a vertical angle θ with the zenith direction of the captured image as an axis, and a horizontal angle φ around the axis of the zenith direction. In this case, if the range of the viewing angle of the circular fisheye image is 180°, the vertical angle θ and the horizontal angle φ are in the range of −90° to 90°. The coordinate values ​​(θ, φ) of the circular fisheye image can be associated with each point on the sphere representing the hemispherical image, as shown in FIG. 4(a). As shown in FIG. 4(a), if the center of the hemisphere is 0 and the three-dimensional coordinates on the sphere are (X, Y, Z), the relationship between the circular fisheye image and the two-dimensional coordinates can be expressed by the following formulas (1) to (3). Here, r is the radius of the hemisphere. Based on the coordinate correspondence shown by these formulas, the circular fisheye image can be pasted inside the hemisphere to generate a hemispherical image in a three-dimensional virtual space.

[0021]

number

[0022]

number

[0023]

number

[0024] When generating a 360° omnidirectional image, circular fisheye images 180° in front of and 180° behind the user are acquired, and respective hemispherical images are generated by the above-mentioned means, and then the images are stitched together to generate a 360° omnidirectional image.

[0025] As described above, since a celestial sphere image and a hemispherical image are images pasted to cover a spherical surface, they are different from the image viewed by the user on the HMD as they are. For example, by performing perspective projection transformation on a part of the image (partial region) such as the region surrounded by the dotted line in Fig. 3(b), it is possible to display an image such as that shown in Fig. 3(c), which is equivalent to the image viewed by the user on the HMD.

[0026] 5 is an explanatory diagram of the positional relationship between a virtual camera in a three-dimensional virtual space in a hemispherical image and an area where perspective projection conversion is performed. The virtual camera corresponds to the position of the viewpoint of a user viewing a hemispherical image displayed as a three-dimensional solid hemisphere. The area where perspective projection conversion is performed is determined by the direction (θ, φ) and angle of view of the virtual camera, and an image of this area is displayed on the display unit 110. In FIG. 5, w indicates the horizontal resolution of the display unit 110, and h indicates the vertical resolution of the display unit 110.

[0027] The user operation unit 108 is an operation member such as a cross key or a touch panel, and is a user interface that allows the user to select and input various parameters of the imaging device 100 and a display method for a captured image. The parameters of the imaging device 100 include, for example, an ISO sensitivity setting value or a shutter speed setting value, but are not limited to these.

[0028] In this embodiment, the display method can be selected from the circular fisheye image (captured image) itself, or an image (converted image) obtained by applying perspective projection conversion processing to the circular fisheye image. In addition, in this embodiment, if the user sets the focus assist function to ON, a peaking process is performed on the captured image, converted image, etc., and a composite image in which detected edge information (peaking image) is superimposed can be displayed. In addition, in this embodiment, if the user selects the perspective projection conversion display, perspective projection conversion is performed on at least the end portion (peripheral portion of the fisheye image) of the circular fisheye image on the initial screen, and the user can select an area of ​​the circular fisheye image to be displayed by perspective projection using the user operation unit 108.

[0029] The display control unit 109 controls the conversion processing unit 107, the peaking processing unit 105, and the image synthesis unit 106 so that an image (at least one of a captured image, a converted image, or a synthesized image) set by the user operation unit 108 is displayed on the display unit 110. Here, a procedure for displaying an image by the display control unit 109 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the display process of the imaging device 100.

[0030] First, in step S601, the user selects ON / OFF of the focus assist function with the user operation unit 108. At this time, the display control unit 109 determines whether the focus assist function is OFF or not. If it is determined that the focus assist function is OFF, the process proceeds to step S602. In step S602, the display control unit 109 determines whether the perspective projection conversion display is selected by the user or not. If it is determined that the perspective projection conversion display is not selected, the process proceeds to step S603. In step S603, the display control unit 109 controls the conversion processing unit 107, the peaking processing unit 105, and the image synthesis unit 106 not to execute processing so that the captured circular fisheye image (captured image) is displayed as is (fisheye display).

[0031] On the other hand, if it is determined in step S602 that the perspective projection conversion display has been selected by the user, the process proceeds to step S604. In step S604, the display control unit 109 controls the conversion processing unit 107 to execute, but not to execute the peaking processing unit 105 and the image synthesis unit 106. Note that in the initial display, an image obtained by perspective projection conversion of the center part of the circular fisheye image is displayed (perspective projection display of the center part of the fisheye). Next, in step S605, it is determined whether or not the user has moved the perspective projection position by the user operation unit 108. If it is determined that the perspective projection position has moved, the display control unit 109 proceeds to step S606. In step S606, the display control unit 109 controls the conversion processing unit 107 to perform perspective projection conversion processing according to the moved position of the perspective projection position, and to display the perspective projection converted image. After the process of step S606, the process returns to step S605.

[0032] On the other hand, if it is determined in step S601 that the focus assist function is ON, the process proceeds to step S607. In step S607, the display control unit 109 determines whether or not the perspective projection conversion display has been selected by the user. If it is determined that the perspective projection conversion display has not been selected, the process proceeds to step S608. In step S608, the display control unit 109 controls not to execute the conversion processing unit 107, but to execute the peaking processing unit 105 and the image synthesis unit 106. At this time, in step S608, the peaking process is applied to the captured circular fisheye image (captured image), and a synthesis image on which the detected edge information (peaking image) is superimposed is displayed (fisheye display with peaking process applied).

[0033] On the other hand, if it is determined in step S607 that the user has selected the perspective projection conversion display, the process proceeds to step S609. In step S609, the display control unit 109 controls the conversion processing unit 107, the peaking processing unit 105, and the image synthesis unit 106 to execute. In this case, in step S609, in the initial display, a peaking process is applied to an image (converted image) obtained by perspective projection conversion of the end portion (peripheral portion of the image) of the circular fisheye image, and a synthesis image in which the detected edge information (peaking image) is superimposed is displayed. Note that the reason why the image obtained by perspective projection conversion of the end portion of the circular fisheye image is displayed in the initial display is that the captured object is significantly distorted in a compressed form at the end portion of the circular fisheye image, and is easily extracted as a high-frequency component, making it difficult for the user to determine whether the image is actually in focus.

[0034] Next, in step S610, the display control unit 109 determines whether or not the user has moved the perspective projection position by the user operation unit 108. If it is determined that the perspective projection position has moved, the process proceeds to step S611. In step S611, the display control unit 109 controls the conversion processing unit 107 so that a perspective projection conversion process is performed according to the moved position of the perspective projection position, and a perspective projection converted image is displayed. After the process of step S611, the process returns to step S610. Note that the timing to turn on the focus assist function may be after the perspective projection conversion display is selected. If the focus assist function is turned on after the perspective projection conversion display is selected, a peaking process is applied at the position where the perspective projection conversion display is performed, and a composite image with the detected edge information superimposed is displayed.

[0035] The display unit 110 is an EVF or a liquid crystal monitor, etc., and has a display panel (an organic EL panel or a liquid crystal panel). The display unit 110 displays an image generated under the control of the display control unit 109 as a live view image. The display unit 110 also functions as a notification unit that notifies the user of a partial region that is to be subjected to perspective projection conversion processing.

[0036] According to this embodiment, the user can easily adjust the focus even in the peripheral parts (edges) of the circular fisheye image by applying peaking processing to the perspective projection image and displaying a composite image with detected edge information superimposed. Therefore, the user can first focus on the central part with less distortion using the circular fisheye image, and then adjust the focus of the peripheral parts (edges) using the perspective projection image.

[0037] When performing the perspective projection conversion display, the area of ​​the circular fisheye image that has been converted and displayed by the perspective projection conversion may be displayed on an OSD as shown in FIG. 7. FIG. 7 is a diagram showing the display contents of the imaging device 100, and shows an example of an OSD display. By displaying the OSD, the area of ​​the original circular fisheye image where the focus is being confirmed can be easily recognized when the user performs an operation to move the perspective projection position. The area displayed as the initial image of the perspective projection image may be fixed to the left end, or may be switched depending on the contents of the captured image. For example, it is possible to calculate the variance value of the pixel values ​​of the captured image, and display a portion where the variance value is large and distortion is likely to be large (for example, a portion where the variance value is larger than a predetermined threshold value).

[0038] In addition, when performing VR shooting that allows stereoscopic viewing using the parallax between both eyes such as VR180, a right-eye circular fisheye image and a left-eye circular fisheye image are recorded as shown in FIG. 8. FIG. 8 is an explanatory diagram of an image captured by VR180. When performing perspective projection conversion display on the image shown in FIG. 8, as shown in FIG. 9, an OSD may be displayed to indicate which of the right-eye circular fisheye image and the left-eye circular fisheye image has been subjected to perspective projection conversion and displayed. FIG. 9 is a diagram showing the display contents of VR180 of the imaging device 100, and is a display example showing that perspective projection conversion is being performed on the left-eye circular fisheye image. In addition, a configuration may be made in which of the right-eye circular fisheye image and the left-eye circular fisheye image is to be displayed by the user operation unit 108.

[0039] In this embodiment, the partial area of ​​the captured image (fisheye image) that is the subject of the perspective projection conversion process is described as being the edge of the captured image, but this is not limited to this and may be any peripheral area of ​​the captured image.

[0040] Second embodiment Next, an imaging device 700 according to a second embodiment of the present invention will be described with reference to Fig. 10 to Fig. 13. Fig. 10 is a block diagram of the imaging device 700 according to this embodiment. The imaging device 700 differs from the imaging device 100 according to the first embodiment in that it has a reduction processing unit 701, in the processing of the image synthesis unit 106 and the display control unit 109 when the focus assist function is turned on, and in the display content on the display unit 110. Note that other configurations and operations of the imaging device 700 are similar to those of the imaging device 100, and therefore description thereof will be omitted.

[0041] An image display procedure by the display control unit 109 when the focus assist function is ON will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the display process of the imaging device 700.

[0042] First, in step S901, when the user turns on the focus assist function using the user operation unit 108, the display control unit 109 controls the conversion processing unit 107, the reduction processing unit 701, and the image synthesis unit 106 to be executed (ON). Next, in step S902, the reduction processing unit 701 reduces the fisheye image and the converted image so that the image (fisheye image) input from the imaging processing unit 103 and the image (converted image) input from the conversion processing unit 107 can be simultaneously displayed on the display unit 110. Then, the reduction processing unit 701 outputs a reduced fisheye image obtained by reducing the fisheye image, and a reduced converted image obtained by reducing the converted image.

[0043] Next, in step S903, the image synthesis unit 106 synthesizes the reduced fisheye image and the reduced perspective projection image input from the reduction processing unit 701 to generate an image as shown in Fig. 12. Next, in step S904, the peaking processing unit 105 executes peaking processing on the synthesized image input from the image synthesis unit 106, and outputs the execution result to the image synthesis unit 106. Next, in step S905, the image synthesis unit 106 synthesizes the image synthesized in step S903 (the image in Fig. 12) with the output of the peaking processing unit 105 to generate an image in which edge information is superimposed on the image in Fig. 12, and causes the display unit 110 to display it.

[0044] In this embodiment, a circular fisheye image and a perspective projection image are first synthesized, and then a synthesized image is generated by superimposing edge information detected by peaking processing. As a result, according to this embodiment, focus adjustment can be performed using a peaking image that simultaneously displays a circular fisheye image and a perspective projection image. Therefore, the user can first focus on the central part with less distortion using the circular fisheye image without switching between the circular fisheye image and the perspective projection image, and then adjust the focus of the image edges using the perspective projection image. As a result, intended focus adjustment can be performed more easily.

[0045] In addition, when performing VR shooting utilizing the parallax of both eyes such as VR180, as shown in Fig. 13, it is also possible to display on the OSD which of the circular fisheye images for the right eye and the left eye is displayed and which of the circular fisheye images for the right eye and the left eye is displayed by perspective projection conversion. Fig. 13 is a diagram showing the display contents of VR180 of the imaging device 700, and shows a state in which a circular fisheye image for the left eye is displayed and is displayed by perspective projection conversion. In addition, it is also possible to configure it so that the user operation unit 108 can switch between the image for the right eye and the image for the left eye.

[0046] Third embodiment Next, an imaging device 700 according to a third embodiment of the present invention will be described with reference to Fig. 10 and Fig. 14 to Fig. 16. The imaging device of this embodiment differs from the imaging device 700 of the second embodiment in the processing performed by the image synthesis unit 106 and the display control unit 109 and the display content on the display unit 110 when the focus assist function is turned on. Note that other configurations and operations of the imaging device of this embodiment are similar to those of the imaging device 700 of the second embodiment, and therefore descriptions thereof will be omitted.

[0047] The procedure for displaying an image by the display control unit 109 when the focus assist function is ON will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the display process of the imaging device in this embodiment.

[0048] First, in step S1101, when the user turns on the focus assist function using the user operation unit 108, the display control unit 109 controls the conversion processing unit 107, the reduction processing unit 701, and the image synthesis unit 106 to be turned on. Next, in step S1102, the conversion processing unit 107 executes perspective projection conversion processing on each of three locations (plurality of partial areas including a first partial area and a second partial area) of the center, left end, and right end of the circular fisheye image input from the imaging processing unit 103. Then, the conversion processing unit 107 outputs three perspective projection images (plurality of converted images including a first converted image and a second converted image). Next, in step S1103, the reduction processing unit 701 reduces each of the three perspective projection images input from the conversion processing unit 107 so that the three perspective projection images can be displayed simultaneously on the display unit 110.

[0049] Next, in step S1104, the image synthesis unit 106 synthesizes the reduced images input from the reduction processing unit 701 to generate an image (three reduced perspective projection images) as shown in FIG. 15. FIG. 15 is a diagram showing the display contents of the imaging device, showing three reduced perspective projection images. Next, in step S1105, the peaking processing unit 105 executes peaking processing on the synthesized image input from the image synthesis unit 106, and outputs the execution result to the image synthesis unit 106. Next, in step S1106, the image synthesis unit 106 synthesizes the image synthesized in step S1104 (the image in FIG. 15) with the output of the peaking processing unit 105, generates an image in which edge information is superimposed on the image in FIG. 15, and causes the display unit 110 to display it. By displaying in this manner, the user can focus on the center of the image in a state where it is actually displayed by the VR goggles, and adjust the focus at the edge of the image.

[0050] In this embodiment, the center, left end, and right end of the image are displayed at the same time, but for example, the images may be synthesized and displayed after perspective projection conversion at other viewpoints such as the upper end and the lower end. Also, a configuration may be made in which the user can set which viewpoint is displayed on the display screen of each perspective projection conversion from the user operation unit 108. Also, when performing VR shooting that allows stereoscopic viewing using the parallax of both eyes such as VR180, both images after perspective projection conversion for the right eye and the left eye may be displayed at the same time as shown in FIG. 16. FIG. 16 is a diagram showing the display contents of the VR180 of the imaging device in this embodiment. With such a display, the user can perform the intended focus adjustment without switching between the image for the right eye and the image for the left eye. Also, as in the case of FIG. 10, the area of ​​the circular fisheye image that is displayed after perspective projection conversion may be displayed on the OSD.

[0051] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0052] According to each embodiment, it is possible to provide an imaging device, an imaging device control method, a program, and a storage medium that allow a user to easily adjust the focus during VR shooting.

[0053] The disclosure of each embodiment includes the following configurations and methods.

[0054] (Configuration 1) An imaging unit that acquires a captured image; a conversion processing unit that performs a predetermined conversion process on at least one partial region of the captured image to generate a converted image; a peaking processing unit that performs peaking processing for focus adjustment on at least one of the captured image and the converted image to generate a peaking image; an image synthesis unit that generates a synthesis image of at least one of the captured image and the converted image and the peaking image; a display unit for displaying the composite image; when the display unit is set to perform the peaking process on the converted image and display the composite image, the display unit displays the composite image of the converted image and the peaking image; The imaging device, wherein the partial region includes a peripheral portion of the captured image. (Configuration 2) 2. The imaging device according to configuration 1, wherein the peripheral portion includes an edge portion of the captured image. (Configuration 3) 3. The imaging device according to claim 1, wherein the display unit displays the composite image of the converted image and the peaking image in an initial display. (Configuration 4) 4. The imaging device according to any one of configurations 1 to 3, wherein the display unit simultaneously displays the captured image and the converted image. (Configuration 5) 5. The imaging device according to configuration 4, wherein the display unit simultaneously displays the captured image and the composite image. (Configuration 6) A reduction processing unit is further included, 6. The imaging device according to configuration 4 or 5, wherein the captured image and the converted image are each an image reduced by the reduction processing unit. (Configuration 7) 7. The imaging device according to configuration 6, wherein the peaking processing unit performs the peaking processing on the captured image or the converted image that has been reduced by the reduction processing unit. (Configuration 8) the partial region includes a first partial region and a second partial region, The conversion processing unit is performing the predetermined conversion process on the first partial region to generate a first converted image; performing the predetermined conversion process on the second partial region to generate a second converted image; 8. The imaging device according to any one of configurations 1 to 7, wherein the display unit simultaneously displays the first converted image and the second converted image. (Configuration 9) The device further includes a user operation unit, 9. The imaging device according to any one of configurations 1 to 8, wherein the conversion processing unit changes a position of the partial region that is to be subjected to the predetermined conversion processing based on a signal from the user operation unit. (Configuration 10) 10. The imaging device according to any one of configurations 1 to 9, wherein the predetermined transformation process is a perspective projection transformation process. (Configuration 11) The imaging device according to configuration 10, wherein the converted image corresponds to an image to be viewed as VR content. (Configuration 12) 12. The imaging device according to any one of configurations 1 to 11, wherein the peaking image is an image that includes edge information in at least one of the captured image and the converted image. (Configuration 13) 13. The imaging device according to any one of configurations 1 to 12, further comprising a notification unit that notifies a user of the partial region that is to be the target of the predetermined conversion process. (Configuration 14) 14. The imaging device according to any one of configurations 1 to 13, wherein the display unit displays, in an initial display, the partial region of the captured image in which the variance value is greater than a predetermined variance value. (Configuration 15) 15. The imaging device according to any one of configurations 1 to 14, wherein the captured image is a fisheye image obtained using a fisheye lens. (Method 1) acquiring a captured image; performing a predetermined conversion process on at least one partial region of the captured image to generate a converted image; performing a peaking process for focus adjustment on at least one of the captured image and the converted image to generate a peaking image; generating a composite image of at least one of the captured image and the converted image and the peaking image; and displaying the composite image. A method for controlling an imaging device, wherein the partial region includes a peripheral portion of the captured image. (Configuration 16) A program for causing a computer to execute the control method according to method 16. (Configuration 17) A computer-readable storage medium storing the program according to configuration 16.

[0055] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0056] 100 Imaging device 101 Lens unit (imaging unit) 102 Image sensor section (image pickup section) 103 Imaging processing section (imaging section) 105 Peaking processing section 106 Image synthesis unit 107 Conversion Processing Unit 110 Display section

Claims

1. an imaging unit that acquires a captured image; a conversion processing unit that performs a predetermined conversion process for correcting image distortion on at least one partial region of the captured image to generate a converted image; a peaking processing unit that performs peaking processing for focus adjustment on at least one of the captured image and the converted image to generate a peaking image; an image synthesis unit that generates a synthesis image of at least one of the captured image and the converted image and the peaking image; a display control unit that controls the composite image to be displayed on a display unit, when the display control unit is set to perform the peaking process on the converted image and display the composite image, the display unit displays the composite image of the converted image and the peaking image; The imaging device, wherein the partial region includes a peripheral portion of the captured image.

2. The imaging device according to claim 1 , wherein the peripheral portion includes an edge portion of the captured image.

3. The imaging device according to claim 1 , wherein the display control unit causes the display unit to display the composite image of the converted image and the peaking image in an initial display.

4. The imaging device according to claim 1 , wherein the display control unit causes the captured image and the converted image to be displayed simultaneously on the display unit.

5. The imaging device according to claim 4 , wherein the display control unit causes the captured image and the composite image to be displayed simultaneously on the display unit.

6. Further comprising a reduction processing unit, 5. The imaging device according to claim 4, wherein the captured image and the converted image are each an image reduced by the reduction processing unit.

7. 7. The imaging device according to claim 6, wherein the peaking processing unit performs the peaking processing on the captured image or the converted image reduced by the reduction processing unit.

8. the partial regions include a first partial region and a second partial region; The conversion processing unit performing the predetermined conversion process on the first partial region to generate a first converted image; performing the predetermined conversion process on the second partial region to generate a second converted image; The imaging device according to claim 1 , wherein the display control unit causes the display unit to simultaneously display the first converted image and the second converted image.

9. The device further includes a user operation unit, 2. The imaging device according to claim 1, wherein the conversion processing unit changes the position of the partial area that is the target of the predetermined conversion processing based on a signal from the user operation unit.

10. 2. The imaging device according to claim 1, wherein the predetermined transformation process is a perspective projection transformation process.

11. The imaging device according to claim 10 , wherein the converted image corresponds to an image to be viewed as VR content.

12. The imaging device according to claim 1 , wherein the peaking image is an image containing edge information in at least one of the captured image and the converted image.

13. 2. The imaging device according to claim 1, further comprising a notification unit that notifies a user of the partial area that is to be subjected to the predetermined conversion process.

14. The imaging device according to claim 1 , wherein the display control unit causes the display unit to initially display the partial region of the captured image whose variance value is greater than a predetermined variance value.

15. 15. The imaging device according to claim 1, wherein the captured image is a fisheye image obtained using a fisheye lens.

16. acquiring a captured image; performing a predetermined conversion process for correcting image distortion on at least one partial region of the captured image to generate a converted image; performing peaking processing for focus adjustment on at least one of the captured image and the converted image to generate a peaking image; generating a composite image of at least one of the captured image and the converted image and the peaking image; and controlling the composite image to be displayed on a display unit, A method for controlling an imaging device, wherein the partial region includes a peripheral portion of the captured image.

17. A program causing a computer to execute the control method according to claim 16.

18. A computer-readable storage medium storing the program according to claim 17.

Images