Photographing device and photographing control program

Abstract

Provided is a photographing device capable of automatically performing photographing processing that avoids generation of moiré fringes. The photographing device comprises: a camera unit provided with a photographing element; a determination part that determines whether a moiré fringe region is present in an image formed on the basis of an output of the photographing element; and a photographing processing part that switches, when the determination part determines that the moiré fringe region is present in the image, a photographing-related optical condition of the camera unit, and continues to perform photographing processing.

Description

Shooting device and shooting control program Technical Field

[0001] The present invention relates to a photographing device and a photographing control program. Background Art

[0002] Photographic devices that generate image data based on image signals output from a photographic element have become widely used. A photographic element generally has a large number of pixels regularly arranged vertically and horizontally, and each pixel performs photoelectric conversion on the subject image formed on the pixel plane to output an image signal corresponding to the subject image. Here, when at least a portion of the subject imaged via the optical lens has a spatial frequency greater than 1 / 2 of the Nyquist frequency specified by the pixel pitch of the photographic element, moiré fringes are generated in the corresponding area of ​​the image generated from the output image signal. Moiré fringes appear in the generated image as false colors that are different from the stripe pattern and actual color that do not actually exist. Therefore, technologies are being developed to notify users in advance when such moiré fringes are expected to be generated in a photographic image (for example, refer to Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2010-10881.

[0006] Summary of the Invention

[0007] Problems to be solved by the invention

[0008] Smartphones are embedded with camera units, and smartphones also function as imaging devices. With the widespread adoption of smartphones, the opportunities for users with limited camera knowledge to enjoy photography are increasing. In this context, there is a desire for imaging devices (specifically smartphones) that automatically perform imaging processing to avoid moiré patterns, without requiring users to have specific camera knowledge.

[0009] The present invention has been made to solve such a problem, and provides an imaging device and the like that can automatically execute an imaging process to avoid the occurrence of moire fringes.

[0010] Solutions for solving problems

[0011] The shooting device in the first aspect of the present invention comprises: a camera unit having a shooting element; a discrimination unit which discriminates whether a moiré fringe area exists in an image formed based on the output of the shooting element; and a shooting processing unit which switches optical conditions related to shooting by the camera unit and continues shooting processing when the discrimination unit discriminates that the moiré fringe area exists in the image.

[0012] The shooting control program in the second aspect of the present invention enables the computer to execute the following steps: a judgment step of judging whether there is a moire fringe area in the image formed based on the output of the shooting element possessed by the camera unit; and a shooting processing step of switching the optical conditions related to the shooting of the above-mentioned camera unit and continuing the shooting processing when it is judged in the above-mentioned judgment step that the above-mentioned moire fringe area exists in the above-mentioned image.

[0013] Effects of the Invention

[0014] According to the present invention, it is possible to provide an imaging device or the like that can automatically execute an imaging process for avoiding the occurrence of moire fringes. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG. 1 is a diagram showing the appearance of a camera according to a first example of the present embodiment.

[0016] FIG. 2 is a diagram showing a main hardware configuration of the imaging device.

[0017] FIG. 3 is a diagram showing an example of a scene in which a moire fringe image is generated.

[0018] FIG. 4 is a diagram for explaining the pixel arrangement of the imaging element and a method for determining moiré fringes.

[0019] FIG5 is a diagram illustrating the first switching process.

[0020] FIG6 is a diagram illustrating the second switching process.

[0021] FIG. 7 is a flowchart showing a processing procedure of the moire avoidance process.

[0022] FIG8 is a diagram showing a main hardware configuration of an imaging device according to a second example of the present embodiment.

[0023] FIG. 9 is a flowchart showing a processing procedure of the moire avoidance process. DETAILED DESCRIPTION

[0024] Hereinafter, the present invention will be described by way of embodiments of the invention, but the invention according to the scope of the claims is not limited to the following embodiments. In addition, not all of the configurations described in the embodiments are necessarily means for solving the problems.

[0025] The following describes the first embodiment and the second embodiment of the present embodiment in sequence. FIG1 is a diagram showing the appearance of the camera 100 of the first embodiment of the present embodiment. In particular, FIG1(A) is a diagram mainly showing the first side of the camera 100, and FIG1(B) is a diagram mainly showing the second side opposite to the first side. The camera 100 of the present embodiment is a so-called smartphone, in other words, a smartphone that also functions as a camera. The following describes the camera function related to the present invention among the functions of the smartphone, and omits descriptions of other functions of the smartphone, such as the use of image data generated by shooting. In addition, in the present embodiment, the camera 100 is described using a smartphone as an example, but it can of course also be a camera that is a single camera, or a device that is embedded in a tablet terminal or the like and has a camera function.

[0026] The photographing device 100 includes a first camera unit 110 and a second camera unit 120 arranged in the same direction on the first surface side. The first camera unit 110 is a camera unit for generating wide-angle images. The second camera unit 120 is a camera unit for generating telephoto images. When the user wants to obtain a wide-angle image, he specifies the first camera unit to shoot, and when the user wants to obtain a telephoto image, he specifies the second camera unit to shoot. The first camera unit 110 and the second camera unit 120 are arranged parallel to the long side of the photographing device 100 in the figure, but the arrangement of the two camera units is not limited to this. For example, they can also be arranged along a straight line obliquely intersecting the long side. In addition, the arrangement of the first camera unit 110 and the second camera unit 120 can be opposite to the position in the figure.

[0027] On the second surface, the camera 100 includes a display 130. Display 130 is a display device using, for example, an organic EL (Electro Luminescence) panel, and displays a real-time image of the subject before shooting (live view display) or after shooting. Furthermore, a selfie camera unit, separate from the first and second camera units 110 and 120, can be provided on the second surface.

[0028] A shutter button 161 is provided on the side of the camera 100. The user can provide a shooting instruction to the camera 100 by pressing the shutter button 161. In addition, a touch panel 162 is provided superimposed on the display 130. The user can also give the camera 100 a shooting instruction by tapping the shutter button displayed on the display 130 instead of pressing the shutter button 161. In addition, the user can also tap any part of the subject image displayed in the live view to designate a certain area including that part as the focus area. In addition, the user can switch between the first camera unit 110 and the second camera unit 120 or select a displayed menu item through a touch action such as tapping.

[0029] 2 is a diagram showing the main hardware configuration of the imaging device 100. The imaging device 100 includes, in addition to the first camera unit 110, the second camera unit 120, and the display 130 described above, a system control unit 150 for controlling these, and peripheral units cooperating with the system control unit 150.

[0030] As described above, the first camera unit 110 is a camera unit for generating wide-angle images and primarily includes a first optical system 111, a first drive mechanism 112, a first imaging element 113, and a first analog front end (AFE) 114. The first optical system 111 is an optical system that forms an image of incident subject light beams onto the imaging surface of the first imaging element 113. Although illustrated in the figure as a single lens, it is typically composed of multiple lenses, at least a portion of which is a focus lens capable of moving forward and backward along the optical axis. The first drive mechanism 112 is a drive mechanism for moving the focus lens of the first optical system 111 along the optical axis and includes an actuator that operates in accordance with instructions from the system control unit 150.

[0031] The first image sensor 113 is, for example, a CMOS image sensor. It has a Bayer color filter arrangement, and each pixel constituting the first image sensor 113 is classified as one of R (red), G (green), and B (blue). Based on instructions from the system control unit 150, the first image sensor 113 transmits an image signal as an output signal to the first AFE 114. The first AFE 114 adjusts the image signal level based on the gain instructed by the system control unit 150, converts the image signal into digital data through A / D conversion, and transmits the digital data to the working memory 141.

[0032] As described above, the second camera unit 120 is a camera unit for generating telephoto images and primarily comprises a second optical system 121, a second drive mechanism 122, a second imaging element 123, and a second analog front end (AFE) 124. The second optical system 121 is an optical system for forming an image of incident subject light beams onto the imaging surface of the second imaging element 123. Although illustrated as a single lens in the figure, the second optical system 121, like the first optical system 111, is generally composed of multiple lenses, at least part of which is a focusing lens that can move forward and backward along the optical axis. However, the second optical system 121 has a longer focal length than the first optical system 111 and a different lens configuration. The second drive mechanism 122 is a drive mechanism for moving the focusing lens of the second optical system 121 along the optical axis and comprises an actuator that operates in response to instructions from the system control unit 150. Furthermore, in the first embodiment, both the first optical system 111 and the second optical system 121 are single-focus optical systems with a fixed focal length.

[0033] The second imaging element 123 is, for example, a CMOS image sensor. The second imaging element 123 has a color filter using the same Bayer arrangement as the first imaging element 113. Each pixel constituting the first imaging element 113 is classified as either an R (red) pixel, a G (green) pixel, or a B (blue) pixel. The second imaging element 123 transmits an image signal as an output signal to the second AFE 124 in accordance with instructions from the system control unit 150. The second AFE 124 adjusts the level of the image signal based on the gain instructed by the system control unit 150, converts the signal into digital data through A / D conversion, and transmits the result to the working memory 141. Furthermore, the first imaging element 113 and the second imaging element may be of the same type, or may be separate imaging elements having different pixel pitches, for example.

[0034] The system control unit 150 is a processor (CPU) that directly or indirectly controls each unit that makes up the imaging device 100. The system control unit 150 functions as a control unit for various functions based on the executed control program. For example, it functions as a focus control unit when performing focus control on the first camera unit 110 and the second camera unit 120, and as a display control unit when displaying captured images on the display 130. In particular, the system control unit 150 functions as a determination unit 151 and an imaging processing unit 152 based on processing directed by the imaging control program. The determination unit 151 primarily determines whether a moiré pattern region exists in a moving image formed based on the output of the imaging element (either the first imaging element 113 or the second imaging element 123) of the active unit selected by the user and in use in the first camera unit 110 or the second camera unit 120. The imaging processing unit 152 primarily switches the active unit from the currently selected one of the first camera unit 110 or the second camera unit 120 to the unselected one, and then continues the imaging process. These specific processes will be described later.

[0035] The imaging device 100 primarily includes a working memory 141, an image processing unit 142, an operation unit 160, a storage unit 170, and a communication interface 180 as peripheral units that cooperate with the system control unit 150. The working memory 141 is a volatile high-speed memory, such as an SRAM (Static Random Access Memory). The working memory 141 receives pixel data sequentially converted by the first AFE 114 and the second AFE 124, aggregates the data into a single frame, and transmits it to the image processing unit 142. Furthermore, the working memory 141 is also used as a temporary storage area during the image processing stages performed by the image processing unit 142 and the focusing stages performed by the system control unit 150.

[0036] The image processing unit 142, comprised of, for example, an ASIC (application-specific integrated circuit) specialized for image processing, performs various image processing operations on the received frame data, including interpolation, to generate image data conforming to a predetermined format. If the generated image data is for storage, it is stored in the storage unit 170; if it is for display, it is displayed on the display 130. Furthermore, if the determination unit 151 determines whether a moiré region exists in the moving image, the image processing unit 142 performs extraction processing on the moving image to extract the moiré region. The specific extraction processing will be described later.

[0037] The operation unit 160 is an input device including a shutter button 161 and a touch panel 162, and is a component that is operated when the user provides instructions to the shooting device 100. When the shooting device 100 accepts voice input, a microphone can also be included in the operation unit 160. The storage unit 170 is a non-volatile memory and includes, for example, an SSD (Solid State Drive). In addition to storing image data generated by shooting, the storage unit 170 also stores constants, variables, setting values, and control programs required for the operation of the shooting device 100. The communication interface 180 may include a communication unit for a 5G line or a wireless LAN. The communication interface 180 is used, for example, when transmitting the generated image data to an external device.

[0038] Similarly to the first and second image sensors 113 and 123 in this embodiment, conventional image sensors used in imaging devices have a large number of pixels regularly arranged vertically and horizontally. Each pixel performs photoelectric conversion on the subject image formed on the pixel plane, outputting an image signal corresponding to the subject image. When at least a portion of the subject image formed by the optical lens has a spatial frequency greater than half the Nyquist frequency defined by the pixel pitch of the image sensor, moiré fringes are generated in the corresponding region of the image generated from the output image signal. Moiré fringes appear in the generated image as a false color that differs from the actual color and a fringe pattern that does not actually exist.

[0039] FIG3 illustrates an example of a scenario in which a moiré image is generated when capturing images using a camera. Specifically, the diagram shows a user capturing a television image 901 displayed on a television monitor 900 using the camera 100. A live view image 301 containing a moiré region 302 is displayed on the monitor 130 as a live view image. As described above, the live view image 301 is an image generated by the output of the imaging element of the active unit selected and in use by the user, either the first camera unit 110 or the second camera unit 120. The live view image 301 is displayed on the monitor 130 as a live view image at, for example, 60 fps. When the shutter button 161, a physical switch, is tapped or the shutter button 390, displayed overlapping the live view image 301, is pressed, the capture processing unit 152 converts the live view image displayed at that time or the image captured at that time into image data for storage in the storage unit 170.

[0040] As shown, moving image 301 includes moiré regions 302, which exhibit a distorted, grid-like pseudo pattern that is absent in television image 901. In television display 900, pixels that emit light, such as one of the RGB colors, are regularly arranged, and this light emission creates television image 901, which can produce moiré in moving image 301. The generation of moiré is influenced not only by the spatial frequency of the subject's optical image formed on the imaging element, but also by the pattern and brightness of television image 901. While this embodiment uses television image 901 as an example subject, the subject is of course not limited to this. For example, moiré can occur when capturing subjects containing repetitive patterns, such as the fabric of clothing or the walls of a building.

[0041] The determination unit 151 determines whether a moiré region exists in the moving image 301 formed based on the image signal output by the image sensor of the moving unit. FIG4 illustrates the pixel arrangement of the first image sensor 113 and the method for determining moiré. Furthermore, since the pixel arrangement of the second image sensor 123 is identical to that of the first image sensor 113, the method for determining moiré in the image formed based on the output of the second image sensor 123 is also the same.

[0042] As described above, the first imaging element 113 is composed of R pixels, G pixels, and B pixels corresponding to the Bayer array. In this embodiment, the focus is on the G pixels that occupy two pixels in the four pixels (two pixels vertically and two pixels horizontally) that form the repeating unit in the Bayer array. As shown in the figure, each G pixel is labeled with reference numerals G00 to G33. In this case, if each pixel value is expressed as V(Gmn), the evaluation value E of the unit area containing eight G pixels (four pixels vertically and four pixels horizontally) is defined as

[0043] E={|(V(G02)+V(G22))-(V(G11)+V(G13))|+|(V(G11)+V(G31))-(V(G20)+V(G22))|} / 4

[0044] When the evaluation value E is a value greater than a preset threshold value, the unit area is determined to be a moire unit area. The moire area 302 shown in FIG3 is a collection area of ​​the moire unit areas determined in this way.

[0045] When determining that a collection of moiré unit regions of a predetermined size or larger exists on the moving image 301, the determination unit 151 determines that a moiré region exists on the moving image 301. The method for determining a moiré region is not limited thereto, and other known methods may be used.

[0046] Next, the processing performed by the image processing unit 152 when the determination unit 151 determines that a moire region exists in the moving image 301 will be described. First, the processing performed by the image processing unit 152 when the moving unit is the second camera unit 120 and the moving image 301 is a telephoto image 320 formed based on the output of the second imaging element 123 will be described.

[0047] FIG5 illustrates a first switching process for switching the active unit from the second camera unit 120 to the first camera unit 110. When the active unit is the second camera unit 120, the active image 301 is a telephoto image 320. When the determination unit 151 determines that the telephoto image 320 contains a moiré region 323, the imaging processing unit 152 switches the active unit from the second camera unit 120 to the first camera unit 110. Note that "switching" here can be automatic or user-triggered. The working memory 141 then receives frame data resulting from the conversion of the image signal output by the first imaging element 113 by the first AFE 114. The image processing unit 142 receives this frame data and generates a wide-angle image 310. Furthermore, the image processing unit 142 clips out an image region corresponding to the angle of view of the second optical system 121, that is, the entire telephoto image 320, from the generated wide-angle image 310 to generate a clipped image 311. It should be noted that the cropped image here can also be an image formed by fusing the image area corresponding to the viewing angle of the wide-angle image in the telephoto image generated by the telephoto camera and the image area other than the aforementioned image area generated by the wide-angle camera.

[0048] Cutout image 311 is displayed on display 130 as live image 301. Specifically, if telephoto image 320 contains moiré region 323, the active unit is automatically switched, and cutout image 311, cutout from wide-angle image 310, is displayed on display 130 as live image 301. When switching from second camera unit 120 to first camera unit 110, the optical system used to form the subject image is switched to first optical system 111, which is different from second optical system 121. In other words, the optical conditions related to imaging are switched. Therefore, even if moiré occurs due to the relationship between the spatial frequency of the subject image formed on the imaging surface of second imaging element 123 and the pixel pitch of second imaging element 123, it is expected that moiré will not occur due to the relationship between the spatial frequency of the subject image formed on the imaging surface of first imaging element 113 and the pixel pitch of first imaging element 113. In other words, as long as the relationship between the spatial frequency of the subject image formed on the imaging surface of the first imaging element 113 and the pixel pitch of the first imaging surface element 113 does not match the conditions that cause moiré fringes, no moiré fringes will appear in the cropped image 311. Specifically, in the illustrated cropped image 311, moiré fringes will not appear in the moiré-corresponding region 313 corresponding to the moiré fringing region 323 of the telephoto image 320.

[0049] Thus, when shutter button 161 is pressed or shutter button 390 is tapped while cropped image 311 is displayed on display 130, imaging processing unit 152 converts cropped image 311 displayed at that time, or cropped image 311 cropped from wide-angle image 310 captured at that time, into image data and stores it in storage unit 170. Automatically performing this first switching process allows the user to obtain image data for a desired image without moiré patterns, without the user being aware of the process. Furthermore, in this embodiment, image processing for cropped image 311 may be performed after wide-angle image 310 is formed, but first AFE 114 selectively performs A / D conversion on the image signal used to form cropped image 311, among the image signals output by first imaging element 113, and transmits the converted signal to working memory 141.

[0050] Next, the processing of imaging processing unit 152 will be described when the active unit is first camera unit 110 and a moire region is determined to exist in wide-angle image 310, which is active image 301. FIG6 is a diagram illustrating a second switching process for switching the active unit from first camera unit 110 to second camera unit 120.

[0051] When the active unit is first camera unit 110, active image 301 is wide-angle image 310. When determination unit 151 determines that wide-angle image 310 contains moiré region 314, imaging processing unit 152 switches the active unit from first camera unit 110 to second camera unit 120. Working memory 141 then receives frame data resulting from conversion of the image signal output by second imaging element 123 by second AFE 124. Image processing unit 142 receives this frame data and generates telephoto image 320.

[0052] On display 130, Tele image 320 is displayed as live image 301. Since the image area corresponding to the angle of view of first optical system 111, that is, the image area corresponding to the entire area of ​​Wide-angle image 310, cannot be cut out from Tele image 320, the entire area of ​​generated Tele image 320 is displayed as is. However, live image 301, displayed as a live view image, switches to an image with a different angle of view at some point, potentially causing discomfort to the user. Therefore, the capture processing unit 152 notifies the user regarding the angle of view adjustment. Specifically, notification display 325 is displayed superimposed on Tele image 320. Notification display 325 includes, for example, a notification message stating, "Switching to Tele lens due to detection of moiré fringes."

[0053] After a certain period of time has passed since the notification was initiated, the imaging processing unit 152 removes the notification display 325. Thus, when switching from the first camera unit 110 to the second camera unit 120, the optical system used to form the subject image is switched to the second optical system 121, which is different from the first optical system 111. In other words, the optical conditions related to imaging are changed. Therefore, even if moiré fringes occur due to the relationship between the spatial frequency of the subject image formed on the imaging surface of the first imaging element 113 and the pixel pitch of the first imaging element 113, it is expected that moiré fringes will not occur due to the relationship between the spatial frequency of the subject image formed on the imaging surface of the second imaging element 123 and the pixel pitch of the second imaging element 123. In other words, as long as the relationship between the spatial frequency of the subject image formed on the imaging surface of the second imaging element 123 and the pixel pitch of the second imaging element 123 does not match the conditions that cause moiré fringes, no moiré fringes will appear in the telephoto image 320. That is, in the illustrated telephoto image 320 , moiré fringes do not appear in the moiré fringe corresponding region 324 corresponding to the moiré fringe region 314 of the wide-angle image 310 .

[0054] In this manner, when shutter button 161 is pressed or shutter button 390 is tapped while Tele image 320 is displayed on display 130, capture processing unit 152 converts Tele image 320 displayed at that time or Tele image 320 newly captured at that time into image data and stores it in storage unit 170. By automatically performing this second switching process, the user can obtain image data of a desired image that does not include a moiré region while being aware of the switch to Tele image 320.

[0055] Next, the process sequence for the moiré avoidance process, which is periodically executed during live view image display, is described. Figure 7 is a flow chart illustrating the moiré avoidance process. The process begins at a point in time when the live view image is displayed that matches the set period.

[0056] In step S101, the determination unit 151 determines whether a moire region exists in the moving image 301. If it is determined that no moire region exists, the moire avoidance process in that cycle ends. If it is determined that a moire region exists, the process proceeds to step S102.

[0057] Entering step S102, the photographing processing unit 152 determines whether the current active unit is the second camera unit 120 or the first camera unit 110. If it is the second camera unit 120, the process proceeds to step S103; if it is the first camera unit 110, the process proceeds to step S106.

[0058] After proceeding to step S103, the imaging processing unit 152 switches the active unit to the first camera unit 110. When the active unit is switched to the first camera unit 110, the process proceeds to step S104, where the working memory 141 receives the frame data converted by the first AFE 114. The image processing unit 142 receives this frame data and generates a wide-angle image 310. Furthermore, the image processing unit 142 cuts out an image area corresponding to the entire area of ​​the telephoto image 320 from the generated wide-angle image 310, generating a cutout image 311.

[0059] Proceeding to step S105, the capture processing unit 152 displays the cutout image 311 generated by the image processing unit 142 on the display 130, thus completing the series of moiré fringe avoidance processes. Alternatively, after starting the live view display of the cutout image 311, the shutter button 161 may be pressed or the shutter button 390 may be tapped to complete the capture and storage instructions, thereby terminating the storage process. Thereafter, the live view display of the cutout image 311 ends, and the active unit returns to the original second camera unit 120.

[0060] After proceeding from step S102 to step S106, the capture processing unit 152 switches the active unit to the second camera unit 120. When the active unit is switched to the second camera unit 120, the process proceeds to step S107, where the working memory 141 receives the frame data converted by the second AFE 124. The image processing unit 142 receives this frame data and generates a Tele image 320. The capture processing unit 152 superimposes a notification display 325 on the Tele image 320 generated by the image processing unit 142 and displays it on the display 130. After a predetermined time has elapsed, the notification display 325 is removed in step S108, concluding the series of moiré avoidance processes. Alternatively, after starting the live view display of the Tele image 320, the shutter button 161 may be pressed or the shutter button 390 may be tapped to complete the capture and storage instruction, terminating the storage process. After this, the live view display of the Tele image 320 ends, and the active unit returns to the original first camera unit 110.

[0061] Next, the second embodiment of the present embodiment will be described. FIG8 is a diagram showing the main hardware configuration of the second embodiment of the present embodiment. The second embodiment of the photographing device 100' is also the same as the photographing device 100 of the first embodiment, and is a so-called smartphone. However, unlike the photographing device 100, the photographing device 100' has a separate camera unit 110', and the separate camera unit 110' has an optical system 111' as a so-called zoom lens that can change the focal length. In the following description, unless otherwise specified, the same reference numerals are given to the same units as the photographing device 100 of the first embodiment, and their descriptions are omitted.

[0062] The camera unit 110' mainly includes an optical system 111', a drive mechanism 112', a first imaging element 113, and a first AFE 114. As described above, in this embodiment, the imaging device 100' includes a separate camera unit 110'. However, since the imaging element and analog front-end functions of the camera unit 110' are substantially the same as those of the first camera unit 110 in the first embodiment, for convenience, they will be referred to as the first imaging element 113 and the first AFE 114 in this description.

[0063] Optical system 111' is used to form an image of incident subject light beams onto the imaging surface of first imaging element 113. Optical system 111' comprises a focusing lens group 111a and a magnification-varying lens group 111b. For convenience, the figures illustrate focusing lens group 111a and magnification-varying lens group 111b as a single lens arranged in tandem. However, each lens group is typically composed of multiple lenses, and the two lens groups do not necessarily need to be arranged in tandem.

[0064] The drive mechanism 112' includes a focus lens drive mechanism 112a and a magnification changing lens drive mechanism 112b. The focus lens drive mechanism 112a is a drive mechanism for moving at least a portion of the lenses of the focus lens group 111a along the optical axis, and includes an actuator that operates according to instructions from the system control unit 150. The system control unit 150 adjusts the focus of the subject image reaching the first imaging element 113 by providing instructions to the focus lens drive mechanism 112a. The magnification changing lens drive mechanism 112b is a drive mechanism for moving at least a portion of the lenses of the magnification changing lens group 111b along the optical axis, and includes an actuator that operates according to instructions from the system control unit 150. The system control unit 150 adjusts the angle of view of the subject image reaching the first imaging element 113 by providing instructions to the magnification changing lens drive mechanism 112b.

[0065] Next, we will describe the characteristic imaging control of the imaging device 100', which controls the moiré fringe avoidance process periodically executed during the display of a live view image. Figure 9 is a flow chart illustrating the moiré fringe avoidance process. This process begins at a time point consistent with a set period during the display of a live view image.

[0066] In step S201, the determination unit 151 determines whether a moiré region exists in the active image 301. In the second embodiment, the active image 301 is always formed based on the output of the first imaging element 113, and in this sense, the camera unit 110' is always the active unit. If it is determined that no moiré region exists, the moiré avoidance process for that cycle ends. If it is determined that a moiré region exists, the process proceeds to step S202.

[0067] If the process proceeds to step S202, the imaging processing unit 152 checks whether the current magnification changing lens group 111b is at the wide-angle end. If not, the process proceeds to step S203; if so, the process proceeds to step S207.

[0068] If the process proceeds to step S203, the imaging processing unit 152 drives the variable magnification lens drive mechanism 112b to cause the variable magnification lens group 111b to zoom toward the wide-angle side by a predetermined angle of view change. If the zoom process is complete, the determination unit 151 determines in step S204 whether a moiré region remains in the moving image 301 generated after the zoom process. If the moiré region remains, the process returns to step S202 and repeats the zoom process toward the wide-angle side. If the moiré region does not remain, the process proceeds to step S205.

[0069] If the process proceeds to step S205, the image processing unit 142 cuts out, from the generated moving image 301, an image region corresponding to the angle of view of the moving image 301 at the start of the moiré avoidance process, thereby generating a cutout image 311. Next, the process proceeds to step S206, and the capture processing unit 152 displays the cutout image 311 generated by the image processing unit 142 on the display 130, thereby concluding the series of moiré avoidance processes. Alternatively, after the live view display of the cutout image 311, the shutter button 161 may be pressed or the shutter button 390 may be tapped to complete the capture and storage instruction, thereby terminating the storage process. Thereafter, the live view display of the cutout image 311 is terminated, and the magnification change lens group 111b is returned to the magnification change state at the start of the moiré avoidance process.

[0070] When the process proceeds from step S202 to step S207, the imaging processing unit 152 drives the variable magnification lens drive mechanism 112b to cause the variable magnification lens group 111b to zoom toward the telephoto side by a predetermined angle of view change. If the zoom process is complete, the determination unit 151 determines in step S208 whether a moiré region remains in the moving image 301 generated after the zoom process. If it is determined that a moiré region remains, the process proceeds to step S211 to determine whether the variable magnification lens group 111b is in the telephoto end position after zooming. If it is not in the telephoto end position, the process returns to step S207 and repeats the zoom process toward the telephoto end position. If it is confirmed to be in the telephoto end position, the variable magnification lens group 111b is returned to the zoom state at the start of the moiré avoidance process, abandoning moiré avoidance and terminating the moiré avoidance process for that cycle.

[0071] If it is determined in step S208 that no moire area remains, the process proceeds to step S209. In step S209, the image processing unit 142 generates the moving image 301, and the processing unit 152 superimposes the notification display 325 on the moving image 301 generated by the image processing unit 142 and displays it on the display 130. As described above, the notification display 325 includes notification content informing of the change in angle of view. After a predetermined time has elapsed, the notification display 325 is removed in step S210, concluding the series of moiré avoidance processes. Alternatively, after starting the live view display of the moving image 301 by changing the magnification to the telephoto side, the shutter button 161 may be pressed or the shutter button 390 may be tapped to complete the instruction to store the image, terminating the storage process. Thereafter, the live view display of the telephoto angle of view is terminated, and the magnification change lens group 111b is returned to the magnification change state at the start of the moiré avoidance process.

[0072] By automatically zooming the variable magnification lens group 111b in this manner, it is possible to switch optical conditions related to shooting. Specifically, even if the relationship between the spatial frequency of the subject image formed on the imaging surface of the first imaging element 113 and the pixel pitch of the first imaging element 113 at a certain field of view angle (a certain focal length) results in moiré fringes, it can be expected that the relationship between the spatial frequency of the subject image formed on the imaging surface of the first imaging element 113 and the pixel pitch of the first imaging element 113 at a different field of view angle (a different focal length) will not result in moiré fringes. By automatically zooming the zoom lens group in this manner, the user can often obtain image data that does not contain moiré fringes.

[0073] While the present embodiment has been described above using two embodiments, the first and second embodiments may also be combined. Furthermore, while the first embodiment describes an imaging device 100 having two camera units, the imaging device may also be configured to have three or more camera units. In this case, if the determination unit 151 determines that a moire region exists, the imaging processing unit 152 preferably switches the active unit to a camera unit having an optical system with a wider angle.

[0074] In addition, although the present application provides a photographing device that can automatically perform photographing processing to avoid the generation of moiré fringes, it does not rule out the situation where the user actively makes adjustments when he knows that moiré fringes have occurred. That is, when the discrimination unit detects the presence of moiré fringes, the photographing processing unit prompts the user and instructs the user to move the photographing device toward / away from the photographing object to avoid the occurrence of moiré fringes.

[0075] Description of reference numerals:

[0076] 100, 100′…shooting device, 110…first camera unit, 110′…camera unit, 111…first optical system, 111′…optical system, 111a…focusing lens group, 111b…magnification changing lens group, 112…first driving mechanism, 112′…driving mechanism, 112a…focusing lens driving mechanism, 112b…magnification changing lens driving mechanism, 113…first shooting element, 114…first analog front end (AFE), 120…second camera unit, 121…second optical system, 122…second driving mechanism, 123…second shooting element, 124…second analog front end (AFE), 130 …display, 150…system control unit, 151…determination unit, 152…shooting processing unit, 141…working memory, 142…image processing unit, 160…operation unit, 161…shutter button, 162…touch panel, 170…storage unit, 180…communication interface, 301…moving image, 302…moire fringe area, 390…shutter button, 310…wide-angle image, 311…cropped image, 313…moire fringe corresponding area, 314…moire fringe area, 320…telephoto image, 323…moire fringe area, 324…moire fringe corresponding area, 325…notification display, 900…TV display, 901…TV image.

Claims

1. A photographing device, characterized in that: have: a camera unit having a photographing element; a determination unit that determines whether a moire fringe area exists in an image formed based on an output of the imaging element; as well as The imaging processing unit switches optical conditions related to imaging by the camera unit and continues imaging processing when the determination unit determines that the moire fringe area exists in the image.

2. The photographing device according to claim 1, wherein: The camera unit includes at least a first camera unit and a second camera unit, wherein the first camera unit has a first shooting element as the shooting element and a first optical system for forming an image of a subject on the first shooting element, and the second camera unit has a second shooting element as the shooting element and a second optical system different from the first optical system for forming an image of a subject on the second shooting element. The determination unit determines whether the moire fringe area exists in a moving image formed based on an output of an imaging element of the active unit selected from the first camera unit and the second camera unit. When the determination unit determines that the moire fringe area exists in the moving image, the imaging processing unit switches the moving unit from one of the first camera unit and the second camera unit to the other, and continues the imaging processing.

3. The photographing device according to claim 2, wherein: When the focal length of the first optical system is shorter than the focal length of the second optical system, the shooting processing unit generates a cut-out image of the image area corresponding to the angle of view of the second optical system when the active unit is switched from the second camera unit to the first camera unit, and the cut-out image is cut out from the first image that can be formed based on the output of the first shooting element.

4. The photographing device according to claim 2, wherein: The photographing processing unit performs notification related to angle of view adjustment to a user when the active unit is switched from the first camera unit to the second camera unit when the focal length of the first optical system is shorter than the focal length of the second optical system.

5. The photographing device according to claim 1, wherein: The camera unit has a variable focal length optical system that can change the focal length and form an image of the subject on the imaging element. The imaging processing unit changes the focal length of the variable focus optical system and continues imaging processing when the determination unit determines that the moire fringe region exists in the image.

6. The photographing device according to claim 5, wherein: When the variable focal length optical system is changed to a shorter focal length than the original focal length, the imaging processing unit generates a cutout image of an image area corresponding to the angle of view of the original focal length, the cutout image being cut out from an image formed based on the output of the imaging element.

7. The photographing device according to claim 5, wherein: The imaging processing unit notifies a user of angle of view adjustment when the variable focal length optical system is changed to a longer focal length side than an original focal length.

8. A shooting control program, characterized in that: Instruct the computer to perform the following steps: A discrimination step for discriminating whether a moire fringe area exists in an image formed based on the output of a photographing element possessed by a camera unit; and a photographing processing step for switching optical conditions related to photographing of the camera unit and continuing photographing processing when it is determined in the discrimination step that the moire fringe area exists in the image.

Images