Image pickup apparatus, control method thereof, storage medium, and computer program product
By identifying and controlling the display unit and hiding or displaying the focus frame, the problem of the photographer's focus difference between the focus display occlusion and the photographer in the evaluation mode is solved, and better operability and shooting effects are achieved.
Patent Information
- Application Number
- CN202410626741.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2024-05-20
- Publication Date
- 2024-11-26
Smart Images

Figure CN119031232A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a camera device and a control method thereof, a storage medium, and a computer program product. Background Art
[0002] In recent years, the number of opportunities and creators for shooting moving images has increased, and Vlog shooting that records daily life is gaining attention. Among them, there is also product review shooting in which a photographer holds a specific object (e.g., a product) and describes the characteristics and charm of the object, and some cameras are also equipped with a review shooting mode. When operating in the review shooting mode, the camera performs the following control: if the object to be introduced is positioned relative to the photographer, it focuses on the object, otherwise, it focuses on the person, thereby achieving improved quality of moving images.
[0003] As a similar prior art, Japanese Patent Laid-Open No. 2010-230871 proposes a technique for assisting a photographer by searching and displaying the next object to be focused on when photographing a large number of subjects. To this end, subject information is pre-registered in a camera. The camera includes a mode for displaying an AF frame and performing AF on a registered subject, and a face detection mode for displaying only an AF frame and at least one name frame of the subject and not performing AF frame tracking.
[0004] Japanese Patent Laid-Open No. 2016-142779 also discloses a camera that, when a user selects a subject on a screen displayed on a display unit of the camera, displays a defocus amount between a current focus position and a focus position of the selected subject, and displays the position of the current focus position on a defocus display.
[0005] Regarding the above-mentioned background technology, there are the following two problems when performing evaluation shooting. The first problem is that when the distance between the photographer and the monitor becomes relatively large, it becomes difficult for the photographer to check whether he or she (the photographer) or the object (the evaluation subject) is in focus.
[0006] The second problem is that although the photographer wants to check the photographer's expression and the appearance of the evaluation object, the visibility of the state of the subject is impaired because the frame is constantly displayed around the focused subject. It is necessary to solve the above problem and improve the coexistence of the focus display and the object visibility. In addition, in order to give priority to solving the second problem, many known product specifications do not perform the focus display during the evaluation mode. Summary of the invention
[0007] The present invention has been made in view of the above-mentioned problems, and aims to provide a technology for realizing comfortable operability in evaluation mode shooting.
[0008] According to one aspect of the present invention, there is provided a camera device, comprising:
[0009] a recognition unit configured to recognize a person as a main subject and recognize an object other than a person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of the subject; and
[0010] a control unit configured to control a display unit configured to display the image captured by the camera unit according to a result of the recognition performed by the recognition unit,
[0011] wherein, when the first mode is selected, the control unit displays on the display unit an image in which a focus display indicating a focused object is synthesized on the image obtained by the imaging unit, regardless of which of the main subject and the sub-subject is focused; and
[0012] When the second mode is selected, the control unit displays, on the display unit, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus.
[0013] According to another aspect of the present invention, a method for controlling a camera device is provided, the method comprising:
[0014] recognizing a person as a main subject and an object other than the person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of a subject; and
[0015] controlling a display unit configured to display the image captured by the camera unit according to a result of the recognition performed in the recognition,
[0016] Wherein, in the control,
[0017] When the first mode is selected, an image in which a focus display indicating a focused object is synthesized on an image obtained by the imaging unit is displayed on the display unit regardless of which of the main subject and the sub-subject is in focus; and
[0018] When the second mode is selected, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus is displayed on the display unit.
[0019] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, which stores a computer program, and the computer program is read out and executed by a computer of a camera device, so that the computer:
[0020] recognizing a person as a main subject and an object other than the person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of a subject; and
[0021] controlling a display unit configured to display the image captured by the camera unit according to a result of the recognition performed in the recognition,
[0022] Wherein, in the control of the display unit,
[0023] When the first mode is selected, an image in which a focus display indicating a focused object is synthesized on an image obtained by the imaging unit is displayed on the display unit regardless of which of the main subject and the sub-subject is in focus; and
[0024] When the second mode is selected, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus is displayed on the display unit.
[0025] According to another aspect of the present invention, there is provided a computer program product, which includes a computer program, and the computer program, when executed by a computer of a camera device, causes the computer to:
[0026] recognizing a person as a main subject and an object other than the person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of a subject; and
[0027] controlling a display unit configured to display the image captured by the camera unit according to a result of the recognition performed in the recognition,
[0028] Wherein, in the control of the display unit,
[0029] When the first mode is selected, an image in which a focus display indicating a focused object is synthesized on an image obtained by the imaging unit is displayed on the display unit regardless of which of the main subject and the sub-subject is in focus; and
[0030] When the second mode is selected, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus is displayed on the display unit.
[0031] Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings). BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1Ais a front perspective view showing the appearance of the configuration of the digital camera according to the first embodiment.
[0033] Figure 1B is a rear perspective view showing the appearance of the configuration of the digital camera according to the first embodiment.
[0034] Figure 2 is a block configuration diagram of a digital camera according to the first embodiment.
[0035] Figure 3 is a diagram showing an example of a setting screen displayed on the digital camera according to the first embodiment.
[0036] Figure 4A is the first part of the transition diagram of the display screen according to the first embodiment.
[0037] Figure 4B This is the latter part of the transition diagram of the display screen according to the first embodiment.
[0038] Figure 5 is a flowchart according to the first embodiment.
[0039] Figure 6 2 is a diagram showing transitions of display screens according to the second embodiment.
[0040] Figure 7 is a flowchart according to the second embodiment.
[0041] Figure 8 3 is a diagram showing transitions of display screens according to the third embodiment.
[0042] Fig. 9 is a flowchart according to the third embodiment. DETAILED DESCRIPTION
[0043] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the present invention. A plurality of features are described in the embodiments, but are not limited to the invention requiring all such features, and a plurality of such features may be appropriately combined. In addition, in the accompanying drawings, the same reference numerals are given to the same or similar configurations, and redundant descriptions thereof will be omitted.
[0044] Hereinafter, a preferred first embodiment of the present invention will be described with reference to the accompanying drawings. Figure 1A and Figure 1B An external view of a digital camera 100 capable of capturing and recording still images and moving images is shown as an example of an apparatus to which the present embodiment is applied. The digital camera 100 is an example of an imaging apparatus. Figure 1A is a front perspective view of the digital camera 100, and Figure 1B is a rear perspective view of the digital camera 100. Figure 1A and Figure 1B , the display unit 28 is a display unit provided on the back of the camera, and is used to display images and various types of information. The touch panel 70a can detect a touch operation on the display surface (operation surface) of the display unit 28. The viewfinder external display unit 43 is a display unit provided on the top surface of the camera, and displays various setting values of the camera, such as shutter speed and aperture, etc.
[0045] The shutter button 61 is an operation portion for instructing shooting. The mode changeover switch 60 is an operation portion for switching between various modes. The terminal cover 40 is a cover for protecting a connector (not shown) that connects a connection cable connected to an external device with the digital camera 100. The main electronic dial 71 is a rotation operation member included in the operation unit 70, and by rotating the main electronic dial 71, changes in setting values such as shutter speed and aperture, etc., can be made.
[0046] The power switch 72 is an operation member for turning on and off the power of the digital camera 100. The sub-electronic dial 73 is a rotation operation member included in the operation unit 70, and can move the selection frame and advance the image, etc. The cross key 74 is a cross key operation member (four-way key) included in the operation unit 70, and has a push button that can be pushed in four directions, such as up, down, left and right parts, etc. An operation can be performed according to the part of the cross key 74 that is pressed in the pressing direction.
[0047] The SET button 75 is included in the operation unit 70, is a push button, and is mainly used to determine the selection item, etc. The moving image button 76 is used to instruct the start and stop of moving image shooting (recording). The AE lock button 77 is included in the operation unit 70, and by pressing the AE lock button 77 in the shooting standby state, the exposure state can be fixed. The enlargement button 78 is included in the operation unit 70, and is an operation button for turning on and off the enlargement mode in the live view display in the shooting mode.
[0048] By operating the main electronic dial 71 after turning on the enlargement mode, the live view image can be enlarged or reduced. In the playback mode, the main electronic dial 71 functions as a zoom button for enlarging the playback image and increasing the magnification. The playback button 79 is included in the operation unit 70, and is an operation button for switching between the shooting mode and the playback mode. By pressing the playback button 79 during the shooting mode, the mode is shifted to the playback mode, and the latest image among the images recorded in the recording medium 200 can be displayed on the display unit 28.
[0049] The menu button 81 is included in the operation unit 70, and when pressed, displays a menu screen on which various settings can be made on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the cross key 74, the SET button 75, or the multi-controller (hereinafter referred to as "MC") 65. The MC 65 can accept direction instructions in eight directions and a push operation in the middle.
[0050] The communication terminal 10 is a communication terminal through which the digital camera 100 communicates with a lens unit 150 (attachable and detachable) to be described later. The eyepiece portion 16 is an eyepiece portion of an eyepiece viewfinder (peep-type viewfinder), and a user can view an image displayed on an internal electronic viewfinder (EVF) 29 through the eyepiece portion 16. The eye approach detection unit 57 is an eye approach detection sensor that detects whether the eye of the photographer has approached the eyepiece portion 16. The cover 202 is a cover for a slot for storing the recording medium 200.
[0051] The grip portion 90 is a holding portion shaped so as to be easily grasped with the right hand when the user holds the digital camera 100. The shutter button 61 and the main electronic dial 71 are arranged at such positions that the shutter button 61 and the main electronic dial 71 can be operated with the index finger of the right hand when the user holds the digital camera by gripping the grip portion 90 with the little finger, ring finger, and middle finger of the right hand. In addition, the sub-electronic dial 73 is arranged at a position that the user can operate with the thumb of the right hand in the same state.
[0052] Figure 2 1 is a block diagram showing a configuration example of a digital camera 100 according to the present embodiment. Figure 2 , the lens unit 150 is a lens unit equipped with an interchangeable photographic lens. Although the lens 103 is generally composed of a plurality of lenses, only one lens is shown here for simplicity. The communication terminal 6 is a communication terminal through which the lens unit 150 communicates with the digital camera 100. The lens unit 150 communicates with the system control unit 50 via the communication terminal 6 and the above-mentioned communication terminal 10, and controls the diaphragm 1 via the diaphragm drive circuit 2 using the internal lens system control circuit 4. Thereafter, the lens unit 150 performs focusing by shifting the lens 103 via the AF drive circuit 3 using the lens system control circuit 4.
[0053] The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50 .
[0054] The image pickup unit 22 is an image pickup element composed of a CCD or CMOS element, etc., which can convert an optical signal of an optical image captured through a lens into an electrical signal. The A / D converter 23 is used to convert an analog signal output from the image pickup unit 22 into a digital signal.
[0055] The image processing unit 24 performs predetermined resizing processing (such as pixel interpolation and reduction, etc.) and color conversion processing on the data from the A / D converter 23 or the data from the memory control unit 15 to be described later. In addition, the image processing unit 24 performs predetermined calculation processing using the captured image data. Based on the calculation results obtained by the image processing unit 24, the system control unit 50 performs exposure control and distance measurement control. Accordingly, automatic focus (AF) processing, automatic exposure (AE) processing, and flash pre-emission (EF) processing based on through-the-lens (TTL) are performed. The image processing unit 24 also performs predetermined calculation processing using the captured image data, and performs automatic white balance (AWB) processing based on TTL based on the obtained calculation results.
[0056] The memory control unit 15 controls data transmission and reception between the A / D converter 23, the image processing unit 24, and the memory 32. The output data from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15 or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, and image data to be displayed on the display unit 28 and the EVF 29. The storage capacity of the memory 32 is sufficient to store a predetermined number of still images and a predetermined amount of time of moving images and audio.
[0057] The memory 32 is also used as an image display memory (video memory). The display image data written to the memory 32 is displayed by the display unit 28 and the EVF 29 via the memory control unit 15. The display unit 28 and the EVF 29 display on a display such as an LCD or an organic EL according to a signal from the memory control unit 15. Live view display (LV display) can be performed by sequentially transmitting the data A / D converted by the A / D converter 23 and stored in the memory 32 to the display unit 28 or the EVF 29 and displaying it. Hereinafter, the image to be displayed in live view is referred to as a live view image (LV image).
[0058] The infrared light emitting diode 166 is a light emitting element for detecting the user's line of sight position within the viewfinder screen, and uses infrared light to illuminate the eyeball (eye) 161 of the user that has approached the eyepiece portion 16. The infrared light emitted from the infrared light emitting diode 166 is reflected by the eyeball (eye) 161, and the infrared reflected light reaches the dichroic mirror 162. The dichroic mirror 162 reflects only infrared light and allows visible light to pass through. The infrared reflected light whose optical path has been changed forms an image on the imaging surface of the line of sight detection sensor 164 via the imaging lens 163. The imaging lens 163 is an optical component that forms a line of sight detection optical system. The line of sight detection sensor 164 is composed of an imaging device such as a CCD type image sensor.
[0059] The sight line detection sensor 164 photoelectrically converts the incident infrared reflected light into an electrical signal, and outputs the resulting electrical signal to the sight line detection circuit 165. The sight line detection circuit 165 includes at least one processor, detects the user's sight line position from the image or movement of the user's eyeball (eye) 161 based on the output signal of the sight line detection sensor 164, and outputs the detected information to the system control unit 50. In this way, the sight line detection block 160 is composed of the dichroic mirror 162, the imaging lens 163, the sight line detection sensor 164, the infrared light emitting diode 166, and the sight line detection circuit 165.
[0060] The sight line detection block 160 of the present embodiment detects the sight line of the user using a method called corneal reflection method. The corneal reflection method is a method for detecting the direction and position of the sight line based on the positional relationship between the reflected light obtained by reflecting the infrared light emitted from the infrared light emitting diode 166 by the cornea of the eyeball (eye) 161 and the pupil of the eyeball (eye) 161. In addition, there are various methods for detecting the direction and position of the sight line, such as a method called sclera reflection method, which utilizes the difference in light reflectivity between the pupil and the white of the eye. It should be noted that any other method of the sight line detection unit other than the above may be used as long as the direction and position of the sight line can be detected.
[0061] Various setting values of the camera including shutter speed and aperture are displayed on the viewfinder external display unit 43 via the viewfinder external display unit driving circuit 44 .
[0062] The nonvolatile memory 56 is an electrically erasable / recordable memory such as a flash ROM, etc. The nonvolatile memory 56 stores constants and programs, etc. for the operation of the system control unit 50. The program here refers to a computer program for executing various flowcharts described later in this embodiment.
[0063] The system control unit 50 is a control unit composed of at least one processor or circuit, and performs overall control of the digital camera 100. The processor may be, for example, a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), a quantum processing unit (QPU), or a combination thereof.
[0064] The system control unit 50 executes the above-mentioned program recorded in the nonvolatile memory 56 to realize each process of the present embodiment which will be described later. For example, a RAM is used as the system memory 52, and constants and variables used for the operation of the system control unit 50 and the program and the like read out from the nonvolatile memory 56 are loaded into the system memory 52. The system control unit 50 also performs display control by controlling the memory 32 and the display unit 28 and the like.
[0065] The system control unit 50 switches the shooting mode based on the setting information set by the user, for example. For example, the shooting mode includes a normal mode in which an image synthesized with a focus display indicating a focused object (i.e., an object to be focused) is displayed regardless of a subject, and an evaluation mode in which an image is displayed without synthesizing a focus display if a person is focused, and an image synthesized with a focus display if an object (not a person) such as a product is focused. For example, the evaluation mode is a product introduction mode for introducing a product. The normal mode is an example of the first mode, and the evaluation mode is an example of the second mode.
[0066] The system timer 53 is a time measuring unit that measures time for various controls and time of a built-in clock.
[0067] The mode switching switch 60, the first shutter switch 62, the second shutter switch 64 and the operating unit 70 are operating units for inputting various operation instructions to the system control unit 50. The mode switching switch 60 switches the operation mode of the system control unit 50 to one of the still image shooting mode and the moving image shooting mode. The modes included in the still image shooting mode include an automatic shooting mode, an automatic scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode) and a program AE mode (P mode). In addition, there are various scene modes and a custom mode as shooting settings for each shooting scene. The mode switching switch 60 allows the user to directly switch to any one of these modes. Alternatively, after once switching to a screen showing a list of shooting modes using the mode switching switch 60, one of the displayed multiple modes can also be selected and switched to the one mode using another operating member. Similarly, the moving image shooting mode can also include multiple modes.
[0068] When the shutter button 61 provided on the digital camera 100 is partially operated or a so-called half-press (shooting preparation instruction) is performed, the first shutter switch 62 is turned on, and the first shutter switch 62 generates a first shutter switch signal SW1. According to the first shutter switch signal SW1, shooting preparation operations such as automatic focus (AF) processing, automatic exposure (AE) processing, automatic white balance (AWB) processing, and flash pre-emission (EF) processing are started.
[0069] When the operation of the shutter button 61 is completed or a so-called full press (shooting instruction) is performed, the second shutter switch 64 is turned on, and the second shutter switch 64 generates a second shutter switch signal SW2. Based on the second shutter switch signal SW2, the system control unit 50 starts a series of shooting processing operations from reading out a signal from the image pickup unit 22 to writing a shot image as an image file in the recording medium 200.
[0070] The operation unit 70 is various operation members used as an input unit for accepting operations from the user. The operation unit 70 includes at least the following operation units: a shutter button 61, a multi-controller 65, a touch panel 70a, a main electronic dial 71, a power switch 72, a sub-electronic dial 73, a cross key 74, a SET button 75, a moving image button 76, an AE lock button 77, a magnification button 78, a playback button 79, and a menu button 81.
[0071] The power control unit 80 is composed of a battery detection circuit, a DC-DC converter, and a switch circuit for switching a block through which a current is to flow, and detects whether a battery is attached, the battery type, and the remaining power. In addition, the power control unit 80 controls the DC-DC converter based on the detection result and the instruction from the system control unit 50, and supplies the necessary voltage to each unit including the recording medium 200 within a necessary period. The power unit 30 includes a primary battery (such as an alkaline battery or a lithium battery), a secondary battery (such as a nickel-cadmium battery, a nickel-hydrogen battery, or a lithium-ion battery), an AC adapter, and the like.
[0072] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium (such as a memory card) for recording captured images, and is composed of a semiconductor memory or a magnetic disk or the like.
[0073] The communication unit 54 is connected wirelessly or by a wired cable, and transmits and receives image signals and audio signals. The communication unit 54 can also be connected to a wireless LAN (local area network) and the Internet. In addition, the communication unit 54 can communicate with external devices using Bluetooth (a registered trademark) or Bluetooth low energy. The communication unit 54 can transmit images (including live view images) captured by the camera unit 22 and images recorded in the recording medium 200, and can also receive images and other various types of information from external devices.
[0074] The orientation detection unit 55 detects the orientation of the digital camera 100 with respect to the gravity direction. Based on the orientation detected by the orientation detection unit 55, it can be determined whether the image captured by the imaging unit 22 is an image captured with the digital camera 100 held horizontally or an image captured with the digital camera 100 held vertically. The system control unit 50 can add direction information corresponding to the orientation detected by the orientation detection unit 55 to the image file of the image captured by the imaging unit 22, or can rotate and record the image. As the orientation detection unit 55, an acceleration sensor or a gyro sensor or the like can be used. It is also possible to detect the movement (pan, tilt, rise and fall, whether or not, etc.) of the digital camera 100 using the acceleration sensor or the gyro sensor as the orientation detection unit 55.
[0075] The eye approach detection unit 57 is an eye approach detection sensor that detects (approach detection) the approach of the eye (object) 161 to the eyepiece portion 16 of the viewfinder (eye approach) or the separation of the eye (object) 161 from the eyepiece portion 16 of the viewfinder (eye separation). The system control unit 50 switches the display unit 28 and the EVF 29 between display (display state) and non-display (non-display state) according to the state detected by the eye approach detection unit 57. More specifically, if the digital camera 100 is at least in the shooting standby state and the setting for switching the display destination of the live view image captured by the imaging unit 22 is the automatic switching setting, during the separation of the eye from the viewfinder, the display is turned on with the display unit 28 as the display destination, and the EVF 29 is not displayed. In addition, during the eye approach, the display is turned on with the EVF 29 as the display destination, and the display unit 28 is not displayed. For example, the eye approach detection unit 57 may use an infrared proximity sensor, and may detect the approach of a certain object to the eyepiece portion 16 of the viewfinder including the EVF 29. When an object approaches, infrared light projected from a light projecting portion (not shown) of the eye proximity detection unit 57 is reflected and received by a light receiving portion (not shown) of the infrared proximity sensor. Based on the amount of infrared light received, the degree of proximity of the object to the eyepiece portion 16 (eye proximity distance) can also be determined.
[0076] In this way, the eye approach detection unit 57 performs eye approach detection to detect the proximity distance of the object to the eyepiece portion 16. Note that in the present embodiment, it is assumed that the light emitting portion and the light receiving portion of the eye approach detection unit 57 are devices separate from the above-mentioned infrared light emitting diode 166 and the line of sight detection sensor 164. However, the infrared light emitting diode 166 may also be used as a light projection unit of the eye approach detection unit 57. In addition, the line of sight detection sensor 164 may also be used as a light receiving portion. When an object approaching the eyepiece portion 16 at a predetermined distance or less is detected from a non-eye approach state (non-approach state), it is determined that eye approach has occurred. If it is detected that the approaching object moves away from the eyepiece by more than a predetermined distance from the eye approach state (approach state), it is determined that eye separation has occurred. For example, the threshold for detecting eye approach and the threshold for detecting eye separation may be different due to providing a hysteresis. In addition, after detecting eye approach, it is assumed that the eye approach state continues until eye separation is detected. After detecting eye separation, it is assumed that the non-eye approach state continues until eye approach is detected. Note that the infrared proximity sensor is one example, and other sensors may be used for the eye proximity detection unit 57 as long as the other sensors can detect the approach of an eye or an object that can be regarded as eye proximity.
[0077] The system control unit 50 can detect the following operations or states based on the output from the visual line detection block 160 .
[0078] - The user's eyes are detected approaching the eyepiece portion 16 for the first time (start of line of sight input).
[0079] - The user's eyes continue to be near the eyepiece portion 16 (user line of sight input state).
[0080] - The user looks into the eyepiece portion 16 .
[0081] - From the eyepiece portion 16, the user's eyes are changed from the approach state to the non-detection state (end of sight line input).
[0082] - The user's eyes are no longer detected at the eyepiece portion 16 (a state in which the user does not input any line of sight).
[0083] The “gaze” mentioned here means that the user's line of sight is within a preset range within a predetermined time period.
[0084] The touch panel 70a and the display unit 28 may be integrally formed. For example, the touch panel 70a is configured so that its light transmittance does not interfere with the display on the display unit 28, and the touch panel 70a is attached to the upper layer of the display surface of the display unit 28. In addition, the input coordinates on the touch panel 70a are associated with the display coordinates on the display screen of the display unit 28. As a result, a GUI (Graphical User Interface) that makes it look as if the user can directly operate the screen displayed on the display unit 28 can be provided. The system control unit 50 can detect the following operations or states on the touch panel 70a.
[0085] - A finger or pen that is not touching the touch panel 70a touches the touch panel 70a for the first time. That is, touching (hereinafter referred to as "touching") starts.
[0086] - Touch the touch panel 70a with a finger or a pen (hereinafter referred to as "touch hold").
[0087] - Moving a finger or a pen while touching the touch panel 70a with the finger or the pen (hereinafter referred to as "touch movement").
[0088] - Remove the finger or pen that is touching the touch panel 70a from the touch panel 70a. That is, end the touch (hereinafter referred to as "touch stop").
[0089] - No object is touching the touch panel 70a (hereinafter referred to as "no touch").
[0090] When a touch is detected, touch persist is also detected at the same time. After a touch, touch persist is usually detected until touch stop is detected. While touch persist is detected, touch move is also detected. Even if touch persist is detected, touch move is not detected unless the touch position moves. Untouch occurs after touch stop is detected for all fingers or pens that are touching.
[0091] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 70a, are notified to the system control unit 50 through the internal bus. The system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 70a based on the notified information. Regarding touch movement, the moving direction of the finger or pen moving on the touch panel 70a can also be determined for each of the vertical component and the horizontal component on the touch panel 70a based on the change in the position coordinates. If a touch movement of more than a predetermined distance is detected, it is determined that a slide operation has been performed.
[0092] An operation in which the user quickly moves a finger a certain distance while touching the touch panel and then removes the finger from the touch panel is called a flick. In other words, a flick is an operation in which the finger quickly traces on the touch panel 70a as if a flicking motion is performed. If a touch movement of more than a predetermined distance at a predetermined speed or more is detected and the touch stop is detected as it is, it can be determined that a flick has been performed (it can be determined that a flick has occurred after a slide operation).
[0093] In addition, a touch operation in which a user touches multiple locations (e.g., two points) at the same time and moves the touch locations closer to each other is called a pinch-in, and a touch operation in which a user moves the touch locations farther from each other is called a pinch-out. Pinch-out and pinch-in are collectively referred to as a pinch operation (or simply pinch).
[0094] The touch panel 70a may be any one of various types of touch panels such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type, etc. The types of touch panels include a type that detects a touch when the touch panel is touched and a type that detects a touch when a finger or a pen approaches the touch panel, and in the present embodiment, either method may be employed.
[0095] When a touch move operation is performed in the eye proximity state, the user can set the method for specifying the position of the position indicator corresponding to the touch move operation to absolute position specification or relative position specification. For example, if the position indicator is an AF frame and the absolute position specification is set, when the touch panel 70a is touched, the AF position associated with the touched position (coordinate input position) is set. That is, the position coordinates at which the touch operation is performed are associated with the position coordinates of the display unit 28. On the other hand, if the relative position specification is set, the position coordinates at which the touch operation is performed are not associated with the position coordinates of the display unit 28. In the relative position specification, the touch position moves from the currently set AF position along the moving direction of the touch move by a distance corresponding to the moving amount of the touch move, regardless of the touched position of the touch panel 70a.
[0096] Figure 3 : is a diagram showing a menu screen for changing settings displayed on the display unit 28 of the camera of the present embodiment.
[0097] The menu screen is roughly divided into areas 300 to 303. The user selects one of the menus by touch input or by operating the up / down / left / right keys and the select key.
[0098] The area 300 includes a selection tab related to general moving image shooting (Rec), a tab related to auto focus (AF), and the like, and the figure shows that general moving image shooting has been selected.
[0099] Area 301 is an area for selecting a mode during moving image shooting. In the present embodiment, in addition to the general P / Tv / Av / M / AUTO modes of a digital camera, an evaluation mode is added as a product evaluation mode.
[0100] Area 302 is a setting for turning on and off the product evaluation mode. This setting allows the user to set the evaluation mode. Here, two types of setting items (i.e., area 301 and area 302) are displayed, but only one of the setting items needs to be displayed.
[0101] Area 303 is a display setting for secondary subject tracking in product evaluation mode. Figure 4A and Figure 4B The details are described, but settings A, B, and C can change the display method used when tracking is performed on the screen after the secondary subject is detected. The information about the normal mode, evaluation mode, and settings A to C described above are examples of setting information.
[0102] The system control unit 50 stores the settings of the above-described areas 301 to 303 in the nonvolatile memory 56 or the like, and the system control unit 50 controls display according to the settings.
[0103] Figure 4A and Figure 4B 2 is a state transition diagram of the display on the display unit 28 of the camera according to the first embodiment. Hereinafter, the state transition will be described with reference to the same drawing.
[0104] Note that for ease of understanding, an example will be described in which a user as a photographer records an image for introducing a product when shooting in the evaluation mode. Here, the user is both a person describing the product and a photographer shooting the product. The product is a non-person object. Hereinafter, the user will be described as the main subject, and the product will be described as the secondary subject.
[0105] State 411 is also a screen showing a state in which the photographer 401 is in focus. If the system control unit 50 does not recognize anything other than the photographer 401, the photographer 401 is basically kept in focus. Although the system control unit 50 is focused on the area 402, a frame indicating the area 402 is not displayed on the screen of the display unit 28. Note that in the normal mode, the system control unit 50 displays an image by synthesizing a frame surrounding at least a portion of the focused object as a focused display, regardless of whether the photographer 401 or the evaluation object is in focus. When the system control unit 50 recognizes the evaluation object, the system control unit 50 transitions to state 412. For example, the system control unit 50 recognizes an object registered in advance in the nonvolatile memory 56 of the digital camera 100 by the user as the evaluation object. Note that when the system control unit 50 recognizes an object other than a person, the system control unit 50 may recognize the recognized object as the evaluation object, or may recognize the evaluation object by a gesture of the person (such as pointing to the object).
[0106] State 412 is a state in which the system control unit 50 focuses on the evaluation object 403 in response to detecting the evaluation object 403 in a stationary state. In other words, the system control unit 50 recognizes the photographer 401 and the evaluation object 403, and generates an image based on the recognition result. At this time, the system control unit 50 focuses on the dotted area 404, but does not display the dotted area 404. Instead, in order to make it easier for the user to understand intuitively, the system control unit 50 displays the image by synthesizing a frame 405 including the evaluation object 403 (i.e., the focus object being focused) as a focus display. Note that although an example in which the frame 405 includes the entire evaluation object 403 in state 413 is shown, the system control unit 50 may also display a frame 405 surrounding at least a portion (e.g., the center) of the evaluation object 403 as a focus display. In addition, if the system control unit 50 detects movement of the evaluation object 403 , the system control unit 50 tracks the evaluation object 403 , transitions from state 412 to state 413 , and continues displaying while moving the frame 405 according to the movement of the evaluation object 403 .
[0107] States 413 to 415 are movement states when the evaluation object 403 moves within the angle of view. First, in state 413, the system control unit 50 tracks the evaluation object 403 within the screen using the same expression as in state 412, and causes the frame 405 to follow the evaluation object 403. Next, in state 414, the system control unit 50 hides the display of the frame 405 while the evaluation object 403 is moving. By hiding the frame 405, the system control unit 50 can reduce the display overlap with the evaluation object 403, and the user can easily check the subject including the evaluation object 403. In state 415, the system control unit 50 expresses that the moving evaluation object 403 is being tracked and focused by displaying the shooting information as the icon 409, instead of hiding the fact that the moving evaluation object 403 is in focus. As described above, the system control unit 50 can also detect the movement of the evaluation object 403 and design a frame expression indicating that the moving evaluation object 403 is in focus. When the system control unit 50 cannot see the evaluation object 403 , the process proceeds to state 416 .
[0108] State 416 is a state in which the system control unit 50 cannot see the evaluation object 403 and focuses on the photographer 401. This state 416 has the same expression as state 411, and the frame 405 and icon 409 indicating that the evaluation object 403 is in focus are not displayed.
[0109] State 417 is a state in which the system control unit 50 cannot detect the evaluation object 406 and gives priority to the nearest object to focus on the entire screen. The area in focus is area 407, and at this time, the system control unit 50 does not display a frame indicating that the evaluation object 406 has been detected, but instead displays an expression 408 indicating that the focus is being adjusted by giving priority to the nearest object on the periphery of the screen. When the evaluation object 406 moves within the screen, the system control unit 50 enters state 418.
[0110] State 418 is a state when the evaluation object 406 moves within the field of view. At this time, the system control unit 50 updates the area to be focused according to the movement of the evaluation object 406, but there is no change in the frame expression, etc. When the system control unit 50 cannot see the evaluation object 406, the process enters state 416.
[0111] The transition states in the present embodiment have been described above.
[0112] Next, we will refer to Figure 5 The flowchart of the first embodiment shown in FIG. 1 is used to describe the processing contents of the system control unit 50 in this embodiment. Each step is realized by the system control unit 50 loading the program stored in the nonvolatile memory 56 into the system memory 52 and executing the program.
[0113] In step S501, the system control unit 50 determines whether the currently set mode is the evaluation mode. If the system control unit 50 determines that it is currently operating in the evaluation mode, the system control unit 50 advances the process to step S502, otherwise, the system control unit 50 ends the process.
[0114] In step S502, the system control unit 50 hides the frame indicating the photographer. Note that the system control unit 50 hides the frame and continues to focus on the photographer.
[0115] In step S503, the system control unit 50 determines whether the secondary subject as the evaluation object has appeared in the captured image. If the system control unit 50 determines that the evaluation object has appeared in the captured image, the process proceeds to step S504, otherwise, the process proceeds to step S505.
[0116] In step S504, the system control unit 50 focuses on the evaluation object and displays a frame on the evaluation object (state 412). Note that if the system control unit 50 cannot recognize the evaluation object but determines that an object other than the photographer is present and closer than the photographer, the system control unit 50 notifies the user by displaying another frame (state 417).
[0117] In step S505, the system control unit 50 determines whether the secondary subject as the evaluation object has moved within the screen. If the system control unit 50 determines that the evaluation object has moved, the process proceeds to step S506, and if the system control unit 50 determines that the evaluation object has not moved, the process proceeds to step S510. The system control unit 50 also proceeds to step S510 when the evaluation object is not detected in step S503.
[0118] In step S506, the system control unit 50 determines the setting of the frame display specification when the evaluation object moves. If setting A to track the frame also during movement is set, the system control unit 50 proceeds to step S507; if setting B to hide the frame is set, the system control unit 50 proceeds to step S508; if setting C to change the expression of the frame is set, the system control unit 50 proceeds to step S509.
[0119] In step S507 , the system control unit 50 tracks the frame displayed in S504 according to the evaluation object (state 413 ).
[0120] In step S508 , the system control unit 50 hides the frame displayed in step S504 (state 414 ).
[0121] In step S509 , the system control unit 50 expresses that the evaluation object is in focus using an expression different from the expression of the frame displayed in step S504 (state 415 ).
[0122] In step S510, the system control unit 50 determines whether the secondary subject as the evaluation object has been unable to be detected. If the system control unit 50 determines that the evaluation object has been unable to be detected, the process proceeds to step S511, and if the evaluation object is still detected, the process proceeds to step S512. The system control unit 50 also proceeds to step S512 when the evaluation object is not detected in step S503.
[0123] In step S511 , the system control unit 50 hides the frame indicating that the evaluation object is in focus, and focuses on the photographer.
[0124] In step S512, the system control unit 50 determines whether the evaluation mode has ended. When the evaluation mode ends, the system control unit 50 ends the processing, and if the evaluation mode continues, the processing of step S502 and thereafter is performed again.
[0125] As described above, in the present embodiment, as shown in the state transition and the flowchart based on the first embodiment, the frame is displayed when the evaluation object is focused in the evaluation mode for evaluating a product, but the frame is not displayed when the photographer is focused, so that comfortable evaluation mode operability can be achieved.
[0126] Note that when shooting the review, the user shoots and records a video introducing the product, but when recording a video introducing the new product, it is naturally necessary to register the new product as a secondary subject in advance. If the product registration operation and the review mode operation are combined into a series of operations, the convenience is increased. In view of this, when the system control unit 50 determines as "yes" in step S501, the system control unit 50 can prompt the user to shoot from multiple preset viewpoints in order to correctly identify the product to be introduced, and then the processing for shooting and registration can be performed. After performing the registration operation, the system control unit 50 can enter the processing of step S502 and thereafter.
[0127] Figure 6 611 to 613 are diagrams showing state transitions in the second embodiment.
[0128] State 611 is a state in which the photographer 601 is in focus. Unless anything other than the photographer 601 is detected, the system control unit 50 basically continues to focus on the photographer 601. At this time, the system control unit 50 focuses on the area 602, and does not display the frame on the screen of the display unit 28. When the evaluation object appears, the system control unit 50 enters state 612 or state 613. At this time, the system control unit 50 switches the focus display based on the relationship in the front-to-back direction between the photographer 601 and the evaluation object. For example, as viewed from the digital camera 100, when the evaluation object is located in front of the photographer 601 (near side), the system control unit 50 enters state 612, and when the evaluation object is located behind the photographer (infinite side), the system control unit 50 enters state 613.
[0129] State 612 is a state in which the evaluation object 603 is detected and the evaluation object located on the near side relative to the photographer 601 (i.e., the subject previously tracked) is in focus. At this time, the area to be focused according to the position of the evaluation object 603 is area 604, and the system control unit 50 displays a frame 605 indicating the position of the evaluation object 603. When the evaluation object 603 moves within the screen, the system control unit 50 causes the same Figure 4A and Figure 4B A similar transition is described in detail below with respect to the transition from state 413 to state 415 in FIG.
[0130] State 613 is a state in which the evaluation object 606 is detected and the evaluation object 606 located at the infinite side relative to the photographer 601 (i.e., the subject previously tracked) is in focus. At this time, the area to be focused according to the position of the evaluation object 606 is area 607, and the system control unit 50 displays a frame 608 indicating the position of the evaluation object 606. When the evaluation object 606 moves within the screen, the system control unit 50 causes the same Figure 4A and Figure 4B A similar transition is described in detail below with respect to the transition from state 413 to state 415 in FIG.
[0131] As described above, the system control unit 50 can also change the frame expression according to whether the evaluation object is located on the near side or the infinite side relative to the photographer as viewed from the camera.
[0132] Figure 7 501 to S512 are flowcharts showing the processing of the second embodiment. Each process is realized by the system control unit 50 loading the program stored in the nonvolatile memory 56 into the system memory 52 and executing the program. Regarding the following flowchart, the processing and control are performed by the system control unit 50. Note that steps S501 to S512 are similar to those of Figure 5 Steps S501 to S512 are the same, so their description is omitted.
[0133] In step S701, the system control unit 50 determines the positional relationship between the sub-subject as the evaluation object detected in step S503 and the photographer, and if the sub-subject as the evaluation object is located in front of the photographer (on the near side relative to the photographer), the system control unit 50 proceeds to step S702, and if the sub-subject as the evaluation object is located behind the photographer (on the infinitely far side relative to the photographer), the system control unit 50 proceeds to step S703.
[0134] In step S702, the system control unit 50 performs a near-side frame display on the evaluation object (state 612). At this time, the secondary subject as the evaluation object continues to be focused.
[0135] In step S703, the evaluation object is displayed with an infinite side frame (state 613). At this time, the secondary subject of the evaluation object continues to be focused.
[0136] As described above, using the state transition and flowchart according to the second embodiment, improvement in operability is achieved by designing the frame expression in the product evaluation mode.
[0137] Figure 8 is a diagram showing the state transition in the third embodiment. This will use Figure 8 States 811 to 813 are shown for description.
[0138] In state 811, the photographer 801 is in focus. If no other object is detected except the photographer 801, the system control unit 50 basically continues to focus on the photographer 801. The area in focus at this time is the area 802, and the system control unit 50 does not display a frame on the screen of the display unit 28. When the evaluation object appears, the system control unit 50 enters state 812.
[0139] State 812 is a state in which the evaluation object 803 is detected and the evaluation object 803 is in focus. At this time, the area to be focused according to the position of the evaluation object 803 is area 804. The difference from the first embodiment and the second embodiment is that the system control unit 50 uses a unit other than the frame expression to notify the user that the evaluation object 803 is in focus. State 812 is the moment when the system control unit 50 detects the evaluation object 803, and has the same angle of view (display 805) as state 811.
[0140] State 813 is a state when a predetermined amount of time has passed since the evaluation object 803 was detected. At this time, the area to be focused according to the position of the evaluation object 803 is area 804. When the system control unit 50 detects the evaluation object 803, it gradually performs zoom processing toward the evaluation object 803 (display 806). By doing so, the photographer can check that the evaluation object 803 is in focus without checking the frame expression. Although the figure shows an example in which the system control unit 50 zooms in the zoom-in direction, the system control unit 50 may also zoom in the zoom-out direction according to the size of the secondary subject that is the evaluation object 803.
[0141] As described above, when the system control unit 50 detects the evaluation object and focuses on the evaluation object, the system control unit 50 automatically performs a zoom operation of the camera according to the evaluation object and notifies the photographer that the evaluation object is in focus.
[0142] Fig. 9 501 to 512 are flowcharts showing the processing of the third embodiment. Each process is realized by the system control unit 50 loading the program stored in the nonvolatile memory 56 into the system memory 52 and executing the program. Regarding the following flowchart, the processing and control are performed by the system control unit 50. Note that steps S501 to S512 are the same as those of Figure 5 Steps S501 to S512 are the same, so their description is omitted.
[0143] In step S901, the system control unit 50 zooms the secondary subject as the evaluation object detected in step S503. The system control unit 50 zooms in the optimal direction (zoom in / zoom out) so that the evaluation object falls within the angle of view. In addition, since the user sets the direction in advance, the system control unit 50 can improve the quality of the moving image.
[0144] In step S902, the system control unit 50 zooms to the original angle of view, in contrast to the zooming in step S901. At this time, the system control unit 50 may also control so as to zoom in on the photographer.
[0145] As described above, using the state transition and the flowchart according to the third embodiment, the system control unit 50 achieves improvement in operability by designing the frame expression in the product evaluation mode.
[0146] Other embodiments
[0147] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.
[0148] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
Claims
1. A camera device, comprising: a recognition unit configured to recognize a person as a main subject and recognize an object other than a person as a secondary subject from an image obtained by an imaging unit configured to capture an image of the subject; as well as a control unit configured to control a display unit configured to display the image captured by the camera unit according to a result of the recognition performed by the recognition unit, wherein, when the first mode is selected, the control unit displays on the display unit an image in which a focus display indicating a focused object is synthesized on the image obtained by the imaging unit, regardless of which of the main subject and the sub-subject is focused; and When the second mode is selected, the control unit displays, on the display unit, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus.
2. The imaging device according to claim 1, in, The control unit does not synthesize the focus display when the secondary subject is moving.
3. The imaging device according to claim 1, in, The control unit displays an image synthesized with a first focus display on the display unit while the secondary subject is stationary, and displays an image synthesized with a second focus display different from the first focus display on the display unit while the secondary subject is moving.
4. The imaging device according to claim 1, in, The control unit switches the focus display based on a relationship between the main subject and the sub-subject in a front-rear direction.
5. The imaging device according to claim 4, in, The focus display is a frame, and when the sub-subject is located on a far side relative to the main subject, the control unit displays the focus display with a larger frame than when the sub-subject is located on a near side relative to the main subject.
6. The imaging device according to claim 1, in, The control unit displays, on the display unit, an image obtained by zooming the secondary subject as the focus display, when the secondary subject is in focus.
7. A method for controlling a camera device, the method comprising: identifying a person as a main subject and an object other than the person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of the subject; as well as controlling a display unit configured to display the image captured by the camera unit according to a result of the recognition performed in the recognition, Wherein, in the control, When the first mode is selected, an image in which a focus display indicating a focused object is synthesized on an image obtained by the imaging unit is displayed on the display unit regardless of which of the main subject and the sub-subject is in focus; and When the second mode is selected, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus is displayed on the display unit.
8. A non-transitory computer-readable storage medium storing a computer program, wherein the computer program is read out and executed by a computer of an image pickup device, causing the computer to: recognizing a person as a main subject and an object other than the person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of a subject; and controlling a display unit configured to display the image captured by the camera unit according to a result of the recognition performed in the recognition, in, In the control of the display unit, When the first mode is selected, an image in which a focus display indicating a focused object is synthesized on an image obtained by the imaging unit is displayed on the display unit regardless of which of the main subject and the sub-subject is in focus; and When the second mode is selected, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus is displayed on the display unit.
9. A computer program product, comprising a computer program, wherein the computer program, when executed by a computer of a camera device, causes the computer to: recognizing a person as a main subject and an object other than the person as a sub-subject from an image obtained by an image pickup unit configured to pick up an image of a subject; and controlling a display unit configured to display the image captured by the camera unit according to a result of the recognition performed in the recognition, in, In the control of the display unit, When the first mode is selected, an image in which a focus display indicating a focused object is synthesized on an image obtained by the imaging unit is displayed on the display unit regardless of which of the main subject and the sub-subject is in focus; and When the second mode is selected, an image in which the focus display is not synthesized when the main subject is in focus, and an image in which the focus display is synthesized when the sub-subject is in focus is displayed on the display unit.
