Image capture method, image capture device, and program

Abstract

An image capture method according to the present disclosure comprises: an image capture step for generating image data by capturing an image via an image capture lens; a detection step for detecting a first subject and a second subject from the image data; a first focusing step for focusing on the first subject; and an adjustment step for performing image capture adjustment in the image capture step on the basis of the state of the second subject.

Description

Photographing method, photographing device, and program

[0001] The technology of the present disclosure relates to a photographing method, a photographing device, and a program.

[0002] Japanese Patent Laid-Open Publication No. 2021-125735 discloses an imaging control device having a detection means capable of detecting multiple types of subjects including a first type and a second type from a captured image, a switching means for switching between a type of subject for which a predetermined process is performed and a second type of subject for which a predetermined process is performed, a selection means for selecting a subject from multiple subjects of the second type in the detected captured image, and a control means. When the switching means has switched the type of subject for which the predetermined process is performed to the first type, the control means controls the display of a first subject of the first type in a first display form and a second subject of the second type in a second display form, and controls the display of the second subject in the first display form in response to the switching means switching the type of subject for which the predetermined process is performed from the first type to the second type.

[0003] International Publication No. 2020 / 080037 discloses an imaging device including an imaging unit, a display unit, first and second detection units, and a control unit. The imaging unit captures an image of a subject to generate a captured image. The display unit displays the captured image. The first detection unit detects at least a portion of the person if the subject is a person. The second detection unit detects at least a portion of the animal if the subject is an animal. The control unit controls the display unit to display a first detection frame corresponding to the person and a second detection frame corresponding to the animal on the captured image. The control unit controls the display unit so that the first detection frame and the second detection frame are displayed in a common display mode when the third detection frame corresponding to a main subject of the subjects is not used.

[0004] One embodiment of the technique of the present disclosure provides a photographing method, a photographing device, and a program that enable improvement in the accuracy of photographing adjustment.

[0005] In order to achieve the above-mentioned object, the photographing method of the present disclosure includes a photographing process for generating image data by photographing through a photographing lens, a detection process for detecting a first subject and a second subject from the image data, a first focusing process for focusing on the first subject, and an adjustment process for adjusting the photographing in the photographing process based on the state of the second subject.

[0006] The condition is brightness, and the adjustment step preferably involves adjusting exposure.

[0007] The adjustment step preferably includes a determination step of determining whether the shooting environment is a specific shooting environment based on the brightness of the second subject, and the adjustment step preferably adjusts the exposure based on the determination result of the determination step.

[0008] In the adjustment step, it is preferable to adjust the exposure by giving priority to the brightness of the second object rather than the brightness of the first object.

[0009] In the adjustment step, it is preferable to adjust the exposure so that the difference between the brightness of the first subject and the brightness of the second subject after the exposure adjustment falls within a predetermined range.

[0010] In the adjustment step, the color tone of the image data may be adjusted.

[0011] In the detection step, it is preferable to detect the first object and the second object using a machine-learned model.

[0012] It is preferable that the first subject and the second subject are different types of subjects.

[0013] Preferably, the second subject is a human being and the first subject is a non-human subject.

[0014] It is preferable that the method further includes a second focusing step for focusing on the second subject, and a selection step for selecting the first focusing step or the second focusing step, and that in the adjustment step, regardless of whether the first focusing step or the second focusing step is selected, the photographing adjustment is performed based on the state of the second subject.

[0015] The photographing device of the present disclosure is a photographing device equipped with a processor, which includes a photographing process that generates image data by photographing through a photographing lens, a detection process that detects a first subject and a second subject from the image data, a first focusing process that focuses the photographing lens on the first subject, and an adjustment process that adjusts the photographing process based on the state of the second subject.

[0016] The program disclosed herein causes a computer to execute a photographing process that generates image data by photographing through a photographing lens, a detection process that detects a first subject and a second subject from the image data, a first focusing process that focuses on the first subject, and an adjustment process that adjusts the photographing process based on the state of the second subject.

[0017] 1 is a diagram illustrating an example of the configuration of an imaging device; FIG. 2 is a diagram illustrating an example of a light receiving surface of an imaging sensor; FIG. 3 is a block diagram illustrating an example of the functional configuration of a processor; FIG. 4 is a diagram conceptually illustrating an example of processing by a machine-learned model; FIG. 5 is a diagram conceptually illustrating an example of processing by a distance measuring unit; FIG. 6 is a diagram conceptually illustrating an example of processing by a photometry unit; FIG. 7 is a diagram conceptually illustrating an example of processing when a second subject is selected as an AF target and an AE target; FIG. 8 is a flowchart illustrating an example of imaging operation by the imaging device; FIG. 9 is a diagram conceptually illustrating photometry processing according to a modified example; FIG. 10 is a flowchart illustrating adjustment processing according to a modified example; FIG. 11 is a block diagram illustrating the functional configuration of a processor according to a modified example; FIG. 12 is a flowchart illustrating an example of imaging operation by the imaging device according to a modified example; FIG. 13 is a flowchart illustrating an example of determination processing by a backlight determination unit;

[0018] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following description, "AF" is an abbreviation for "Auto Focus." "MF" is an abbreviation for "Manual Focus." "AE" is an abbreviation for "Auto Exposure." "IC" is an abbreviation for "Integrated Circuit." "CPU" is an abbreviation for "Central Processing Unit." "ROM" is an abbreviation for "Read Only Memory." "RAM" is an abbreviation for "Random Access Memory." "CMOS" is an abbreviation for "Complementary Metal Oxide Semiconductor."

[0021] "FPGA" is an abbreviation for "Field Programmable Gate Array." "PLD" is an abbreviation for "Programmable Logic Device." "ASIC" is an abbreviation for "Application Specific Integrated Circuit." "OVF" is an abbreviation for "Optical View Finder." "EVF" is an abbreviation for "Electronic View Finder."

[0022] The technology of the present disclosure will be described using an interchangeable lens digital camera as an example of an embodiment of an imaging device. Note that the technology of the present disclosure is not limited to interchangeable lens digital cameras, and can also be applied to digital cameras with an integrated lens.

[0023] 1 shows an example of the configuration of an image capturing device 10. The image capturing device 10 is a digital camera with an interchangeable lens. The image capturing device 10 is composed of a main body 11 and a photographing lens 12 that is interchangeably attached to the main body 11 and includes a focus lens 31. The photographing lens 12 is attached to the front side of the main body 11 via a camera-side mount 11A and a lens-side mount 12A.

[0024] The main body 11 is provided with an operation unit 13 including a dial, a release button, etc. The operation modes of the photographing device 10 include, for example, a still image photographing mode, a video photographing mode, and an image display mode. The operation unit 13 is operated by the user when setting the operation mode. The operation unit 13 is also operated by the user when starting still image photographing or video photographing.

[0025] The operation unit 13 is also operated by the user when selecting a focus mode. The focus modes include AF mode and MF mode. AF mode is a mode in which focus control is performed on an AF area within the angle of view. The user can set the AF area using the operation unit 13. MF mode is a mode in which the user manually controls focus by operating a focus ring (not shown). Note that automatic exposure (AE) control is performed in AF mode. The photographing device 10 may also be configured so that the user can set the AF area via a display 15 having a touch panel function or a viewfinder 14 having a line of sight detection function.

[0026] The photographing device 10 also has an automatic subject detection mode that automatically detects multiple subjects within the angle of view. For example, when the automatic subject detection mode is set in the AF mode, the subject that is closest to the currently set AF area among the multiple detected subjects is selected as the AF target.

[0027] The main body 11 is also provided with a viewfinder 14. The viewfinder 14 is a hybrid viewfinder (registered trademark). A hybrid viewfinder is a viewfinder that selectively uses, for example, an optical viewfinder (hereinafter referred to as "OVF") and an electronic viewfinder (hereinafter referred to as "EVF"). A user can observe an optical image or a live view image of a subject displayed by the viewfinder 14 through a viewfinder eyepiece (not shown).

[0028] A display 15 is provided on the rear side of the main body 11. Images based on image data obtained by shooting, various menu screens, etc. are displayed on the display 15. The user can also observe a live view image displayed on the display 15 instead of the viewfinder 14.

[0029] The main body 11 and the photographic lens 12 are electrically connected by electrical contacts 11B provided on the camera-side mount 11A coming into contact with electrical contacts 12B provided on the lens-side mount 12A.

[0030] The photographing lens 12 includes an objective lens 30, a focus lens 31, a rear end lens 32, and an aperture 33. The components are arranged along the optical axis A of the photographing lens 12 in the following order from the objective side: objective lens 30, aperture 33, focus lens 31, and rear end lens 32. The objective lens 30, focus lens 31, and rear end lens 32 constitute an optical system. The type, number, and arrangement order of the lenses that make up the optical system are not limited to the example shown in FIG. 1 .

[0031] The photographing lens 12 also has a lens drive control unit 34. The lens drive control unit 34 is configured with, for example, a CPU, RAM, ROM, etc. The lens drive control unit 34 is electrically connected to a processor 40 in the main body 11.

[0032] The lens drive control unit 34 drives the focus lens 31 and the diaphragm 33 based on a control signal transmitted from the processor 40. The lens drive control unit 34 controls the drive of the focus lens 31 based on a control signal for focus control transmitted from the processor 40 in order to adjust the position of the focus lens 31.

[0033] The diaphragm 33 has an aperture whose diameter is variable around the optical axis A. The lens drive control unit 34 controls the drive of the diaphragm 33 based on an exposure adjustment control signal sent from the processor 40 in order to adjust the amount of light incident on the light receiving surface 20A of the image sensor 20.

[0034] The main body 11 also contains an image sensor 20, a processor 40, and a memory 42. The operations of the image sensor 20, the memory 42, the operation unit 13, the viewfinder 14, and the display 15 are controlled by the processor 40.

[0035] The processor 40 is configured with, for example, a CPU, RAM, ROM, etc. In this case, the processor 40 executes various processes based on a program 43 stored in the memory 42. The processor 40 may be configured with a collection of multiple IC chips. The memory 42 also stores a machine-learned model LM that has undergone machine learning for detecting a subject.

[0036] The image sensor 20 is, for example, a CMOS image sensor. The image sensor 20 is positioned so that its optical axis A is perpendicular to the light-receiving surface 20A and is located at the center of the light-receiving surface 20A. Light (subject image) that has passed through the photographing lens 12 is incident on the light-receiving surface 20A. A plurality of pixels are formed on the light-receiving surface 20A, which generate image signals by performing photoelectric conversion. The image sensor 20 photoelectrically converts the light incident on each pixel to generate and output image data PD including the image signals.

[0037] A color filter array in a Bayer pattern is arranged on the light receiving surface 20A of the image sensor 20, with a color filter of either R (red), G (green), or B (blue) arranged opposite each pixel. Note that some of the pixels arranged on the light receiving surface of the image sensor 20 are phase difference detection pixels that output phase difference detection signals for focus control.

[0038] FIG. 2 shows an example of the light receiving surface 20A of the image sensor 20. A plurality of imaging pixels 21 and a plurality of phase difference detection pixels 22 are arranged on the light receiving surface 20A. The imaging pixels 21 are pixels on which the above-described color filters are arranged. The imaging pixels 21 receive light beams that pass through the entire area of ​​the exit pupil of the imaging optical system. The phase difference detection pixels 22 receive light beams that pass through half of the area of ​​the exit pupil of the imaging optical system. In the example shown in FIG. 2 , some of the diagonally arranged G pixels in the Bayer array are replaced with phase difference detection pixels 22. The phase difference detection pixels 22 are arranged at regular intervals in the vertical and horizontal directions on the light receiving surface 20A. The phase difference detection pixels 22 are divided into first phase difference detection pixels that receive light beams that pass through half of the area of ​​the exit pupil and second phase difference detection pixels that receive light beams that pass through the other half of the area of ​​the exit pupil.

[0039] The imaging pixels 21 output imaging signals for generating an image of a subject. The phase difference detection pixels 22 output phase difference detection signals. The image data PD output from the imaging sensor 20 includes the imaging signals and the phase difference detection signals.

[0040] FIG. 3 shows an example of the functional configuration of the processor 40. The processor 40 executes processes in accordance with a program 43 stored in the memory 42 to realize various functional units. As shown in FIG. 3, for example, the processor 40 realizes a main control unit 50, an imaging control unit 51, an image processing unit 52, a display control unit 53, an image recording unit 54, and a detection unit 55. In this embodiment, the detection unit 55 includes a subject detection unit 56, a distance measurement unit 57, and a photometry unit 58. The subject detection unit 56 operates when the automatic subject detection mode is set. The distance measurement unit 57 and the photometry unit 58 operate when the AF mode is set.

[0041] The main control unit 50 performs overall control of the operation of the imaging device 10 based on instruction signals input from the operation unit 13. The imaging control unit 51 controls the imaging sensor 20 to perform imaging processing that causes the imaging sensor 20 to generate image data PD. The imaging control unit 51 drives the imaging sensor 20 in a still image shooting mode or a video shooting mode. The imaging sensor 20 outputs image data PD generated by capturing an image via the imaging lens 12. The image data PD output from the imaging sensor 20 is supplied to the image processing unit 52 and the detection unit 55.

[0042] The image processing unit 52 acquires the image data PD output from the image sensor 20, and performs image processing on the image data PD, including white balance adjustment and gamma correction processing.

[0043] The display control unit 53 displays on the display 15 a live view image based on the image data PD that has been subjected to image processing by the image processing unit 52. The image recording unit 54 records the image data PD that has been subjected to image processing by the image processing unit 52 in the memory 42 as a recorded image PR when the release button is fully pressed.

[0044] The subject detection unit 56 reads the machine-learned model LM stored in the memory 42 and performs a detection process to detect all detectable subjects appearing in the image data PD using the machine-learned model LM. The machine-learned model LM is configured, for example, by a convolutional neural network.

[0045] The machine-learned model LM is generated by performing machine learning on a machine-learning model using a large amount of training data in the learning phase. The machine-learned model that has undergone machine learning in the learning phase is stored as the machine-learned model LM in the memory 42. Note that the learning process for the machine-learned model is performed, for example, by an external device.

[0046] The machine-learned model LM is not limited to being configured as software, but may be configured as hardware such as an IC chip, etc. Furthermore, the machine-learned model LM may be configured as an assembly of multiple IC chips.

[0047] The subjects detected by the subject detection unit 56 include humans, animals (dogs, cats, etc.), birds, trains, cars, etc. Note that the subjects include body parts such as faces and eyes. In the present disclosure, a subject other than a human or a body part thereof is referred to as a first subject, and a human or a body part thereof is referred to as a second subject. The first subject and the second subject are different types of subjects.

[0048] The distance measuring unit 57 selects the subject (first subject or second subject) closest to the currently set AF area as the AF target from among the multiple subjects detected by the subject detection unit 56. Note that if the AF area has not been set by the user, the distance measuring unit 57 selects the subject present near the center of the image data PD as the AF target. Furthermore, the distance measuring unit 57 may select the subject present near the center of the image data PD as the AF target regardless of the position of the AF area.

[0049] The distance measurement unit 57 detects a distance measurement value that indicates the distance from the image sensor 20 to the AF target. Specifically, the distance measurement unit 57 acquires a phase difference detection signal from an area corresponding to the AF target in the image data PD output from the image sensor 20, and outputs the distance calculated based on the acquired phase difference detection signal as a distance measurement value. The distance measurement value corresponds to a defocus amount that indicates the amount of deviation from the in-focus position of the focus lens 31.

[0050] The photometry unit 58 selects an AE target from the multiple subjects detected by the subject detection unit 56 and calculates a photometric value for exposure adjustment based on the brightness of the AE target. In principle, the photometry unit 58 selects the second subject as the AE target. If the multiple subjects detected by the subject detection unit 56 do not include the second subject, the photometry unit 58 selects the first subject selected as the AF target by the distance measurement unit 57 as the AE target.

[0051] The photometry unit 58 calculates a photometric value for exposure adjustment based on the image data PD output from the image sensor 20. In this embodiment, the photometry unit 58 calculates a photometric value for the entire image data PD (hereinafter referred to as a full-screen photometric value) and a photometric value for the subject that is the AE target (hereinafter referred to as a subject photometric value), and calculates the photometric value for exposure adjustment by weighting the subject photometric value more heavily than the full-screen photometric value.

[0052] The main control unit 50 performs a focusing process to bring the subject targeted for AF into focus by moving the focus lens 31 via the lens drive control unit 34 based on the distance measurement value detected by the distance measurement unit 57. In this manner, in this embodiment, focusing control is performed using the phase difference detection method.

[0053] Furthermore, the main control unit 50 performs adjustment processing to bring the brightness of the AE target into an appropriate range by changing at least one of the aperture value and the shutter speed based on the photometric value for exposure adjustment calculated by the photometric unit 58. For example, the main control unit 50 changes the aperture value by controlling the aperture 33 via the lens drive control unit 34. The main control unit 50 also changes the shutter speed by controlling the image sensor 20 via the imaging control unit 51. Note that the photometric unit 58 calculates the photometric value for exposure adjustment based on the full-screen photometric value and the subject photometric value weighted more heavily than the full-screen photometric value, so that the brightness of the AE target is prioritized within the appropriate range while also keeping the brightness of the entire screen within the appropriate range.

[0054] FIG. 4 conceptually illustrates an example of processing by the machine-learned model LM. Image data PD is input to the machine-learned model LM. The machine-learned model LM detects all subjects appearing in the image data PD and outputs subject detection information along with the type of the detected subject and a detection score. In the example shown in FIG. 4, two subjects, a "dog's face" and a "human face," are detected from the image data PD. In addition, in the example shown in FIG. 4, the "dog's face" is detected as the first subject S1, and the "human face" is detected as the second subject S2.

[0055] Fig. 5 conceptually illustrates an example of processing by the distance measuring unit 57. In the example shown in Fig. 5, the distance measuring unit 57 selects the first subject S1 that is closest to the currently set AF area α as the AF target, and detects a distance measurement value D that indicates the distance from the image sensor 20 to the first subject S1.

[0056] 6 conceptually illustrates an example of processing by the photometry unit 58. In the example shown in FIG. 6, the photometry unit 58 selects a second subject S2, which is not an AF subject, as an AE subject. After calculating a full-screen photometry value EVa and a subject photometry value EVs2, the photometry unit 58 calculates a photometry value EV for exposure adjustment using the following formula (1): ＥＶ = (1-w) x 2 ＥＶａ + w x 2 ＥＶｓ２ (1) where w is a weight and is a value that satisfies 0.5<w<1.

[0057] 7 conceptually illustrates an example of processing when a second subject S2 is selected as the AF target and the AE target. As shown in FIG. 7, when the second subject S2 is the subject closest to the currently set AF area α, the distance measuring unit 57 selects the second subject S2 as the AF target and detects a distance measurement value D that indicates the distance from the image sensor 20 to the second subject S2. In this case, the photometry unit 58 selects the second subject S2, which is the AF target, as the AE target and calculates a photometry value EV for exposure adjustment.

[0058] In this way, the distance measurement unit 57 selects the subject (first subject S1 or second subject S2) closest to the AF area α intended by the user as the AF target, but the photometry unit 58 preferentially selects the second subject S2 as the AE target. This is because if the subject selected as the AF target were directly used as the AE target, it would be difficult to adjust the exposure for the first subject S1, such as a dog, due to the wide variety of colors. For example, if the first subject S1 is a black dog, it is difficult to determine whether the low photometry value is due to the black color or the low brightness, and exposure adjustment may result in overexposure. In contrast, a human face, for example, has a limited variety of colors. Therefore, by using the second subject S2 as the AE target, the accuracy of exposure adjustment is improved.

[0059] The photographing method of the present disclosure includes a first focusing step of focusing on a first subject S1, a second focusing step of focusing on a second subject S2, and a selection step of selecting either the first focusing step or the second focusing step, and regardless of whether the first focusing step or the second focusing step is selected, exposure adjustment is performed based on the brightness of the second subject S2.

[0060] Fig. 8 is a flowchart showing an example of the photographing operation by the photographing device 10. Fig. 8 shows a case where the automatic subject detection mode is set in the AF mode.

[0061] First, the main control unit 50 determines whether the user has half-pressed the release button (step S10). If the release button has been half-pressed (step S10: YES), the main control unit 50 controls the imaging control unit 51 to cause the imaging sensor 20 to perform an imaging operation (step S11). The image data PD output from the imaging sensor 20 is input to the detection unit 55.

[0062] The subject detection unit 56 performs a detection process to detect all detectable subjects appearing in the image data PD using the machine-learned model LM (step S12).

[0063] The distance measurement unit 57 performs a selection process to select, as the AF target, the subject (first subject or second subject) closest to the currently set AF area α from among the multiple subjects detected by the subject detection unit 56 (step S13).The distance measurement unit 57 then detects a distance measurement value indicating the distance to the subject selected as the AF target (step S14).The main control unit 50 performs the above-mentioned focusing process based on the distance measurement value detected by the distance measurement unit 57 (step S15).

[0064] The photometry unit 58 selects the second subject as an AE target and calculates a photometric value for exposure adjustment based on the brightness of the second subject (step S16). The main control unit 50 performs the above-mentioned adjustment process based on the photometric value for exposure adjustment calculated by the photometry unit 58 (step S17).

[0065] The main controller 50 determines whether the user has fully pressed the release button (step S18). If the release button has not been fully pressed (i.e., if it is still half-pressed) (step S18: NO), the main controller 50 returns the process to step S11 and causes the image sensor 20 to perform an image capture operation again. The processes of steps S11 to S17 are repeatedly executed until the main controller 50 determines in step S18 that the release button has been fully pressed.

[0066] If the release button is fully pressed (step S18: YES), the main control unit 50 causes the imaging sensor 20 to perform an imaging operation (step S19). The image processing unit 52 performs image processing on the image data PD output from the imaging sensor 20 (step S20). The image recording unit 54 records the image data PD that has been image processed by the image processing unit 52 in the memory 42 as a recorded image PR (step S21).

[0067] In the above flowchart, step S11 corresponds to the "photographing process" according to the technology of the present disclosure. Step S12 corresponds to the "detection process" according to the technology of the present disclosure. Step S13 corresponds to the "determination process" according to the technology of the present disclosure. Steps S14 and S15 correspond to the "focusing process" according to the technology of the present disclosure. Steps S16 and S17 correspond to the "adjustment process" according to the technology of the present disclosure.

[0068] As described above, according to the imaging device 10 of this embodiment, the first subject and the second subject are detected from the image data, and when focusing on the first subject, exposure adjustment is performed based on the brightness of the second subject, thereby improving the accuracy of imaging adjustment.

[0069] Various modifications of the above embodiment will be described below.

[0070] In the above embodiment, the photometry unit 58 calculates the photometry value EV for exposure adjustment based on the full-screen photometry value EVa and the subject photometry value EVs2 of the second subject S2. Alternatively, the photometry unit 58 may calculate the photometry value EV for exposure adjustment using the following formula (2) based on the full-screen photometry value EVa, the subject photometry value EVs2 of the second subject S2, and the subject photometry value EVs1 of the first subject S1. ＥＶ = (1-w １ -w ２ ) x 2 ＥＶａ +w １ ×2 ＥＶｓ１ +w ２ ×2 ＥＶｓ２ ... (2) Here, w １ and w ２ is the weight, and w １ <w ２ Satisfy the relationship.

[0071] That is, in the adjustment step, the exposure may be adjusted based on the brightness of the first subject and the second subject, with the brightness of the second subject being given priority over the brightness of the first subject.

[0072] Furthermore, when adjusting the exposure based on the brightness of the first and second subjects, the exposure may be adjusted so that the difference between the brightness of the first and second subjects after exposure adjustment falls within a predetermined range. In this case, as shown in FIG. 9 , the photometer 58 calculates the full-frame photometric value EVa, the photometric value EVs2 of the second subject S2, and the photometric value EVs1 of the first subject S1, and calculates the photometric value EV for exposure adjustment based on the above equation (2). The photometer 58 also calculates the difference value ΔEV between the photometric value EVs2 and the photometric value EVs1. For example, the difference value ΔEV is the absolute value of the difference between the photometric value EVs2 and the photometric value EVs1.

[0073] 10 shows an adjustment process according to a modified example. In this modified example, the main controller 50 determines whether the photometric value EV calculated by the photometry unit 58 is within an appropriate range (step S170). If the photometric value EV is within the appropriate range (step S170: YES), the main controller 50 determines whether the difference value ΔEV is within a predetermined range (step S171).

[0074] If the photometric value EV is not within the appropriate range (step S170: NO), or if the difference value ΔEV is not within the predetermined range (step S171: NO), the main control unit 50 changes the exposure value by changing at least one of the aperture value and the shutter speed (step S172). If the difference value ΔEV is within the predetermined range (step S171: YES), the main control unit 50 ends the process.

[0075] In the above embodiment, exposure adjustment is performed based on the brightness of the second subject. However, instead of or in addition to this, it may be determined whether the shooting environment is a specific shooting environment based on the brightness of the second subject. "Determining whether the shooting environment is a specific shooting environment" includes indirectly determining whether the shooting environment is a specific environment based on subject recognition, the full-screen photometric value EVa, the subject photometric value EVs, etc. For example, it may be determined whether the shooting environment is backlit based on the brightness of the second subject.

[0076] 11 shows the functional configuration of the processor 40 according to a modified example. This modified example differs from the above embodiment in that the detection unit 55 is provided with a backlight determination unit 59 in addition to the subject detection unit 56, distance measurement unit 57, and photometry unit 58.

[0077] 12 is a flowchart showing an example of a photographing operation by the photographing device 10 according to a modified example. The photographing operation of this modified example differs from the photographing operation of the above embodiment in that a determination process (step S30) is performed by the backlight determination unit 59 after step S16. Step S30 corresponds to the "determination step" according to the technology of the present disclosure. The determination step is included in the adjustment step.

[0078] 13 shows an example of the determination process performed by the backlight determination unit 59. First, the backlight determination unit 59 determines (step S300) whether or not a second subject is present among the multiple subjects detected by the subject detection unit 56. If a second subject is not present (step S300: NO), the process ends without performing the determination process.

[0079] If a second subject is present (step S300: YES), the backlight determination unit 59 calculates the difference between the full-screen photometric value EVa calculated by the photometry unit 58 and the subject photometric value EVs2 (step S301).The backlight determination unit 59 then determines whether the calculated difference is equal to or greater than a certain value (step S302).If the difference is not equal to or greater than the certain value (step S302: NO), the backlight determination unit 59 ends the process.

[0080] If the difference is equal to or greater than a certain value (step S302: YES), the backlight determination unit 59 determines that the shooting environment is backlit (step S303), and corrects the exposure adjustment photometric value EV by increasing the weight w in the above equation (1) (step S304).

[0081] FIG. 14 shows an example of adjustment processing when backlight determination is performed. This modification differs from the adjustment processing shown in FIG. 10 in that step S173 is added between step S170 and step S171. In this modification, if the photometric value EV is within the appropriate range (step S170: YES), the main controller 50 determines whether the backlight determination unit 59 has determined that backlight exists (step S173). If the backlight determination unit 59 has not determined that backlight exists (step S173: NO), the main controller 50 determines whether the difference value ΔEV is within a predetermined range (step S171). If the backlight determination unit 59 has determined that backlight exists (step S173: YES), the main controller 50 ends the processing.

[0082] In other words, in this modified example, when the shooting environment is backlit, exposure adjustment based on the brightness of the second subject is given priority over keeping the difference between the brightness of the first subject and the brightness of the second subject within a specified range.

[0083] By making a backlight determination as described above and adjusting the exposure by increasing the weighting of the second subject when the subject is backlit, it is possible to prevent the second subject from becoming overexposed or underexposed.

[0084] In the above embodiment, the subject detection unit 56 detects one second subject in addition to the first subject. However, if multiple second subjects are detected, the photometering unit 58 may select the brightest or darkest second subject as the AE subject based on the brightness of each second subject. Furthermore, if multiple second subjects are detected, the photometering unit 58 may select the AE subject based on the size of the second subject. Furthermore, the photometering unit 58 may select multiple second subjects as the AE subject and calculate the photometric value EV for exposure adjustment by using a weighted average of the photometric values ​​of the multiple second subjects as the subject photometric value EVs2 described above. For example, the photometric values ​​of the second subjects may be weighted so that the weight increases in ascending order of the photometric value of the first subject, which is the AF subject.

[0085] In the above embodiment, exposure adjustment is performed based on the brightness of the second subject, but the technology disclosed herein is not limited to the brightness of the second subject and can be applied to a photographing device that performs photographing adjustment during the photographing process based on the state of the second subject.

[0086] For example, the technology disclosed herein can be applied to a photographing device equipped with a so-called film simulation function that determines the photographic scene and adjusts the color tone of the image data PD based on the determined photographic scene. In this case, the detection unit 55 determines the photographic scene by analyzing the image data PD. Specifically, the detection unit 55 determines the photographic scene (landscape, portrait, indoors, night view, etc.) using the presence of a second subject as one of the conditions. The image processing unit 52 changes the color tone of the image data PD according to the photographic scene determined by the detection unit 55. Changing the color tone refers to changing the gradation, contrast, saturation, etc.

[0087] The technology disclosed herein can also be applied to adjusting white balance, dynamic range, etc. By adjusting white balance, dynamic range, etc. for a second subject that is different from a first subject selected as an AF target, the adjustment accuracy is improved.

[0088] [Other Modifications] In the above embodiment, the subject detection unit 56 performs the detection process using the machine-learned model LM, but the detection process may be performed by image analysis using an algorithm, without being limited to the machine-learned model LM.

[0089] In the above embodiment, focus control is performed by moving the focus lens 31, but this is not limited to this, and focus control may also be performed by changing the thickness of the focus lens 31, moving the image sensor 20, etc.

[0090] The technology of the present disclosure is not limited to digital cameras, but can also be applied to electronic devices such as smartphones and tablet terminals that have a photographing function.

[0091] In the above embodiment, the hardware structure of the control unit, with processor 40 being an example, can use the various processors listed below. The various processors listed above include a CPU, which is a general-purpose processor that functions by executing software (programs), as well as a processor such as an FPGA, whose circuit configuration can be changed after manufacture. FPGAs include dedicated electrical circuits, such as PLDs or ASICs, which are processors with a circuit configuration designed specifically to execute specific processes.

[0092] The control unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple control units may be configured with a single processor.

[0093] There are several possible examples of configuring multiple control units with a single processor. A first example is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple control units. A second example is a form in which a processor is used to realize the functions of an entire system including multiple control units with a single IC chip, as typified by a system on chip (SOC). In this way, the control unit can be configured as a hardware structure using one or more of the various processors described above.

[0094] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0095] Furthermore, the technology of the present disclosure extends to a computer-readable storage medium that non-transitory stores a program in addition to the program.

[0096] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0097] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. a photographing step for generating image data by photographing through a photographing lens; a detection step of detecting a first object and a second object from the image data; a first focusing step of focusing on the first object; an adjustment step of adjusting the photographing in the photographing step based on a state of the second subject; A method of shooting including.

2. The state is brightness, In the adjusting step, the exposure is adjusted. The photographing method according to claim 1 .

3. the adjusting step includes a determining step of determining whether or not a photographing environment is a specific photographing environment based on the brightness of the second subject; In the adjustment step, exposure is adjusted based on the result of the determination step. The photographing method according to claim 2 .

4. In the adjustment step, the exposure is adjusted by giving priority to the brightness of the second object rather than the brightness of the first object. The photographing method according to claim 2 .

5. In the adjustment step, exposure is adjusted so that a difference between the brightness of the first subject and the brightness of the second subject after the exposure adjustment falls within a predetermined range. The photographing method according to claim 2 .

6. In the adjustment step, a color tone of the image data is adjusted. The photographing method according to claim 1 .

7. In the detection step, the first object and the second object are detected using a machine learning model. The photographing method according to claim 1 .

8. The first subject and the second subject are different types of subjects. The photographing method according to claim 1 .

9. The second subject is a human being and the first subject is a non-human subject. The photographing method according to claim 8.

10. a second focusing step of focusing on the second object; a selection step of selecting the first focusing step or the second focusing step; Further comprising: In the adjustment step, regardless of whether the first focusing step or the second focusing step is selected, the photographing adjustment is performed based on a state of the second subject. The photographing method according to any one of claims 1 to 9.

11. An imaging device including a processor, The processor, A photographing process for generating image data by photographing through a photographing lens; a detection process for detecting a first object and a second object from the image data; a first focusing process for focusing the photographing lens on the first object; an adjustment process for adjusting the photographing process based on a state of the second subject; An imaging device including:

12. A photographing process for generating image data by photographing through a photographing lens; a detection process for detecting a first object and a second object from the image data; a first focusing process for focusing on the first object; an adjustment process for adjusting the photographing process based on a state of the second subject; A program that causes a computer to execute the following.